WO2007049378A1 - 映像識別装置 - Google Patents
映像識別装置 Download PDFInfo
- Publication number
- WO2007049378A1 WO2007049378A1 PCT/JP2006/312144 JP2006312144W WO2007049378A1 WO 2007049378 A1 WO2007049378 A1 WO 2007049378A1 JP 2006312144 W JP2006312144 W JP 2006312144W WO 2007049378 A1 WO2007049378 A1 WO 2007049378A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- cut point
- distance
- shot
- frame
- calculating
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/147—Scene change detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/254—Analysis of motion involving subtraction of images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Definitions
- the present invention relates to a video identification device for identifying video in an important section from a video signal.
- Non-Patent Document 2 the difference between frames is evaluated using an evaluation function, and the amount of difference between frames is set to a threshold set by learning other video content. There is a method to detect the change point of the frame as a cut point when it becomes larger.
- Non-Patent Document 1 Nikkei Electronics No. 892 2005. 1.31 p. 51
- Non-patent document 2 “Cut point detection from MPEG data using partial duplication”, IEICE Transactions Vol. J81—D2 No. 7, (July 1998), Yasuyuki Nakajima, Kiyono Urahara, Yoneyama Ikuo, published by The Institute of Electronics, Information and Communication Engineers, pages 1564-1575
- the conventional video identification device is configured as described above, when the brightness of an adjacent frame exceeds a fixed threshold, a change point of the frame is detected as a cut point. Even when such a video signal is input, there is a problem that the detection accuracy of the cut point may deteriorate because the threshold value is fixed.
- the present invention has been made to solve the above-described problems, and an object of the present invention is to provide a video identification device that can accurately detect a cut point when any video signal is input. To do.
- the video identification apparatus includes a feature calculating unit that calculates a distance between a feature amount extracted this time by the feature amount extracting unit and a feature amount force previously extracted by the feature amount extracting unit, and the distance
- the distance between the feature quantities calculated by the calculation means is provided with a threshold calculation means for obtaining a statistic of the distance between the feature quantities calculated by the calculation means, and calculating the threshold value for determining the cut point as well.
- the threshold value calculated by the threshold value calculation means are compared, and the cut point is discriminated from the comparison result.
- the distance calculation means for calculating the distance between the feature quantity extracted this time by the feature quantity extraction means and the feature quantity force feature quantity previously extracted by the feature quantity extraction means, and the distance calculation A threshold value calculating means for calculating a statistical amount of the distance between the feature amounts calculated by the means, and calculating a threshold value for determining the statistical force cut point, and the distance between the feature amounts calculated by the distance calculating means and the threshold value Since the threshold value calculated by the calculation means is compared and the cut point is discriminated from the comparison result, the cut point can be accurately detected when any video signal is input. .
- FIG. 1 is a block diagram showing a video identification device according to Embodiment 1 of the present invention.
- FIG. 2 is an explanatory diagram showing changes in luminance values and cut points.
- FIG. 3 is a flowchart showing the processing contents of the video identification apparatus according to Embodiment 1 of the present invention.
- FIG. 4 is a block diagram showing a video identification device according to Embodiment 2 of the present invention.
- FIG. 5 is a block diagram showing an important shot discriminating unit of a video discriminating apparatus according to Embodiment 2 of the present invention.
- FIG. 1 is a block diagram showing a video identification device according to Embodiment 1 of the present invention.
- a feature quantity extraction unit 1 represents the characteristics of a video frame from the video signal.
- a process for extracting feature quantities is performed.
- the feature quantity extraction unit 1 constitutes a feature quantity extraction means.
- the interframe distance calculation unit 2 uses a predetermined evaluation function, and the feature amount of the current frame extracted by the feature amount extraction unit 1 this time and the feature amount of the immediately preceding frame stored in the feature amount buffer 3 ( The feature amount extraction unit 1 compares the feature amount of the frame extracted last time) and calculates the distance (dissimilarity) between the feature amounts.
- interframe distance the distance between the feature quantities of the current frame and the immediately preceding frame.
- the feature amount buffer 3 stores the feature amount of the immediately preceding frame, and when the interframe distance calculation unit 2 calculates the interframe distance, in order to prepare for the next interframe distance calculation, the feature amount buffer 3 stores the immediately preceding frame.
- the feature quantity is updated to the feature quantity of the current frame extracted this time by the feature quantity extraction unit 1.
- the inter-frame distance calculation unit 2 and the feature amount buffer 3 constitute distance calculation means.
- the cut point determination data calculation unit 4 calculates the statistic of the interframe distance calculated by the interframe distance calculation unit 2, calculates the threshold for determining the cut point for the statistical power, and sets the threshold for the cut point determination. Performs processing to output to cut point determination data buffer 5.
- the cut point determination data buffer 5 is a memory for storing the cut point determination threshold value calculated by the cut point determination data calculation unit 4.
- the cut point determination data calculation unit 4 and the cut point determination data buffer 5 constitute threshold value calculation means.
- the cut point determination unit 6 compares the interframe distance calculated by the interframe distance calculation unit 2 with the cut point determination threshold value stored in the cut point determination data buffer 5, and determines the cut point from the comparison result.
- a process for determining the The cut point determination unit 6 constitutes cut point determination means.
- FIG. 3 is a flowchart showing the processing contents of the video identification apparatus according to Embodiment 1 of the present invention. It is
- the feature quantity extraction unit 1 extracts a feature quantity that represents the characteristics of the frame from the video signal (step ST1).
- Examples of the feature amount representing the feature of the frame include, in addition to the difference from the past frame, a color histogram, color arrangement information, texture information, motion information, and the like. Alternatively, a plurality of feature amounts may be used.
- the interframe distance calculation unit 2 extracts the feature amount of the immediately preceding frame from the feature amount buffer 3 (the feature amount extraction unit 1 previously extracted). Frame feature value).
- the inter-frame distance calculation unit 2 compares the feature quantity of the current frame with the feature quantity of the immediately preceding frame using a predetermined evaluation function, and is the distance (dissimilarity) between these feature quantities.
- the interframe distance is calculated (step ST2).
- the inter-frame distance calculation unit 2 calculates the inter-frame distance
- the inter-frame distance calculation unit 2 updates the content stored in the feature amount buffer 3 to the feature amount of the current frame.
- the cut point determination unit 6 compares the inter-frame distance with a cut-point determination threshold stored in the cut-point determination data buffer 5. (Step ST3).
- the cut point determination unit 6 determines that the cut point is a cut point and outputs a determination result indicating that (step ST4).
- step ST5 if the inter-frame distance is not larger than the threshold for cut point determination, it is determined that it is not a cut point, and a determination result indicating that is output (step ST5).
- the cut point determination unit 6 may determine the cut point in consideration of the force for determining the cut point, for example, the shot time, using the threshold for cut point determination.
- the determination result output from the cut point determination unit 6 is information indicating whether or not it is a cut point.
- the user or external program power S, video content editing, and summary playback It can be used when carrying out.
- the cut point determination data calculation unit 4 indicates that the determination result of the cut point determination unit 6 is a cut point. Is stored, the contents stored in the cut point determination data buffer 5 are initialized to a predetermined value (step ST6).
- the cut point determination unit 6 determines whether it is not a cut point. If the determination result of the cut point determination unit 6 indicates that it is not a cut point, the statistic of the interframe distance calculated by the interframe distance calculation unit 2 is obtained, and the statistic is used to determine the force point. And the stored contents of the cut point determination data buffer 5 are updated with the threshold value (step ST7).
- the threshold for cut point determination is calculated as follows.
- Actual video content consists of multiple shots, and shots that are unlikely to be a cut point immediately after the cut point that is the cut break are assumed to be followed by multiple frames. Conceivable.
- the distance between the n_1st frame of each shot and the nth frame is Dist.
- the n-th frame in the i-th sailboat is considered to be a cut point.
- the first frame of the i-th shot is the 0th frame.
- the above threshold is assumed to change adaptively, and Th.
- the cut point determination data calculation unit 4 determines the i th shot.
- the cut point determination data calculation unit 4 calculates the average distance avg (Dist) and the distance variance var (Dist)
- the average value avg (Dist) and the distance variance var (Dist) are calculated as follows: n 1 n l n
- the threshold Th is calculated.
- Th avg (Dist) + a-var (Dist) (1)
- ⁇ is a coefficient
- the average value avg (Dist) and the variance value var (Dist) are all in the i-th shot.
- the average value and variance value of the first frame n_ 1st frame not the average value and variance value of the next frame.
- the reason why the 0th frame is not used for calculating the average value or variance value but is used for calculating the average value or variance value from the first frame is that the distance Dist of the 0th frame and the previous
- the nth frame is not used for calculating the average value or the variance value, and the nth frame up to the n_l frame is used for calculating the average value or the variance value. This is because it is possible to immediately determine whether or not it is a cut point.
- An approximate value may be used.
- the coefficient may be changed according to the content content.
- the feature amount of the current frame extracted by the feature amount extraction unit 1 and the feature of the immediately preceding frame stored in the feature amount buffer 3 are recorded.
- Inter-frame distance calculation unit 2 that calculates the inter-frame distance, which is the distance between feature quantities, and calculates the inter-frame distance statistic calculated by the inter-frame distance calculation unit 2, and the statistical power is also used for cut point determination.
- a cut point determination data calculation unit 4 for calculating the threshold value of the frame, and the interframe distance calculated by the interframe distance calculation unit 2 and the cut point determination threshold value calculated by the cut point determination data calculation unit 4 Since the cut point is discriminated from the comparison result, the cut point can be accurately detected regardless of the input of any video signal.
- the first embodiment even when there is a motion in the shot, it is possible to discriminate the variation in the cut point and the motion in the shot by statistically analyzing the motion. Therefore, it is possible to set a threshold value for cut point determination, and as a result, it becomes possible to improve the detection accuracy of cut points as compared with the case where a conventional fixed threshold value is used. The reason is as follows.
- the change of the luminance value in the frame is used for detecting the cut point, and the threshold value for detecting the cut point is a fixed value.
- FIG. 2 is an explanatory diagram showing changes in luminance values in such a case.
- the versatility is enhanced by using the feature amount in addition to the simple luminance value difference. Furthermore, when the distance, which is the evaluation result by the evaluation function, is large, the point is set as a hot point.
- the threshold value adaptively, the threshold value is automatically increased for shots with a large change. If the shot is small, the threshold value is automatically reduced. Therefore, a significant improvement in cut point detection and versatility can be expected.
- the feature amount when extracting the feature amount, the feature amount may be extracted from the encoded data of the compressed image that is not from the video signal.
- the distance between frames that are at least two frames apart from adjacent frames may be calculated to speed up the calculation process.
- a process such as weighting a frame close to the current frame may be performed so as to cope with the temporal change of the fluctuation state in the shot. .
- FIG. 4 is a block diagram showing a video identification apparatus according to Embodiment 2 of the present invention.
- the important shot discriminating unit 7 When the important shot discriminating unit 7 indicates that the judgment result of the cut point judging unit 6 is a cut point, the cut point one before the cut point (the previous cut point is Processing for determining whether or not a shot starting from the determined cut point is an important shot To implement.
- the important shot discriminating unit 7 constitutes an important shot discriminating means.
- FIG. 5 is a block diagram showing the important shot discriminating unit 7 of the video discriminating apparatus according to Embodiment 2 of the present invention.
- the shot length calculating unit 11 shows that the judgment result of the cut point judging unit 6 is a cut.
- the time difference between the current frame time and the shot start point time stored in the shot start point buffer 12 is obtained, and the time difference is used as the shot length to determine the important shot determination unit. Execute the process to output to 13.
- the shot start point buffer 12 is a memory that stores the time of the shot start point.
- the important shot determination unit 13 starts from the cut point immediately before the cut point determined by the cut point determination unit 6 Is determined to be an important shot, and the next shot after the shot starting from the previous cut point is the important shot, or the shot starting from the previous cut point and the next shot are the important shots. Then, a process of outputting the determination result is performed.
- the cut point determination unit 6 determines the inter-frame distance and the cut point determination stored in the cut point determination data buffer 5 as in the first embodiment. Compare thresholds for
- the cut point determination unit 6 determines that the cut point is a cut point, and outputs a determination result indicating that to the important shot determination unit 7. On the other hand, if the inter-frame distance is not larger than the threshold for cut point determination, it is determined that the distance is not a cut point, and a determination result indicating that is output to the important shot determination unit 7.
- the important shot determination unit 7 indicates that the determination result of the cut point determination unit 6 is a cut point. If it is indicated, it is determined whether or not the shot starting from the previous cut point is an important shot.
- the shot length calculation unit 11 of the important shot determination unit 7 can determine the shot point because the shot length of the shot can be obtained by the difference between the start time of the i-th shot and the start time of the (i + 1) -th shot
- the time difference between the current frame time 1J and the shot start point time stored in the shot start point buffer 12 is obtained. Therefore, the time difference is output to the important shot determination unit 13 as the shot length.
- the shot length calculation unit 11 updates the stored contents of the shot start point buffer 12 to the current frame time.
- the important shot determination unit 13 of the important shot determination unit 7 compares the shot length with a preset threshold value.
- the important shot determination unit 13 determines that the shot starting from the cut point immediately before the cut point determined by the cut point determination unit 6 is the important shot. Is determined, and the determination result is output.
- the force that the important shot determination unit 13 determines that the shot starting from the cut point one before the cut point is an important shot is the next shot after the shot starting from the previous cut point.
- the shot may be determined to be an important shot, or both the shot starting from the previous cut point and the next shot may be determined to be important shots.
- the cut point immediately before that cut point is set as the starting point. Since it is configured to determine whether or not a shot to be performed is an important shot, for example, a user can perform a very complicated process such as a large number of video processing methods and audio processing methods without increasing the calculation load. There is an effect that an important shot can be easily grasped.
- the second embodiment is based on an important narration and a long shot length in the dialogue part, particularly when the conversation is the main content.
- the calculation load is very small, and important shots can be judged even on devices with low calculation capabilities.
- the speed may be increased by using a frame far away from the frame that contacts P. Even in this case, the start time of the important shot to be output is slightly shifted from the original start time force of the important shot.
- the video identification device is used when identifying video in an important section from a video signal, and is suitable for video identification with excellent cut point detection accuracy. Yes.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Television Signal Processing For Recording (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/991,606 US8014606B2 (en) | 2005-10-25 | 2006-06-16 | Image discrimination apparatus |
CN200680039236XA CN101292267B (zh) | 2005-10-25 | 2006-06-16 | 影像识别装置 |
JP2007542240A JP4620126B2 (ja) | 2005-10-25 | 2006-06-16 | 映像識別装置 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2005-310157 | 2005-10-25 | ||
JP2005310157 | 2005-10-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2007049378A1 true WO2007049378A1 (ja) | 2007-05-03 |
Family
ID=37967500
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2006/312144 WO2007049378A1 (ja) | 2005-10-25 | 2006-06-16 | 映像識別装置 |
Country Status (5)
Country | Link |
---|---|
US (1) | US8014606B2 (ja) |
JP (1) | JP4620126B2 (ja) |
KR (1) | KR100963701B1 (ja) |
CN (1) | CN101292267B (ja) |
WO (1) | WO2007049378A1 (ja) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102802088B (zh) * | 2012-08-29 | 2015-04-15 | 上海天跃科技股份有限公司 | 一种基于实时传输协议的数据传输方法 |
CN106303498B (zh) * | 2015-05-30 | 2018-10-16 | 北京智谷睿拓技术服务有限公司 | 视频显示控制方法和装置、显示设备 |
CN106303315B (zh) | 2015-05-30 | 2019-08-16 | 北京智谷睿拓技术服务有限公司 | 视频显示控制方法和装置、显示设备 |
CN106303499B (zh) * | 2015-05-30 | 2018-10-16 | 北京智谷睿拓技术服务有限公司 | 视频显示控制方法和装置、显示设备 |
CN113777616B (zh) * | 2021-07-27 | 2024-06-18 | 武汉市异方体科技有限公司 | 一种运动车辆测距方法 |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2000285242A (ja) * | 1999-01-29 | 2000-10-13 | Sony Corp | 信号処理方法及び映像音声処理装置 |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5708767A (en) | 1995-02-03 | 1998-01-13 | The Trustees Of Princeton University | Method and apparatus for video browsing based on content and structure |
JP3823333B2 (ja) * | 1995-02-21 | 2006-09-20 | 株式会社日立製作所 | 動画像の変化点検出方法、動画像の変化点検出装置、動画像の変化点検出システム |
US6393054B1 (en) * | 1998-04-20 | 2002-05-21 | Hewlett-Packard Company | System and method for automatically detecting shot boundary and key frame from a compressed video data |
US6928233B1 (en) * | 1999-01-29 | 2005-08-09 | Sony Corporation | Signal processing method and video signal processor for detecting and analyzing a pattern reflecting the semantics of the content of a signal |
EP1129573A2 (en) * | 1999-07-06 | 2001-09-05 | Koninklijke Philips Electronics N.V. | Automatic extraction method of the structure of a video sequence |
US7038736B2 (en) * | 2000-09-21 | 2006-05-02 | Canon Kabushiki Kaisha | Moving image processing apparatus and method, and computer readable memory |
JP2002290959A (ja) * | 2001-03-23 | 2002-10-04 | Fujitsu General Ltd | 侵入者検出方法および装置 |
JP4082664B2 (ja) | 2002-09-20 | 2008-04-30 | Kddi株式会社 | 映像検索装置 |
WO2005050986A1 (ja) * | 2003-11-19 | 2005-06-02 | National Institute Of Information And Communications Technology, Independent Administrative Agency | 映像内容の提示方法及び装置 |
-
2006
- 2006-06-16 WO PCT/JP2006/312144 patent/WO2007049378A1/ja active Application Filing
- 2006-06-16 JP JP2007542240A patent/JP4620126B2/ja not_active Expired - Fee Related
- 2006-06-16 KR KR1020087009779A patent/KR100963701B1/ko active IP Right Grant
- 2006-06-16 US US11/991,606 patent/US8014606B2/en not_active Expired - Fee Related
- 2006-06-16 CN CN200680039236XA patent/CN101292267B/zh not_active Expired - Fee Related
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2000285242A (ja) * | 1999-01-29 | 2000-10-13 | Sony Corp | 信号処理方法及び映像音声処理装置 |
Also Published As
Publication number | Publication date |
---|---|
JP4620126B2 (ja) | 2011-01-26 |
US20090116751A1 (en) | 2009-05-07 |
JPWO2007049378A1 (ja) | 2009-04-30 |
KR20080055941A (ko) | 2008-06-19 |
KR100963701B1 (ko) | 2010-06-14 |
CN101292267A (zh) | 2008-10-22 |
US8014606B2 (en) | 2011-09-06 |
CN101292267B (zh) | 2012-08-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4699476B2 (ja) | 映像要約装置 | |
US7551234B2 (en) | Method and apparatus for estimating shot boundaries in a digital video sequence | |
Dirfaux | Key frame selection to represent a video | |
US20070146549A1 (en) | Apparatus for automatically generating video highlights and method thereof | |
EP1914994A1 (en) | Detection of gradual transitions in video sequences | |
US20060078292A1 (en) | Apparatus and method for embedding content information in a video bit stream | |
US20050123052A1 (en) | Apparatus and method for detection of scene changes in motion video | |
WO2007049378A1 (ja) | 映像識別装置 | |
JP2009147911A (ja) | 映像データ圧縮前処理方法およびこれを用いた映像データ圧縮方法と映像データ圧縮システム | |
WO2003098549A1 (en) | Scene change detector algorithm in image sequence | |
CN102314591A (zh) | 静止前景物体的检测方法和设备 | |
CN108830146A (zh) | 一种基于滑动窗口的非压缩域镜头边界检测方法 | |
KR101667011B1 (ko) | 입체 영상의 장면 전환 검출 장치 및 방법 | |
JPWO2010084738A1 (ja) | 照合加重情報抽出装置 | |
WO2006009035A1 (ja) | 信号検出方法,信号検出システム,信号検出処理プログラム及びそのプログラムを記録した記録媒体 | |
JP4225503B2 (ja) | 動画像のカット点検出装置 | |
JPH10224741A (ja) | 動画像のカット点検出方法 | |
JP4036321B2 (ja) | 映像の検索装置および検索プログラム | |
JP4924423B2 (ja) | 特徴量の予測誤差に基づいて動画像のカット点を検出する装置 | |
CN111859019A (zh) | 获取页面切换响应时间的方法及相关设备 | |
JP2004333279A (ja) | 目標運動判別装置および方法 | |
CN111191484A (zh) | 视频图像中人物说话的识别方法和装置 | |
WO2022049704A1 (ja) | 情報処理システム、情報処理方法、及びコンピュータプログラム | |
CN116343097A (zh) | 一种视频封面确定方法、装置、设备及介质 | |
JP4261603B2 (ja) | 動画像のカット点検出装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200680039236.X Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2007542240 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 11991606 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020087009779 Country of ref document: KR |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 06766832 Country of ref document: EP Kind code of ref document: A1 |