CN100545856C - Video content analysis system - Google Patents

Video content analysis system Download PDF

Info

Publication number
CN100545856C
CN100545856C CN 200610140834 CN200610140834A CN100545856C CN 100545856 C CN100545856 C CN 100545856C CN 200610140834 CN200610140834 CN 200610140834 CN 200610140834 A CN200610140834 A CN 200610140834A CN 100545856 C CN100545856 C CN 100545856C
Authority
CN
China
Prior art keywords
video
module
video content
content analysis
information
Prior art date
Application number
CN 200610140834
Other languages
Chinese (zh)
Other versions
CN101021904A (en
Inventor
景 习
磊 苏
鲍东山
Original Assignee
北京新岸线网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京新岸线网络技术有限公司 filed Critical 北京新岸线网络技术有限公司
Priority to CN 200610140834 priority Critical patent/CN100545856C/en
Publication of CN101021904A publication Critical patent/CN101021904A/en
Application granted granted Critical
Publication of CN100545856C publication Critical patent/CN100545856C/en

Links

Abstract

一种视频内容分析系统,属于视频分析和检索技术领域,是一个自动分析提取视频内容信息的系统。 A video content analysis systems, video analysis and retrieval belonging to the technical field of automatic analysis is a video content information extraction system. 系统框架包括:输入接口、任务调度模块、视频流获取模块、视频内容分析模块(核心模块,结构如摘要附图所示)、视频分析结果审核模块、输出接口和用户界面。 Framework system comprising: an input interface, task scheduling module, a video stream acquisition module, the video content analysis module (core module, the structure as shown in the drawings Abstract), video analysis results of an examination module, an output interface and a user interface. 系统流程是:系统接收来自网络或本机的视频分析命令,由任务调度模块决定任务的执行顺序,经过视频流获取模块解码要分析的视频,之后视频内容分析模块从视频中分析提取出镜头信息、镜头关键帧信息、场景信息、场景关键帧信息、关键帧图像信息以及人脸信息,分析结果根据需求进行审核,最后以XML文件形式存储,通过输出接口上传到视频信息数据库中。 System processes are: The system receives a video analyzes the command from the network, or the machine, the execution order of tasks is determined by the task scheduling module, acquires the video block decoding to be analyzed through the video stream after the video content analysis module extracts lens information from the video , shot key frame information, scene information, the scene key frame information, key frame image information and face information, review the results of the analysis on demand, and finally stored in an XML file, uploaded via the interface to output video information in the database. 系统用于支持基于内容的视频检索服务。 The system used to support content-based video retrieval service.

Description

视频内容分析系统 Video Content Analysis System

技术领域 FIELD

本发明属于视频分析和检索技术领域,具体涉及一个视频内容分析系统。 The present invention is in video retrieval and analysis technologies, and particularly relates to a video content analysis system.

背景技术 Background technique

在这个知识爆炸的时代,随着Internet的广泛使用,数据库系统存储能力的不断扩大,人们面对的视频数据越来越多,视频信息应用亦越来越广泛,视频检索的需求越来越迫切。 In this era of knowledge explosion, with the widespread use of the Internet, the database system storage capacity continues to expand, video data people are faced with more and more video information is also applied more widely, more and more urgent demand for video retrieval .

传统的视频检索系统包含一个视频信息数据库系统,存储视频的相关信息,这些视频信息往往仅包含人工输入的视频元数据信息,信息形式单一、信息量小,难以满足用户的检索需求。 Conventional video information retrieval system includes a video database system, the stored video, the video information is often only contain video metadata information manually entered, the information in the form of a single, small amount of information, it is difficult to meet the needs of the user to retrieve.

为了改变视频数据检索的这种状况,必须将无序的视频数据有序化,从而建立基于内容的视频检索工具,让用户能随时检索到想要的视频数据,让视频能自动地适应环境,可以互动式操作,快速地检索, 并在网上迅速、可靠地传送。 To change this situation retrieval of video data, video data must be disorderly orderly, so as to establish content-based video retrieval tool, allowing users to retrieve video data at any time you want, so that the video can automatically adapt to the environment, interactive operations can quickly retrieve and quickly, reliably transmitted on the internet.

对于视频数据来说,所谓无序,即只存储了一连串线性排列时域信号——图片序列,但并未提供更深层的结构,类似一篇文章的关键词、大纲或主题等内容。 For video data, the so-called disorderly, that only stores a string arrangement of linear time-domain signal - sequence of pictures, but did not provide a deeper structure, similar to an article such as keywords, topics or content outline. 对于无结构的数字视频流,必须首先进行视频分割,将其划分为基本组成单元,再从中提取出视频的结构信息和高层语义信息,即必须对视频进行基于内容的分析,才能够实现基于内容的视频数据检索应用。 For unstructured digital video stream, must first video segmentation, which is divided into the basic unit, and then extracts the structure information and high-level semantic information about the video, i.e. to be content-based analysis of video can be achieved based on the content the video data retrieval application.

目前,市场上还没有一套完整的、自动的视频内容分析系统,能够完成包括分析提取视频镜头、场景、镜头关键帧、场景关键帧、关键帧图像信息和人脸信息等功能。 Currently, the market does not have a complete, automated video content analysis system that includes a complete analysis extract video camera, scene, shot key frame, scene key frame, key frame information and face image information and other functions.

发明内容 SUMMARY

针对目前不存在一套视频内容分析系统的情况,本发明的目的在于设计一种包括接收用户视频分析指令、视频解码、视频内容分析、 视频内容信息审核和视频内容信息上传到视频信息数据库等一系列流程的视频内容分析系统;系统能从视频中提取出包括镜头、场景、 镜头关键帧、场景关键帧、关键帧图像信息和人脸信息等丰富的视频内容信息;系统以全自动方式从视频中提取出丰富的视频内容信息, 可为基于内容的视频检索提供支持。 For the case of a video content analysis systems currently do not exist, the object of the present invention is to design a video analysis comprising receiving a user command, the video decoding, video content analysis, and review the video content information upload video content information to the video information of a database series process video content analysis system; system extracts from the video key frame includes a lens, scene, shot, content-rich video information of the scene key frame, key frame image information and the facial information; automatic manner from the video system extracted a wealth of video content information, provides support for content-based video retrieval.

所述的视频内容分析系统主要包括七个部分:输入接口、任务调度模块、视频流获取模块、视频内容分析模块、视频内容分析结果审核模块、输出接口和用户界面。 The video content analysis system mainly consists of seven parts: an input interface, task scheduling module, a video stream acquisition module, the video content analysis module, the video content analysis results of an examination module, an output interface and a user interface. 系统结构图如图l所示。 The system configuration diagram shown in FIG. L.

输入接口用于接收视频内容分析命令,系统包含两种命令接收方式:从网络接收视频分析命令或者接受本机用户分析视频文件的命 An input interface for receiving video content analysis command, the command receiving system comprising two ways: receiving a command from a network or video analysis unit accept the user commands the analysis of the video file

任务调度模块用于根据视频内容分析任务的优先级,调度分配任务的执行次序。 Means for performing scheduling according to the priority order, the task scheduling assignment of the video content analysis tasks.

视频流获取模块,用于依据一定解码规则从外部视频源获取视频流。 Video stream obtaining means for obtaining a decoding rule based on certain video stream from an external video source.

视频内容分析模块,是本系统的核心功能模块,如图2所示,用 Video content analysis module, the module is the core function of the system, shown in Figure 2, with

于进行视频内容分析,其中包括六个子模块:镜头分割子模块、镜头关键帧提取子模块、场景分割子模块、场景关键帧提取子模块、关键帧图像分析子模块和人脸分析子模块。 In video content analysis, which includes six sub-modules: shot segmentation sub-module, shot key frame extraction sub-module, scene segmentation sub-module, the scene key frame extraction sub-module, sub-module key frame image analysis and facial analysis sub-module.

镜头分割子模块,用于将视频按内容分割为一系列基本的单元一一镜头;镜头关键帧提取子模块,用于在镜头分割完成后,从镜头中提取能够代表镜头主要内容的视频帧;场景分割子模块,用于在镜头分割和镜头关键帧提取之后,将镜头组合成有语义意义的视频场景; 场景关键帧提取子模块,用于提取场景中具有代表性的帧来表示场景;关键帧图像分析模块,用于从镜头关键帧和场景关键帧中提取帧图像底层信息,包括图像的纹理、颜色和边缘;人脸分析子模块,用于从视频中提取视频高层语义信息——人脸信息。 Shot segmentation sub-module, for dividing the video content by a series of basic units of eleven lenses; shot key frame extraction sub-module, for dividing the video frame after completion of the lens, the lens is extracted from the main content of the lens can represent; scene segmentation sub-module, for subsequent segmentation and shot key frame extraction lens, a combined lens into a video scene have semantic meaning; scene sub key frame extraction module for extracting representative frames to represent the scene scene; key frame image analysis module for extracting from a shot key frame and key frame scenarios in the bottom frame image information, comprising a texture image, color and edge; face analysis sub-module, for extracting the video information from the high-level semantic video - human face information.

视频内容分析结果审核模块,用于审核视频内容分析模块产生的视频内容分析结果是否有误差,并能够对产生误差的地方进行人工修改。 Video content analysis results of an examination means for examining the results of video content analysis module generates video content analysis whether there is an error, and be able to manually modify the local errors.

输出接口,用于将视频内容分析结果上传到视频信息数据库。 Output interface for video content analysis results are uploaded to the video information database. 用户界面,主要包含两个界面:视频分析过程界面和视频分析结果审核界面。 User interface consists of two main interface: video analysis and video analysis of the results of the audit interface interface. 视频分析过程界面用于在视频内容分析过程中显示分析进度、分析状态和分析结果;视频分析结果审核界面用于修改视频内容分析的结果。 Video analysis interface for displaying video content analysis process progress analysis, state analysis, and the analysis result; a video interface for modifying the analysis results of an examination result of the video content analysis.

系统业务流程:系统通过输入接口接收来自网络的视频融合分析服务器指令或来自用户界面输入的本机视频内容分析请求,任务调度模块根据任务的优先级决定视频分析任务的执行顺序,开始启动视频分析服务,视频流获取模块解码相应的视频,将解码出的视频流发送给视频内容分析模块,视频内容分析模块对解码出的视频流进行包括镜头分割、镜头关键帧提取、场景分割、场景关键帧提取、人脸分析、 关键帧图像分析等一系列视频内容分析过程;之后,在用户审核模式下,视频内容分析结果审核模块会对视频内容分析结果进行审核,审核通过后,视频内容结构化信息将通过输出接口以XML形式上传到视频信息数据库,在非审核模式下,视频内容分析模块产生的视频内容 Business Process: The system receives a video from a network via the input interface fusion analysis server command or the machine from a user interface input video content analysis request, task scheduling module according to the execution order priority video analysis tasks task started video analysis service, a video stream to obtain the corresponding block decoding video, transmitting the decoded video stream to the video content analysis module, the video content analysis module to decode the video stream includes a shot segmentation, shot key frame extraction, scene segmentation, key scenes frame extraction, face analysis, image analysis and a series of key frame video content analysis process; Thereafter, the user audit mode, the video content analysis module will review the video content analysis result for review by the audit, the video content structure information the XML form to spread information database via video output interface, in the non-audit mode, video content analysis module generated video content

结构化信息将以XML形式直接上传到视频信息数据库。 XML structured information will be uploaded directly to the video information in the form of a database.

应该理解,本发明的前面的一般描述和随后的详细描述都是示范性和解释性的,目的是提供所要求的发明的进一步解释。 It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory object of the invention to provide further explanation required.

附图说明 BRIEF DESCRIPTION

图1是本发明的系统总体框架图 1 is a system overall framework of the present invention of FIG.

图2是本发明的视频内容分析子模块的结构图 FIG 2 is a configuration diagram of the video content analysis sub-module of the present invention.

图3是本发明的视频分析过程界面 FIG 3 is a video interface of the present invention, the analysis

图4是本发明的视频分析结果审核界面 FIG 4 is a video analysis result of the review interface of the present invention

图5是本发明的视频内容分析信息以XML文件保存的格式具体实施方式 FIG 5 is a video format of the present invention DETAILED DESCRIPTION analysis information stored in an XML file

如图1所示,本发明主要包含视频流获取模块、视频内容分析模块、视频内容分析结果审核模块、输入接口、输出接口、任务调度模块和用户界面。 As shown in FIG 1, the present invention mainly includes streaming video acquisition module, the video content analysis module, the audit module the video content analysis, input interface, output interface, user interface and task scheduling module.

系统通过输入接口接收视频分析命令,接收方式有两种:通过网络TCP连接从视频融合分析服务器接收视频分析命令和从本地获取用户的视频分析命令。 Through the input interface receives a video system analyzes the command, there are two receiving mode: Analysis coupled to receive video from the video server command fusion analysis and video analysis of the acquired user command from the local network via TCP. 其中视频融合分析服务器是视频分析的总体调度服务器,用于分发视频分析指令。 Wherein the video analysis server integration is a video analysis of the overall scheduling server for distributing video analysis instruction. 在命令中包含需要分析的视频文 Needs analysis contained in the command video files

件的各项信息,包括视频分析任务ID (本发明中称为PID)、优先级(包含高、普通和低三个级别)、任务基本信息(屏幕长度、屏幕宽度、帧率)、文件的基本信息(供视频内容分析界面显示)。 The member information, including video analysis task ID (the PID referred to as the present invention), the priority (including high, normal, and low levels), the basic task information (the length of the screen, the screen width, the frame rate), the file basic information (for video content analysis interface display).

任务调度模块将任务信息以链表形式存放,优先级高、到达时间早的任务将加入到任务链表的前面,反之,优先级低、到达时间晚的任务将放置在任务链表的后面。 Task scheduling module task information stored as a list, a high priority task arrival time is earlier will be added to the list of the previous task, conversely, low priority, the task arrival time later placed behind the task list.

视频流获取模块从外部视频源获取各种格式的视频流,包括MPEG1、 MPEG2、 MPEG4和H. 263等。 Video stream acquiring module acquires video stream in various formats from an external video source, including MPEG1, MPEG2, MPEG4, and H. 263 and the like. 采用目前常用的开源解码器,如Xvid、 ffmpeg、 ffdshow等进行解码,将整个视频流解码或者只解码视频的I帧。 Using open source decoder currently used, such as Xvid, ffmpeg, ffdshow like decoding, decoding the entire video stream or only the decoded I frame of video.

视频内容分析模块,用于对经过解码的视频流进行视频内容结构化分析,如图2所示,其中包括六个子模块: «镜头分割子模块 Video content analysis module for structuring video content decoded video stream analysis, shown in Figure 2, which includes six sub modules: «Shot Segmentation submodule

镜头是对视频流进行处理的最小物理单元,在同一组镜头中,视频帧的图像特征保持稳定。 Lens is a minimum physical unit of processing the video stream, in the same group lens, the image features of the video frame remain stable. 颜色是图像的重要底层特征,本发明采用基于颜色直方图的切割算法,用//(/,"表示帧/颜色直方图中对应颜色A的像素点总数。A的范围为[O, N], N是颜色离散值域区间的最大值。颜色直方图之间的距离采用直方图求交的方法度量,两幅图像f和尸之间的距离W/,/)计算公式如下: Color is an important feature of the underlying image, the present invention employs cut algorithm based on color histogram, a // (/ "represents the total number of pixels of a frame range / color corresponding to the color histogram .A point A is [O, N] , N is the maximum interval range of discrete color distance between the color histogram method using a histogram intersection metric, the distance W between the two images and dead f /, /) is calculated using the following formula:

|>in(/f(/,/^/f(/',W) |> In (/ f (/, / ^ / f (/ ', W)

具体实现时,釆用滑动窗口的方式,窗口默认大小为16,在每一窗中求出直方图距离变化最大的帧,距离为"_,同时计算出同一窗中直方图距离变化的平均值,距离为《,,设置阈值为T,当",",>7时,认为该点为镜头切割点,反之,认为当前窗口中无切割点。该方法能够有效地避免毛刺对镜头分割的影响。 In specific implementation, preclude the use of a sliding window mode, the default window size is 16, the histogram is obtained from the largest variation in each window frame, a distance "_, in the same window at the same time calculates the average change in the distance histogram , a distance "T ,, the threshold value is set, when" ",> 7, the point that the cutting point of the lens, on the contrary, that the current window without cutting point. this method can effectively avoid the influence of burrs on shot segmentation .

根据不同应用和不同视频的需要,滑动窗口大小和阈值可以进行调整。 According to different applications and different video sliding window size and the threshold value can be adjusted.

參镜头关键帧提取子模块 Reference shot key frame extraction submodule

聚类方法在人工智能、模式识别和语音识别等领域应用广泛,本发明采用基于聚类的关键帧提取算法。 Clustering method is widely used in artificial intelligence, pattern recognition and speech recognition applications, the present invention is based clustering algorithm for key frame extraction.

设某个镜头S,包含n个图像帧,可以表示为S,.-化,,…,iU,其中i^为首帧和&为尾帧。 A lens disposed S, includes n frame images, can be expressed as S, of .- ,, ..., iU, where i ^ & headed for the frame and the end frame. 相邻两帧之间的相似度定义为这相邻两帧颜色直方图的相似度,预定义一个阈值^控制聚类的密度。 Define the similarity between two predefined threshold value a ^ to control the density of adjacent clusters which two adjacent color histogram similarity.

计算当前帧&与现存某个聚类质心之间的距离,如果该值大于5,则该帧与该聚类之间距离较大,^不加入该聚类中。 Calculating a distance between a & existing cluster centroid and the current frame, if the value is greater than 5, then the frame with the larger distance between the cluster, the cluster without adding ^. 如果&,与现存所有聚类质心之间的距离均大于S ,《,"形成新的聚类,i^,为新聚类的质心;否则将该帧加入到与之相似度最大的聚类中,使该帧与这个聚类的质心距离最小,并如下相应调整该聚类质心: & If, with the distances between all cluster centroids are greater than existing S, "," form a new cluster, i ^, for the new cluster centroid; otherwise the frame is added to the cluster with the greatest similarity in the frame of this cluster centroid and the minimum distance, and adjust the cluster centroid as follows:

= cewfroc/ x +1) +1 /(F" +1) x《.m = Cewfroc / x +1) +1 / (F "+1) x" .m

其中,""的《、""的d和F。 Wherein, "" the "," "d and F. 分别是聚类原有质心、聚类更新后质 They are the original cluster centroid, after clustering updated quality

心和该聚类帧数。 Heart and the cluster number of frames.

通过上面方法将镜头S,所包含的n个图像帧,分别归类到不同聚 By the above method the lens S, n included in the image frames, are classified into different poly

类后,就可以选择关键帧。 After class, key frames can be selected. 从每个聚类中抽取离聚类质心最近的作为这个聚类的代表帧,所有聚类的代表帧就构成了镜头的关键帧。 Extracting the representative frame from the cluster centroid of the cluster as the latest from each cluster, all the clusters representative frame constitutes a key frame of the lens.

对聚类结果,算法加入了一些约束条件,规定任一聚类的总帧数不能小于镜头总帧数的10%,对质心相似的聚类进行合并。 Clustering results, adding some constraints algorithm, a clustering according to any predetermined number of frames not less than 10% of the total number of frames of the lens, heart confrontation similar clusters are merged.

算法中阈值^可以根据不同应用不同视频的需要,进行调整,以得到不同数量的镜头关键帧。 ^ Threshold algorithm can be applied according to different needs of different video adjustment to obtain a different amount of shot key frames.

*场景分割子模块 * Scene segmentation submodule

场景是视频所蕴含的高层抽象概念和语义的表达,对于故事片, 本系统采用基于镜头聚类的算法,即将视频镜头关键帧按聚类算法进行聚类,将时间上连续、关键帧属于同一聚类的视频镜头合并为场景, 完成故事片的场景分割;对于新闻视频,根据新闻视频所具有的特点——两个主持人之间往往包含一个完整的新闻片段,本发明采用基于主持人镜头的场景分割算法,首先调用人脸分析子模块对视频镜头关键帧进行人脸检测,将包含人脸的镜头定为候选主持人镜头,之后,根据主持人镜头持续时间(如大于5秒)和镜头视频帧颜色直方图距 Abstract high-level expression of the video scene is inherent and semantics for the feature film, the system uses a lens-based clustering algorithm, i.e. the video shot clustering keyframes by clustering algorithm, the temporally successive, belonging to the same poly keyframe class combined video shot scene, feature films complete scene cut; news videos, news video in accordance with the characteristics of - between two hosts often contain a complete news segment, the present invention employs a scene shot on the host segmentation algorithm, called first face analysis sub-module of the video shot key frames for face detection, face including the lens as the lens candidate host, after the host according to the duration of the lens (e.g., greater than 5 seconds) and a video camera frame color histogram distance

离变化的方差(如方差小于0.025)具有的特点,剔除一部分伪主持人镜头,最后调用人脸分析子模块对剩下的主持人镜头采用人脸识别的方法,提取出正确的主持人镜头,以两个连续出现的主持人镜头之间的镜头构成一个场景,完成新闻视频的场景分割。 Variance from the change (e.g., variance of less than 0.025) has characteristics of a part excluding the lens pseudo host, last call analysis submodule face recognition method of the remaining lens with the host, host extract the correct lens, constitute a scene shot in a shot between the host two consecutive complete news video scene segmentation. *场景关键帧提取子模块 * Scene key frame extraction sub-module

本发明釆用基于镜头关键帧聚类的场景关键帧提取算法,对属于场景的所有镜头关键帧进行聚类,聚类算法与之前所讲的镜头关键帧提取算法类似,提取出代表场景信息的场景关键帧。 The present invention preclude scene key frames with key frames based on the shot clustering extraction algorithm, all belonging to the scene shot key frames are clustered, the clustering algorithm previously spoken shot key frame extraction similar algorithms to extract a representative scene information scene key frame.

*关键帧图像分析子模块 * Key frame image analysis sub-module

本发明对关键帧图像提取颜色、纹理和形状三个特征参数,颜色特征的表示方法有颜色直方图、颜色相关图、颜色矩等方法;纹理特征的表示方法有共生矩阵、Tamura纹理特征、Gabor特征等方法;形状特征的表示方法有傅立叶描述子、小波轮廓描述法等。 The present invention is extraction of the key frame image color, texture and shape of the three characteristic parameters of the color feature representation has a color histogram, color correlogram, color moments or the like; texture feature representation are co-occurrence matrix texture features Tamura, Gabor It characterized the like; shape feature representation are Fourier descriptors, wavelet contour description method. 这些方法都是本领域的公知技术。 These methods are well known techniques in the art.

*人脸分析子模块 * Face analysis sub-module

该模块能够对视频帧、镜头关键帧和场景关键帧进行人脸检测定位和人脸识别。 The module can be positioned face detection and recognition of video frames, a scene shot key frames and key frames. 本发明的人脸检测定位算法采用基于Adaboost的方法,关于算法的详细描述请参考文献"Rapid Object Detection using a Boosted Cascade of Simple Features" [Paul Viola and Michael Jones, Computer Vision and Pattern Recognition, Vol. l,pp:511-518,2001 ]。 Face detection location algorithm according to the present invention employs a method based on Adaboost, a detailed description of the algorithm, please refer to document "Rapid Object Detection using a Boosted Cascade of Simple Features" [Paul Viola and Michael Jones, Computer Vision and Pattern Recognition, Vol. L , pp: 511-518,2001]. 人脸识别方法釆用Fisher脸算法,关于算法的详细描述请参考文献"Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection" [Belhumeur PN , Hespanha _J. P. , Kriegman DJ. IEEE Transactions on Pattern . Analysis and Machine Intelligence, Vol. 19, Issue 7, pp:711 - 720, 1997]。 Face recognition methods preclude the use of Fisher algorithm, a detailed description of the algorithm, please reference... "Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection" [Belhumeur PN, Hespanha _J P., Kriegman DJ IEEE Transactions on Pattern Analysis . and Machine Intelligence, Vol 19, Issue 7, pp: 711 - 720, 1997].

视频内容分析结果审核模块,用于审核视频内容分析结果是否存在误差。 Review the video content analysis module, configured to review the video content analysis result whether there is an error. 在系统运行时是否加载该模块由用户动态配置,当用户配置加载该模块时,视频内容分析模块分析产生的视频内容分析结果必须经过视频内容分析结果审核模块的审核,用户可以通过视频分析结果审核界面修改视频内容分析结果中有误差的地方,审核通过后视频分析结果才能上传给视频信息数据库;当用户配置不加载该模块时,视频内容分析结果可直接上传给视频信息数据库。 During system operation if the module is loaded dynamically configured by the user, when the user profile to load the module, the video content analysis module analyzes the video content analysis result must be reviewed video content analysis results of the audit module, the user can video analysis audit interface to modify the video content analysis result where there is an error, the analysis results of the audit to upload video to the video information database; when the user configure the module is not loaded, the video content analysis results may be uploaded directly to the video information database. 其中,视频信息数据库用于存储与视频相关的元数据信息、视频内容分析结果信息、音频信息和视频字幕信息。 Wherein the video information database for storing metadata information related to the video, the video content analysis result information, audio information and video caption information.

输出接口,用于保存和输出视频内容分析结果。 Output interface for saving and output video content analysis results. 在经过视频内容分析、视频内容分析结果审核后,本发明采用XML文件的形式保存视频内容分析结果,该XML文件命名为:PID_text.xml (PID代表该任务的ID),具体格式如图5,其内容如下: After analysis of the video content, the video content analysis after review, the present invention takes the form of XML files stored video content analysis result, the XML file is named: PID_text.xml (PID behalf of the task ID), the specific format shown in Figure 5, which reads as follows:

1) 所分析的视频文件的元数据信息(包括视频文件名、视频文件路径、视频文件类型、视频内容结构化信息入库时间等); 1) metadata information analyzed video files (including video file name, file path video, a video file type, a video content structure information storage time, etc.);

2) 视频镜头信息(包括镜头开始时间、结束时间、镜头关键帧数量);3) 视频镜头关键帧信息(包括关键帧所在的时间点、关键帧编号、关键帧图片名); 2) video camera information (including the shot start time, end time, the number of shot key frames); 3) video shot key frame information (including a point in time where the key frame, key frame number, key frame picture name);

4) 视频场景信息(包括场景开始时间、结束时间、所包含的镜头数量、场景关键帧的数量); 4) video scene information (scenes including start time, end time, the number of lenses included in the key frame number of the scene);

5) 视频场景关键帧信息(包括关键帧所在的时间点、关键帧编号、关键帧图片名); 5) Video scene key frame information (including a point in time where the key frame, key frame number, key frame picture name);

6) 关键帧图像信息(包括关键帧的颜色信息、纹理信息和形状信息); 6) key frame image information (key frame including color information and texture information and shape information);

7) 人脸信息(包括人脸所在帧的时间点、人脸编号、人脸对应的人物名字)。 7) the face information (including a point in time where the frame of the face, the face number, face the corresponding character name).

如果当前完成任务为从网络视频融合分析服务器发出的指令,则分析任务结束后,将任务结束状态信息通知视频融合分析服务器,并且将相应的视频内容分析结果XML文件上传至视频信息数据库。 If the current task is the integration of the analysis server command issued from the online video, the analysis after the end of the task, the task will end status information notification fusion analysis video server, and upload corresponding video content analysis results XML file to the video information database.

如果当前任务为本机用户发出的指令,则直接将结果保存到本机。 If the current task native instruction issued by the user, save the results directly to the machine.

本发明采用XML格式存储视频内容分析结果信息,实际应用中, 也可以采用其它方式。 The present invention employs XML format for storing video content analysis result information, practical applications, can also be used in other ways.

用户界面,主要包含两个界面:视频分析过程界面和视频分析结果审核界面。 User interface consists of two main interface: video analysis and video analysis of the results of the audit interface interface. 视频分析过程界面用于在视频内容分析过程中显示分析 Video analysis interface for displaying video content analysis process Analysis

进度、分析状态和分析结果,如图3;视频分析结果审核界面用于方 The progress, status, and analysis results, shown in Figure 3; the interface for video analysis result party audit

便用户可以通过界面修改视频内容分析的结果,如图4。 Users can then modify the results of the analysis of video content through the interface, shown in Figure 4.

视频分析过程界面,如图3,主要分为五个子块,视频内容分析信息LOG栏21,用于视频分析过程中不断更新显示产生的镜头分割点位置、场景分割点位置、人脸信息等;视频播放栏22,用于在视频分析过程中播放视频,播放速度与系统的分析速度相同,即系统当前播放帧与系统当前分析帧位置相同;视频信息栏23,用于显示当前分析视频的各项信息,包括视频文件名、文件格式、文件大小、视频持续时间长、视频位置等;系统状态栏24,用于显示当前系统与视频融合分析服务器的连接状态、己接收到的视频分析命令、已完成视频分析的视频文件名等;视频关键帧信息栏25,用于不断更新显示当前已分析出的视频关键帧图片。 Video analysis interface 3, is divided into five sub-blocks, the video content analysis information LOG column 21, a video analysis process for the lens to keep displaying the generated dividing point position, a scene division position, face information; video playback column 22, the video analysis process for video playback, the playback speed and the same analysis system speed, i.e., the system displays the current system and the current frame is the same frame position analysis; video information column 23 for displaying the respective current video analysis item information, including the video file name, file format, file size, video long duration, video location; system status bar 24 for displaying the current status of the connection system with the video integration analysis server has received the command to video analysis, completed video analysis of the video file name; video key frame information field 25, a continuously updated display of the currently analyzed video key frame picture.

视频结果审核界面,如图4,主要分为四个子块,视频文件栏31 以列表形式显示当前需要审核视频内容分析结果的视频文件名称;视频分析结果信息栏32以列表形式显示当前审核的视频内容分析结果;视频帧信息栏33用于显示所审核视频的视频帧,可以直接进行添加、删除和修改镜头和场景的操作;关键帧栏34用于显示当前审核的视频的关键帧图片;视频播放栏35用于播放当前审核视频,可以响应用户命令播放指定的视频镜头和场景;视频信息栏36用于显示当前分析视频的属性信息,包括视频文件名、文件格式、文件大小、 视频持续时间长、视频位置等。 Video interface audit results, shown in Figure 4, is divided into four sub-blocks, column 31 shows the current video file to review the video content analysis result of the video file name in a list form; video analysis result information field 32 displays the video in the current audit list form content analysis results; video frame information field 33 for displaying the video frame review of video, you can directly add, delete and modify operations shots and scenes; key frame bar 34 for displaying the key frame picture of the current review of the video; video playing field 35 for the current audit video player can play the specified response to user commands video shots and scenes; field 36 for displaying the video information of the current duration of the analysis of the video attribute information includes a video file name, file format, file size, video long, video location. 用户通过视频结果审核界面对视频分析结果进行审核和修改,修改之后的结果同样与XML文件形式保存到视频信息数据库或本地。 User to review and modify the video analysis results of an examination by the video interface, the same result after revision and save an XML file or database to the local video information.

Claims (23)

1.一种视频内容分析系统,其特征在于:该系统包括: 一个输入接口,用于接收视频内容分析指令; 一个任务调度模块,用于调度视频内容分析任务的执行顺序; 一个视频流获取模块,用于依据一定解码规则从外部视频源获取视频流; 一个视频内容分析模块,用于视频内容结构化分析,其中包括六个子模块:将视频按内容分割为一系列基本镜头的镜头分割子模块、在镜头分割完成后从镜头中提取能够代表镜头主要内容的视频帧的镜头关键帧提取子模块、在镜头分割和镜头关键帧提取之后将镜头组合成有语义意义的视频场景的场景分割子模块、提取场景中具有代表性的帧的场景关键帧提取子模块、从镜头关键帧和场景关键帧中提取帧图像底层信息的关键帧图像分析子模块、从视频中提取人脸信息的人脸分析子模块; 一个视频内容分析结果审核模块 A video content analysis system, characterized in that: the system comprising: an input interface for receiving video content analysis command; a scheduling module for scheduling the execution order of the video content analysis tasks; a video stream acquisition module , configured to obtain video streams from an external video source decoding according to certain rules; a video content analysis module for structural analysis of video content, which includes six sub modules: the video content is divided by a series of sub-dividing the basic camera lens module , after the shot segmentation completion extracted from the lens can represent lens main contents of the shot key video frames frame extraction sub-module, segmentation and shot key frame extraction lens lens combined into a semantic meaning video scene scene segmentation submodule analysis of the human face, the scene extracting the key frames in a scene representative frame extracting sub-module, extracts frame image underlying information analysis sub-module key frame image from the shot key frame and key frame of a scene, a face information extraction from video sub-module; a video content analysis results of the audit module ,用于审核视频内容分析模块产生的结果是否有误差,并能够对产生误差的地方进行人工修改; 一个输出接口,用于将视频内容分析结果上传到视频信息数据库; 用户界面,包括一个视频分析过程界面和一个视频分析结果审核界面; 与其它相关系统的接口,包括:与视频融合分析服务器的接口、与视频信息数据库的接口。 The results for auditing the video content analysis module generated if there is an error, and the local errors can be manually modified; an output interface for video uploaded to video content analysis result information database; user interface, including a video analysis process interface and a video interface audit results; interfaces to other related systems, including: the integration of video analysis server interfaces, database interfaces and video information.
2. 如权利要求1所述的视频内容分析系统,其特征在于:输入接口,既可以通过用户界面接收用户的分析本地视频命令,也可以通过系统带有的TCP/IP网络连接接收来自视频融合分析服务器的视频分析命令。 2. The video content analysis system according to claim 1, wherein: an input interface receiving a user's analysis of both local video command through the user interface, TCP / IP network may also be connected with the system by receiving video fusion video analysis server analyzes the command.
3. 如权利要求1所述的视频内容分析系统,其特征在于:视频流获取模块,其中,外部视频源可以是视频文件或视频输入设备。 Video content analysis system as claimed in claim 1, wherein: the video stream acquisition module, wherein the external video source may be a video file or video input device.
4. 如权利要求1所述的视频内容分析系统,其特征在于:视频流获取模块,支持包括MPEG1、 MPEG2、 MPEG4、 H. 263等视频流格式的解码。 4. The video content analysis system according to claim 1, wherein: the video stream acquisition module support including MPEG1, MPEG2, MPEG4, H. 263 decoding the video stream and other formats.
5. 如权利要求1所述的视频内容分析系统,其特征在于:视频流获取模块,解码时有两种形式:解码整个视频文件或者只解码视频文件中的I帧。 5. The video content analysis system according to claim 1, wherein: the video stream acquisition module, there are two forms of decoding: video decoding the entire file or only the decoded I frames in the video file.
6. 如权利要求1所述的视频内容分析系统,其特征在于:用户界面,其中的视频分析过程界面显示的状态信息包括与视频融合分析服务器的连接状态、已接收到的视频分析命令和视频内容分析过程的信息。 6. The video content analysis system according to claim 1, wherein: the status information of the user interface, wherein the interface display process of video analysis comprising analysis of the fusion connection status with the video server, that has received the command analyzes the video and video content analysis information process.
7. 如权利要求1所述的视频内容分析系统,其特征在于:用户界面,其中的视频分析过程界面能够实时的播放当前正在分析的视频帧,显示当前已分析出的镜头关键帧和场景关键帧图像, 同时还能够显示当前分割的镜头和场景的切分时间点。 7. The video content analysis system according to claim 1, wherein: the user interface, wherein the interface is capable of playing video in real-time analysis of the video frame currently being analyzed, the analyzed currently displayed shot key frames and key scenes frame image, but also able to display the current cut shots and scenes divided time points.
8. 如权利要求l所述的视频内容分析系统,其特征在于:视频内容分析模块,其中的六个子模块,在具体实施时,可以按用户需求加载所需的模块,可以全部加载,也可以只加载部分模块,完成全部功能或用户指定的部分功能。 8. The video content analysis system according to claim l, wherein: the video content analysis module, where the six sub-module, in the specific embodiment, the load module may be required according to user needs and to be fully loaded, can be loading only a part of the module, complete all or user-specified functional portion functions.
9. 如权利要求1所述的视频内容分析系统,其特征在于:视频内容分析模块,其中的镜头分割子模块,采用基于滑动窗和直方图距离的镜头分割算法。 9. The video content analysis system according to claim 1, wherein: the video content analysis module, wherein the module sub-shot segmentation, segmentation based on using sliding window and the lens distance histogram.
10. 如权利要求1所述的视频内容分析系统,其特征在于:视频内容分析模块,其中的镜头关键帧提取子模块根据视频镜头分割子模块的结果,对镜头内的视频帧进行聚类分析, 产生能够代表镜头信息的关键帧。 10. The video content analysis system according to claim 1, wherein: the video content analysis module, wherein the key frame extraction submodule shot segmentation result submodule according to a video camera, a video frame within a shot clustering analysis , representatives of the lens can produce information of key frames.
11. 如权利要求1所述的视频内容分析系统,其特征在于:视频内容分析模块,其中的场景分割子模块根据相应类别视频的结构特点,能够完成视频场景的分割。 11. The video content analysis system according to claim 1, wherein: the video content analysis module, wherein the scene cut module according to the structural characteristics of the respective sub-categories of video, it is possible to complete the divided video scene.
12. 如权利要求1所述的视频内容分析系统,其特征在于:视频内容分析模块,其中的场景分割子模块必须在镜头分割子模块和镜头关键帧提取子模块的功能完成之后才能进行,同时需要调用人脸分析子模块。 It can be carried out after the video content analysis module, wherein the sub-module needs to be divided scene segmentation sub-module and a lens sub-module key frame extraction is completed the lens, while: 12. The video content analysis system according to claim 1, characterized in that analysis sub-module needs to be called a human face.
13. 如权利要求1所述的视频内容分析系统,其特征在于:视频内容分析模块,其中的场景关键帧提取子模块根据场景分割子模块的结果,选取场景中包含的镜头的关键帧,进行聚类分析,产生代表场景信息的关键帧。 13. The video content analysis system according to claim 1, wherein: the video content analysis module, wherein the sub-module key frame extraction scene segmentation result sub-module according to a scene, a scene shot selected keys included, for cluster analysis, generate the key frame represents the scene information.
14. 如权利要求1所述的视频内容分析系统,其特征在于:视频内容分析模块,其中的关键帧图像分析子模块,对提取出来的视频镜头关键帧和视频场景关键帧图像进行颜色、纹理和形状三种特征的信息提取。 14. The video content analysis system according to claim 1, wherein: the video content analysis module, wherein the key frame image analysis sub-module, video footage of the extracted key frame and key frame video image scene color, texture and extract three features of the shape information.
15. 如权利要求1所述的视频内容分析系统,其特征在于:视频内容分析模块,其中的人脸分析子模块,对视频帧、视频镜头关键帧和视频场景关键帧进行人脸检测和人脸识另IJ,提取视频中包含的人脸信息。 15. The video content analysis system according to claim 1, characterized in that: the video content analysis module, wherein the face analysis sub-module, a video frame, the video shot key frame and key frame video scene face detection and human the other face recognition IJ, people extract the video information contained in the face.
16. 如权利要求1所述的视频内容分析系统,其特征在于:输出接口,根据视频内容分析模块得到的视频分析结果,生成视频内容结构化信息XML文件。 16. The video content analysis system according to claim 1, wherein: the output interface, the video analysis of the video content analysis module obtained results, to generate a video content of the XML file structure.
17. 如权利要求1所述的视频内容分析系统,其特征在于:视频内容分析结果审核模块,该模块是可选的,由用户配置决定,如果配置为不需审核,则视频内容结构化分析结果不经审核直接入视频信息数据库,若配置为需要审核,则视频内容结构化分析结果需要用户审核通过后才上传到视频信息数据库。 Structure analysis of the video content of video content analysis results of an examination module, which is optional, is determined by the user configuration, if the configuration need to review: 17. The video content analysis system according to claim 1, characterized in that the results are not audited directly into the video information database, if configured to require a review of the structure of the video content analysis requires the user to upload video review information database through before.
18. 如权利要求9所述的视频内容分析系统,其特征在于:镜头分割子模块,镜头分割算法的滑动窗口大小和切分阈值可以改变,以适应不同应用、不同视频的需要。 18. The video content analysis system according to claim 9, wherein: the sub-shot segmentation module, shot segmentation and a sliding window size threshold segmentation algorithm may be varied to suit different applications, different video.
19. 如权利要求10所述的视频内容分析系统,其特征在于:镜头关键帧提取子模块,可以改变聚类算法的阈值,以改变 19. The video content analysis system according to claim 10, characterized in that: a lens sub-key frame extraction module, may change the threshold clustering algorithm to change
20. 如权利要求ll所述的视频内容分析系统,其特征在于:场景分割子模块,其中包括针对新闻视频的场景分割算法, 算法包含镜头持续时间分析、镜头视频帧直方图距离变化的方差分析、镜头关键帧人脸检测和镜头关键帧人脸识别等过程。 20. The video content analysis system according to claim ll, wherein: the scene segmentation sub-module, which comprises a segmentation algorithm for the scene of the news video, duration analysis algorithm includes a lens, a histogram of the video frame shot ANOVA varying distance , shot key frame face detection and key frame shot and face recognition process.
21. 如权利要求13所述的视频内容分析系统,其特征在于:场景关键帧提取子模块,可以改变聚类算法的阈值,以改变场景关键帧的数量。 21. The video content analysis system according to claim 13, wherein: the scene extracting key frame sub-module, clustering threshold may be changed to vary the number of key frames of a scene.
22. 如权利要求16所述的视频内容分析系统,其特征在于:视频内容结构化信息XML文件,包含视频文件元数据信息、 视频镜头信息、视频镜头关键帧信息、视频场景信息、视频场景关键帧信息、关键帧图像的颜色信息、关键帧图像的纹理信息、关键帧图像的形状信息和人脸信息。 22. The video content analysis system according to claim 16, wherein: the video content of the XML file structure, file metadata information contains video, video shot information, shot key frame of video information, the video scene information, video scene key frame information, the color information of the key frame image, the key frame image texture information, shape information of the key frame image and the facial information.
23. 如权利要求22所述的视频内容分析系统,其特征在于:视频内容结构化信息XML文件,其中的视频文件元数据信息包括:视频文件名、视频文件路径、视频文件类型、视频内容结构化信息入库时间。 23. The video content analysis system according to claim 22, wherein: the video content of the XML file structure wherein files of video metadata information comprising: a video file name, video file path, the file type of video, video content structure information storage time.
CN 200610140834 2006-10-11 2006-10-11 Video content analysis system CN100545856C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 200610140834 CN100545856C (en) 2006-10-11 2006-10-11 Video content analysis system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 200610140834 CN100545856C (en) 2006-10-11 2006-10-11 Video content analysis system

Publications (2)

Publication Number Publication Date
CN101021904A CN101021904A (en) 2007-08-22
CN100545856C true CN100545856C (en) 2009-09-30

Family

ID=38709663

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 200610140834 CN100545856C (en) 2006-10-11 2006-10-11 Video content analysis system

Country Status (1)

Country Link
CN (1) CN100545856C (en)

Families Citing this family (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5493709B2 (en) 2009-03-13 2014-05-14 株式会社リコー Video editing device
CN101599179B (en) 2009-07-17 2011-06-01 北京邮电大学 Method for automatically generating field motion wonderful scene highlights
CN101833569A (en) * 2010-04-08 2010-09-15 中国科学院自动化研究所 Method for automatically identifying film human face image
CN101860663A (en) * 2010-04-29 2010-10-13 公安部第三研究所 Chip for realizing video structuring description treatment and method thereof
CN101867729B (en) * 2010-06-08 2011-09-28 上海交通大学 Method for detecting news video formal soliloquy scene based on features of characters
CN101901620A (en) * 2010-07-28 2010-12-01 复旦大学 Automatic generation method and edit method of video content index file and application
CN102156707A (en) * 2011-02-01 2011-08-17 刘中华 Video abstract forming and searching method and system
CN102819528B (en) * 2011-06-10 2016-06-29 中国电信股份有限公司 The method and apparatus generating video frequency abstract
CN102945234A (en) * 2011-08-15 2013-02-27 联想(北京)有限公司 Data information inquiry method and electronic device
CN102572514A (en) * 2012-01-11 2012-07-11 复旦大学 Service system based on video content epitome
CN103313122B (en) * 2012-03-09 2018-02-27 联想(北京)有限公司 A kind of data processing method and electronic equipment
CN104050599A (en) * 2013-03-17 2014-09-17 北京智慧农业有限公司 Edible agricultural product quality safety traceability system and method based on video playback
CN104216925A (en) * 2013-06-05 2014-12-17 中国科学院声学研究所 Repetition deleting processing method for video content
CN103327304B (en) * 2013-06-13 2016-04-20 北京林业大学 A kind of flower growth key-frame extraction system based on Accumulation Mechanism
CN103366163B (en) * 2013-07-15 2016-07-20 北京丰华联合科技有限公司 Face detection system and method based on incremental learning
IN2014CH00673A (en) * 2014-02-13 2015-08-14 Samsung R & D Institute India- Bangalore Private Limited A method and system to construct knowledge graph and knowledge-based assistive networks
CN105323634B (en) * 2014-06-27 2019-01-04 Tcl集团股份有限公司 A kind of reduced graph generating method and system of video
CN104270553B (en) * 2014-09-28 2017-12-08 北京奇艺世纪科技有限公司 A kind of switching detection method of video scene and device
CN104298748A (en) * 2014-10-13 2015-01-21 中南民族大学 Device and method for face search in videos
CN104320670A (en) * 2014-11-17 2015-01-28 东方网力科技股份有限公司 Summary information extracting method and system for network video
CN104735468B (en) * 2015-04-03 2018-08-31 北京威扬科技有限公司 A kind of method and system that image is synthesized to new video based on semantic analysis
CN106162222B (en) * 2015-04-22 2019-05-24 无锡天脉聚源传媒科技有限公司 A kind of method and device of video lens cutting
CN104902279B (en) * 2015-05-25 2018-11-13 浙江大学 A kind of method for processing video frequency and device
CN104881880B (en) * 2015-06-18 2017-10-10 福建师范大学 A kind of shot segmentation method based on sequential nature and subspace clustering
CN105096228B (en) * 2015-07-09 2019-05-24 东方网力科技股份有限公司 Video image information treating method and apparatus
CN105450987A (en) * 2015-11-12 2016-03-30 北京弘恒科技有限公司 Intelligent recognition platform video monitoring early warning system
CN107329821A (en) * 2016-04-29 2017-11-07 中国电信股份有限公司 Block of video data summary method for scheduling task and device
CN106126590A (en) * 2016-06-20 2016-11-16 北京航空航天大学 A kind of unmanned plane based on key message scouts video retrieval method
CN108701144A (en) * 2016-11-28 2018-10-23 微软技术许可有限责任公司 Automatically the content to show emotion and rich image index are detected from video
CN106778653A (en) * 2016-12-27 2017-05-31 北京光年无限科技有限公司 Towards the exchange method and device based on recognition of face Sample Storehouse of intelligent robot
CN106686441A (en) * 2016-12-30 2017-05-17 中广热点云科技有限公司 Video interaction method
EP3565243A1 (en) * 2017-01-20 2019-11-06 Huawei Technologies Co., Ltd. Method and apparatus for generating shot information
CN107995518A (en) * 2017-12-22 2018-05-04 青岛海信电器股份有限公司 Method for displaying image and device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003153192A (en) 2001-04-20 2003-05-23 Mitsubishi Electric Research Laboratories Inc Method and system for analyzing compressed video including a plurality of frames
CN1477566A (en) 2003-07-18 2004-02-25 北京大学计算机科学技术研究所 Method for making video search of scenes based on contents
CN1710563A (en) 2005-07-18 2005-12-21 北大方正集团有限公司 Method for detecting and abstracting importent new case

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003153192A (en) 2001-04-20 2003-05-23 Mitsubishi Electric Research Laboratories Inc Method and system for analyzing compressed video including a plurality of frames
CN1477566A (en) 2003-07-18 2004-02-25 北京大学计算机科学技术研究所 Method for making video search of scenes based on contents
CN1710563A (en) 2005-07-18 2005-12-21 北大方正集团有限公司 Method for detecting and abstracting importent new case

Also Published As

Publication number Publication date
CN101021904A (en) 2007-08-22

Similar Documents

Publication Publication Date Title
Hanjalic et al. An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis
JP5091086B2 (en) Method and graphical user interface for displaying short segments of video
US8717367B2 (en) Automatically generating audiovisual works
US9218367B2 (en) Method and interface for indexing related media from multiple sources
Brezeale et al. Automatic video classification: A survey of the literature
US8364698B2 (en) Apparatus and software system for and method of performing a visual-relevance-rank subsequent search
US7131059B2 (en) Scalably presenting a collection of media objects
US9870798B2 (en) Interactive real-time video editor and recorder
US8699806B2 (en) Method and apparatus for automatically summarizing video
US20030095720A1 (en) Video production and compaction with collage picture frame user interface
US8051081B2 (en) System and method for generating media bookmarks
CN1538351B (en) Method and computer for generating visually representative video thumbnails
Rui et al. Constructing table-of-content for videos
EP2402867B1 (en) A computer-implemented method, a computer program product and a computer system for image processing
US9779287B2 (en) Classification and organization of consumer digital images using workflow, and face detection and recognition
US6892193B2 (en) Method and apparatus for inducing classifiers for multimedia based on unified representation of features reflecting disparate modalities
Smoliar et al. Content based video indexing and retrieval
US20030117428A1 (en) Visual summary of audio-visual program features
US20090100454A1 (en) Character-based automated media summarization
KR20090091311A (en) Storyshare automation
US7555148B1 (en) Classification system for consumer digital images using workflow, face detection, normalization, and face recognition
JP4254160B2 (en) Videowork or audio information editing system and program
US7760917B2 (en) Computer-implemented method for performing similarity searches
US5664227A (en) System and method for skimming digital audio/video data
US7551755B1 (en) Classification and organization of consumer digital images using workflow, and face detection and recognition

Legal Events

Date Code Title Description
C06 Publication
C10 Request of examination as to substance
ASS Succession or assignment of patent right

Owner name: BEIJING NUFRONT NETWORK TECHNOLOGY CO., LTD.

Free format text: FORMER OWNER: BAO DONGSHAN

Effective date: 20080201

C41 Transfer of the right of patent application or the patent right
C14 Granted
C17 Cessation of patent right