JP4168940B2 - Video display system - Google Patents

Video display system Download PDF

Info

Publication number
JP4168940B2
JP4168940B2 JP2004016774A JP2004016774A JP4168940B2 JP 4168940 B2 JP4168940 B2 JP 4168940B2 JP 2004016774 A JP2004016774 A JP 2004016774A JP 2004016774 A JP2004016774 A JP 2004016774A JP 4168940 B2 JP4168940 B2 JP 4168940B2
Authority
JP
Japan
Prior art keywords
video
object
representative image
unit
display system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2004016774A
Other languages
Japanese (ja)
Other versions
JP2005210573A (en
Inventor
和也 佐藤
淑彦 秦
哲司 羽下
俊治 野沢
一裕 阿部
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to JP2004016774A priority Critical patent/JP4168940B2/en
Publication of JP2005210573A publication Critical patent/JP2005210573A/en
Application granted granted Critical
Publication of JP4168940B2 publication Critical patent/JP4168940B2/en
Application status is Expired - Fee Related legal-status Critical
Anticipated expiration legal-status Critical

Links

Description

  The present invention relates to a video display system, and more particularly to a video display system that recognizes an object shown in video data and explicitly displays the shot object.

A conventional video display system, for example, as disclosed in Patent Document 1, divides video data into a plurality of periods, performs classification by calculating the characteristics of each period from an image, and also represents a representative video of each period. By selecting an image and displaying this representative image for each classification type, it is intended to make it easy to grasp the contents of the video and search for a specific scene.
Further, for example, in Patent Document 2, the video data is divided by luminance difference comparison, and the scene types are classified into still image scenes, camera motion scenes, object motion scenes, and the like for each divided period. A representative frame corresponding to the type is detected.

  However, in these conventional video display systems, it is assumed that the method of dividing into a plurality of periods is to be divided at regular intervals or by detecting cut switching, and the target is the presence of individual objects. It is not a thing. Also, it is assumed that the representative image of each period is simply the top image of each period, or the motion information has the maximum value or the maximum screen similarity. It is difficult to grasp whether a featured object is an image captured within that period, and it is not sufficient for purposes such as searching for an object characterized by a certain behavior.

On the other hand, according to Patent Document 3, video content management is performed for an object appearing in video data, and a desired video portion is searched based on this. That is, as described in the abstract, moving objects in the video from the surveillance camera are detected in the video sequence by the motion segmentation device using a motion segmentation method. An object is tracked through segmented data in an object tracking device. A symbolic representation of the video is generated in the form of an annotated graph describing the object and its movement. The motion analyzer analyzes the results of tracking the object and annotates the motion of the graph with an index describing some events. The index is then used using a rule-based classification scheme to identify interesting events such as object appearance / disappearance, placing / removing, entering / exiting and movement / stillness. Is attached. A clip of the video defined by the spatio-temporal question and the event and object based question is called to show the desired video.
However, there is no particular mechanism for extracting a representative image that explicitly displays what kind of features the captured object has during that period.

Japanese Unexamined Patent Publication No. 2003-283968 (page 3, FIG. 1) JP-A-9-233422 (first page, FIGS. 1 and 2) JP-A-10-84525 (first page, FIG. 5)

  In the conventional video display system, the method of dividing the video is not strictly based on the individual object shown in the image, and the representative image to be displayed has the characteristics in the divided video period of the imaged object. It is not expressed explicitly. Therefore, the representative image to be displayed cannot sufficiently represent the contents of the video during that period, making it difficult to grasp what is shown in each video period. Therefore, for example, when trying to search for a video portion in which an object having a certain feature is captured, it is very difficult to narrow down a desired video portion while confirming that a subject that meets the search condition is captured. ing.

  The present invention has been made in order to solve the above-described problems of the prior art, and it is an object of the present invention to provide a video display system capable of facilitating the understanding of video content in a shorter time. Is.

  The video display system according to the present invention detects a video input unit that inputs video data, a video storage unit that stores the input video data, and individual objects appearing in the video data. Recognizing a period captured over a plurality of image frames and capturing it as one video period unit, and associating a series of identical objects captured within the video period unit with the video period unit as one object, An object processing unit for extracting object features as metadata; and at least one image satisfying a predetermined criterion from among video data stored in the video storage unit based on the metadata in each video period unit The frame is extracted as the representative image base in the video period unit, and the object associated with the video period unit is strong. Processed the representative image-based as a representative image processing unit for generating a representative image in the video period unit, in which a display unit for displaying the representative image.

  According to the present invention, the video period is divided for each object appearing in the video, and the feature of each target object during the period is emphasized and displayed. It is possible to handle a video period in a meaningful unit in which individual objects are continuously captured, and the factors divided into the video period units are clearly displayed. As a result, there is an effect that it is possible to easily grasp the content of the video based on what is shown in the video in a short time, and therefore it becomes easy to search for a desired video from the video data.

Embodiment 1 FIG.
FIG. 1 shows a functional block configuration of a video display system according to Embodiment 1 of the present invention, and is a diagram for explaining components and operation procedures of the video display system. The configuration and operation of the video display system according to this embodiment will be described below with reference to this figure. This system is constructed in an environment where processing is executed by a computer, such as a personal computer or a workstation.

The video input unit 10 receives input of video data transmitted from a video camera, a video recording / playback apparatus, or a network such as a LAN (Local Area Network) or a public line, and the input video data is stored in a subsequent video storage unit. 20 and the object processing unit 30.
Note that the video input unit 10 has an A / D (Analog-Digital) conversion function when video data is input from a device such as an analog video camera. On the other hand, when video data that has already been digitized is input via a LAN or the like, the A / D conversion function is not necessary, but communication protocol processing corresponding to a physical and logical interface is performed. And has a function of extracting only the video data portion.

  The video storage unit 20 stores the input video data, and an HDD (Hard Disk Drive) or a RAM (Random Access Memory) so that the representative image processing unit 50 can read the video data later. ) Using electronic media.

The object processing unit 30 includes an object extraction unit 31 and a metadata extraction unit 32, and the function thereof will be described below.
First, the object extraction unit 31 cuts out each object shown in the video data passed from the video input unit 10 from the background image, and the same object is a plurality of image frames (hereinafter referred to as image frames). Is also simply referred to as a frame.) Recognizing the periods that are captured in between, and grouping them into one video period unit, and recognizing a series of identical objects that are captured within this video period unit as one Associate as an object. Hereinafter, the object associated with the video period unit in this way may be referred to as a target object or an object corresponding to the video period unit.
Note that, for the method of extracting an object from a video and recognizing that the same object is captured between a plurality of frames, for example, image difference extraction as disclosed in “Image Monitoring Device” of Japanese Patent Application Laid-Open No. 2001-076156. In general, it is assumed that this is automatically performed by using an image processing technique such as template search. However, it is conceivable to use manual processing as necessary.

Next, the metadata extraction unit 32 extracts the characteristics of the target object in one video period unit in the form of metadata. As an extraction method, it is generally performed to automatically extract features such as an area, a position, and a movement locus of an object by using an image processing technique such as that disclosed in Japanese Patent Laid-Open No. 2001-076156. Assumed. However, it is also conceivable to use manual processing for this as necessary.
The extracted metadata is held in a form that facilitates subsequent processing. For example, when a computer having a file system is used to realize this system, a method such as saving as a metadata file is used.

  The metadata storage unit 40 stores the metadata extracted by the metadata extraction unit 32. FIG. 2 shows an example of metadata managed in the metadata storage unit 40 in the form of a table. As an example of the extracted metadata, for example, first frame information at the time when one object starts to appear in the video (frame information here is pointer information for identifying the corresponding frame, and includes a frame number and a time stamp. Etc.), the last frame information in which the object is shown at the end, or when the object is shown in the maximum area within the corresponding video period unit, or when the object is a human, the face is shown best. There are specific representative frame information that applies to conditions such as Alternatively, as the characteristics of the object itself, color information that the object has in the period, average motion vector amount, trajectory information such as which position it has passed through, and dwell time information at the passage location can also be mentioned. In addition, as an example of more detailed information, in each frame unit in which the object is captured, position coordinate information on the screen where the object is captured (the position is, for example, the center of gravity, an ellipse or a rectangle circumscribing the object, etc. Closed curve, a position that targets a closed curve such as an ellipse or a rectangle that contains a certain percentage of the area of the object in the region, etc.), if the object region part, ie the object is a person, for example, the shape of a person Bitmap information of the selected area.

  In addition, an example of metadata as an indirect feature amount with respect to the feature amount directly derived from the image processing as described above is given below. Based on the shape and motion characteristics of the object extracted from the video, for example, whether it is a person or a car, whether it is a person standing or sleeping, etc. In the case of a person, information such as race, gender and age can be considered from the body shape and face shape. Alternatively, an individual registered in the system in advance based on characteristics such as the shape of a person's face can be specified, or information for specifying an individual from an ID card or tag for identifying a personal ID is provided separately from video data information. If it is obtained, it may be possible to have the personal ID or name of the target object as metadata. In order to extract these features, usually, a dedicated image processing algorithm is provided in the metadata extraction unit 32. However, in some cases, it is also conceivable to perform processing through manual operation.

The representative image processing unit 50 includes a representative image base selecting unit 51 and an object emphasizing unit 52, and generates a representative image for easily expressing the characteristics of each video period unit including each object. However, the representative image in the present invention is not particularly limited to be composed of only one still image, and all representation forms including those having properties such as time-varying moving images are all representative images. I will call it.
The function will be described below.

First, the representative image base selection unit 51 selects a frame serving as a base for generating a representative image. That is, in each video period unit, at least one image frame satisfying a predetermined standard is extracted as the representative image base in the video period unit from the video data stored in the video storage unit 20 based on the metadata.
Satisfying a predetermined standard means that, for example, when a frame having the maximum area on the screen of an object is extracted in advance as one of the metadata items in the metadata extraction process, these frames are represented. You can choose an image base. In addition, it is not necessary that the extraction of the frame to be the representative image base in the metadata is completed in advance as described above. For example, if the area of the object in each frame is stored in the metadata, the representative image base The selection unit 51 may determine the representative image base after calculating the area having the maximum value in the metadata.

Other representative image base selection examples include when the object area is not the maximum but the average, when the object is not the area but the position is closest to a predetermined location such as the center of the screen, and the amount of object movement When the object is low, when the object performs a specific action or action, when the object is facing the front, in the case of a person, when the face is well reflected, the whole body is reflected Standards such as those of time may be used.
In addition, as a way of expressing the feature of an object, when it is desired to show a movement such as a movement locus, a plurality of frames may be selected as a representative image base.

As a function of the representative image processing unit 50, the object enhancement unit 52 then applies the selected representative image base so as to emphasize the characteristics of the captured object, that is, the object associated with the video period unit. Apply processing.
For example, when metadata has a position coordinate of a closed curve (for example, a rectangle or an ellipse) circumscribing the object on the screen, the object is surrounded by a closed curve circumscribing the object on the representative image base selected earlier. In this way, the image added with the superimposed display is made a representative image again.
Alternatively, as another example, the object area may be surrounded by a closed curve (for example, a rectangle or an ellipse), or only the area of the object, that is, if the object is, for example, a person. It is also possible to surround only a region that is covered or to be colored with a conspicuous color, or to change brightness or saturation.

Alternatively, emphasis can be achieved by enlarging and displaying an object region portion or a region within a circumscribed curve (for example, a rectangle or an ellipse) of the object. In this case, a representative image may be obtained by re-superimposing and displaying the enlarged object area on the original representative image base, or only the part in which the area in the circumscribed curve of the object is enlarged to the maximum is taken out and the representative area is represented again. You may re-specify as an image. Alternatively, the original representative image and the enlarged object image may be displayed in parallel to form a representative image.
Alternatively, if a method in which the resolution is partially changed in the video data compression method is used, a method of displaying only the object region with high definition is also conceivable.

In the above examples, the representative image base is processed based on only one frame shot at a certain time so that the object associated with the video period unit is emphasized, and the representative in the stationary state Although it is an example which produces | generates an image, as another example, the example of the object emphasis method when wanting to show motion information, such as a movement locus | trajectory of an object, is given below.
The generation of the representative image indicating the temporal change of the object is performed by, for example, selecting a plurality of frames as a representative image base at a predetermined time interval and using any one frame as a background image. By repeatedly superimposing and deleting images that are cut out of only the object area from the frame, only the object area is displayed as if it is moving on the background image, or the superimposed image is erased. By repeating the superimposing process in order, the moving trajectory like a stroboscopic photograph is displayed, or the image of the object area is superimposed only on representative points such as the first and last, The movement trajectory represented by a curve can be superimposed and displayed, for example, multiple frames selected can be simply framed as a quasi-video Or to the display, it is a method such as. Of course, these methods may be used in parallel with the method of emphasizing each object region by surrounding it with a frame as described above.

  The examples shown above are all emphasized representations using a graphical method, but feature information held as metadata is superimposed on the representative image by text, displayed additionally around the representative image, or by voice A method of highlighting the features of an object by expressing it is also conceivable.

  In addition, among the methods described above, a plurality of display methods may be displayed simultaneously. One of the methods of simultaneously expressing a plurality of display methods is an expression method in which, for example, a representative image created by the method B is displayed side by side next to the representative image created by the method A. Alternatively, if the method A is to generate a representative image with only one frame as described above, and the method B is a method to show motion information as described above, it is generated by A. Alternatively, a method of superimposing and displaying a partial area of the object by the method B on the representative image may be used.

  On the other hand, the condition input unit 60 serves as a search condition input unit for inputting a search condition and a display condition input unit for inputting a condition related to a display method in the present embodiment, and is a user who uses the video display system. In the case of an application for performing a search, the search condition is accepted, information is displayed to the representative image processing unit 50 for the display condition, and information is passed to the metadata evaluation unit 70 for the search condition. As an input method, for example, the input is performed via a user interface found on a general personal computer using a keyboard, mouse, touch panel, or the like, or an input device such as a dedicated button or dial.

When a display condition is received from the condition input unit 60, the representative image base selection unit 51 performs selection so as to meet the condition, and the object enhancement unit 52 performs processing of the representative image so as to meet the condition. .
The display conditions here are variations on the display method as described in the description of the representative image processing unit 50. For example, a representative image base selection criterion at the still image level, such as selecting the object with the largest area as the representative image base, or selecting the object whose position on the screen is close to the center instead of the size is designated. Alternatively, as a method of emphasizing the object in the representative image, for example, highlighting such as surrounding the object area with a frame, coloring the object area, or using the object area portion that is maximized as the representative image is used. You may specify the method. Or, if you want to express the movement of an object, specify the display method, such as strobe photo sequential display, sequential display like a quasi-movie with the object area expanded to the maximum extent, or display of traces superimposed on a curve. May be.

  When a search condition is received from the condition input unit 60, the information is passed to the metadata evaluation unit 70, where the content of the metadata is collated with the input condition, and an object having metadata that meets the condition. The representative image processing unit 50 generates a representative image for an object that satisfies the conditions.

As an example of the condition search, for example, a case where the average movement vector information of the object is included in the content of the metadata will be described. For example, it is assumed that the metadata has a value obtained by projecting an average movement vector from the appearance of an object on the screen to the exit on the horizontal axis of the screen. That is, for example, numerical data such as a positive numerical value when moving to the right side, a negative numerical value when moving to the left side, and zero when there is no average left-right movement is included.
When the user gives a condition of “person moving to the right” among the people shown in the video, the value of the object average movement vector is examined in the metadata, and the value is a positive number. Extract only objects that match the search criteria. Of course, the determination threshold value may be provided with a value close to zero instead of determining whether it is strictly positive or not positive.
Of course, even in the case of using such a simple average movement vector, not only the horizontal direction but also the vertical and diagonal directions may be used, or environmental information about what is shown on the screen will be given. Thus, the feature of moving to the right can be replaced with, for example, the action of entering the entrance, and the feature of moving to the left can be replaced with the action of exiting the entrance.

  Finally, the display unit 80 displays the representative image of each object generated by the representative image processing unit 50 on a display device such as a display. When there are multiple objects to be displayed, they are arranged in order of time based on the start time of each video period, representative image base time, etc., or not only the order of arrangement based on the time but also the relative position on the time axis The arrangement position for displaying the representative image may be determined at the intervals.

In addition, the metadata evaluation unit 70 informs the representative image processing unit 50 of a list of objects that meet the conditions, and at the same time, gives a value obtained by scoring the matching degree when the condition matching is performed, and a plurality of search conditions. If it is, a list of objects that match each condition may be transmitted to the display unit 80.
In that case, for example, it is possible to display in order from the object with the highest matching score, or to display for each object that meets the individual conditions when there are a plurality of conditions. As an example of the score of the matching degree, for example, when searching for a person moving rightward as described above, the larger the object average movement vector value, the larger the rightward movement, so that a plurality of displays If there are objects, there is also a method of displaying them in this numerical order instead of simply in time order.

FIG. 3 is a flowchart in which the functions and processes of the video display system according to the present embodiment described so far are simply rearranged in chronological order, and the operation of the video display system according to the present embodiment based on this figure. Will be explained again.
First, the video data input by the video input unit 10 is stored in the video storage unit 20 (step ST1). Next, an object is extracted from the video data by the object extraction unit 31 (step ST2), and the feature amount of the object is extracted as metadata by the metadata extraction unit 32 and stored in the metadata storage unit 40 (step ST3). .

  On the other hand, since the input of the search condition and the input of the display condition are input from the user, the position on the time series is generally indefinite and may not exist depending on the case. For example, all the conditions to be given can be set at the initial operation of the system, or usage can be considered in which all objects are displayed without any narrowing conditions. Here, in the example shown in FIG. 3, the search condition is first input in the condition input unit 60 (step ST4), and the object to be displayed is determined in the metadata evaluation unit 70 (step ST5). Next, the display condition is input in the condition input unit 60 (step ST6), the base of the representative image is selected by the representative image base selection unit 51 based on the display condition (step ST7), and the object enhancement unit 52 The representative image is processed and generated (step ST8). Finally, the representative image is displayed on the display unit 80 (step ST9).

Next, the relationship between the object, the video period unit corresponding to the object, and the representative image will be described in more detail with reference to FIG.
First, object extraction will be described. In FIG. 4, two persons are shown in five video frames with frame numbers 1 to 5. Specifically, the first frame (frame 1) is one person, the second to fourth frames (frames 2 to 4) are two persons, and the fifth frame (frame 5) is One person is shown, and this is detected by image difference extraction processing or the like.
Further, the same person is continuously captured in the first to fourth frames (frames 1 to 4) by the template search process or the like, and the second to fifth frames (frames 2 to 5). It is recognized that another person is still in the image. The first person is called object 1 and the second person is called object 2, and the corresponding video period units are frame 1 to frame 4 for object 1 and frame 2 to frame 5 for object 2. Up to.
As in this example, two video period units may overlap in time. That is, even if the video data is divided, the same video portion may be included in a plurality of video period units.
Next, metadata obtained by extracting features in each video period (from frame 1 to frame 4 and from frame 2 to frame 5) is extracted from object 1 and object 2.

Next, generation of a representative image will be described. In the example of FIG. 4, the object 1 first selects the frame 1 as the representative image base of the object 1 from the frames 1 to 4, and further circumscribes the area in the frame 1 where the object 1 is shown. An object is emphasized by superimposing and displaying a quadrangle (corresponding to a frame, which is a rectangle in FIG. 4) with a thick line, and a representative image is obtained.
Similarly, in the example of the object 2, first, the frame 4 is selected from the frames 2 to 5 as the representative image base, and further, a rectangle circumscribing the area where the object 2 is captured (corresponding to a frame, in FIG. The object region is emphasized by superimposing the rectangle with a thick line to form a representative image.

  FIG. 5 is an example of a screen display in the present embodiment. For a certain video data, the user specifies the search time range, specifies the condition of “person moving right”, and displays the first 5 of the 20 objects that meet the condition. It represents the state of being. Among the metadata possessed by the five objects, representative images are arranged in thumbnail order from the top in the order of the first frame time. As the display contents of the representative image, a plurality of display methods are displayed side by side as shown below.

One is a method in which a frame in which each corresponding object is captured in the maximum area within each video period unit is used as a representative image base, and a circumscribed rectangle (frame) surrounding the region of the object is superimposed and displayed as processing for enhancing the object. A still image is displayed. Although some images include a plurality of people, only the target object is highlighted, and other people treat the object as a part of the background. This is a representative image 1 indicating a stationary state.
The second is a display in which only the area of the target object is enlarged to the maximum (next right side in FIG. 5). Further, regarding this display, the object region portion is cut out from a plurality of frames over a time range in units of video periods, not simply as a still image, and sequentially displayed. In the case where the video period is long, the movement is made like a quasi-moving picture by thinning out frames at a predetermined interval instead of all the frames during the period. This is a representative image 2 showing the temporal change of the object.
Next to that (on the right side in FIG. 5), as other metadata information about each object, the start time and end time of the object, the time of the frame that is the base of the representative image 1 (still image), etc. It is written together with text.

  In FIG. 5, the processing is performed to emphasize an object on one image frame extracted as a representative image base to obtain a representative image 1 indicating a stationary state. However, the representative image 1 is not subjected to processing to emphasize the object. One image frame extracted as a base may be used as a representative image 1 indicating a still state as it is.

  As an application example regarding the use of this video display system, the user selects one of the representative images of the objects that meet the search conditions in this way, and clicks the “video display execution” button in the lower left of FIG. When the button is pressed, it is assumed that an application is created so that video playback in a period (video period unit) in which the corresponding object is captured is started.

  FIG. 6 shows another example of generating a representative image. In this example, a strobe photograph is displayed by paying attention to the movement of the object. That is, a plurality of frames are selected at a predetermined time interval, and one of the frames is used as a background image, and then an image obtained by cutting out an object area portion from each frame is photographed at a corresponding position and size. Superimposition is repeated sequentially from the top in the order of time, and finally the display is made with the movement locus remaining as in the stroboscopic photograph shown in the figure.

  In the example of FIG. 6, two people are shown in addition to the target object, but this is only shown as a part of the background image and is not an object to be emphasized. On the other hand, a person who is a target object is distinguished by using a method in which an image in which an object region portion is surrounded by a frame is further superimposed and displayed as a highlighting method.

  Alternatively, it is the same in that the object area image is superimposed on the background image at the corresponding position and size, instead of being a stroboscopic photograph in which all object area image superposition is left, as shown in FIG. As in the second display method (representative image 2), the display may be performed so that the target object area looks like a moving image by sequentially repeating superposition and deletion. This is because the representative image 2 by the second sequential display method in FIG. 5 is not displayed side by side next to the representative image 1 indicating the first stationary state, but the object region image is not enlarged. This is equivalent to the interpretation that the position is superimposed on the representative image 1 indicating the first stationary state. That is, the method of performing a plurality of display methods simultaneously may display not only a plurality of (two in this example) representative images side by side but also display them as one representative image.

In this way, a period in which the same object is continuously captured in the video data is treated as one video period unit, and a series of the same object captured in the video period unit is associated with the video period unit as one object. Further, by generating and displaying a representative image so that the object is emphasized in this video period unit (for example, the position where the object is photographed is emphasized as in the representative image 1 of FIG. 4 or FIG. 5). Thus, there is an effect that the video can be handled in a meaningful period unit in which individual objects in the video are continuously captured, and the content of the video can be easily grasped within the video period unit.
In addition, even if you want to search for a part of the video data where the desired object is shown, you can grasp the video content in units of video periods in which each object is shown. There is an effect that a desired video portion can be easily narrowed down even within a period unit.

  In particular, there is an effect that the behavior of the object in units of video periods can be grasped more clearly by using a representative image representing the movement of the object as in the sequential display portions of FIGS.

  Further, in the present embodiment, a search condition input unit 60 to which a search condition is input and a metadata evaluation unit 70 that extracts an object having metadata that matches the input search condition are provided. Since the representative image is displayed in the video period unit associated with the object extracted by the metadata evaluation unit 70, the user can easily search the video data portion where the desired image is shown. In this case as well, since it is possible to grasp the video content in units of video periods in which each object is shown, it is possible to easily narrow down a desired video portion from a plurality of listed candidate video period units. The effect is obtained.

  Further, by combining a plurality of display methods with different emphasis viewpoints as shown in FIGS. 5 and 6, the object portion can be enlarged in detail while looking over the entire camera angle of view in which the image is projected spatially. On the other hand, on the time axis, the video at a certain time can be seen in detail by one still image while looking over the movement of the whole video period by sequentially displaying the object region images. That is, it is possible to compare the whole with the details for each video period unit, and it is possible to more easily grasp the video content.

  In addition, the display condition input unit 60 for inputting conditions related to the display method is provided, and the representative image processing unit 50 generates a representative image in accordance with the input conditions related to the display method. Therefore, the user instructs the conditions related to the display method. Thus, it is possible to change which representative image is used as a base and which enhancement method is used to generate the representative image. Therefore, it is possible to select which feature of the object is displayed with attention, and as a result, there is an effect that the user can more easily grasp the video content corresponding to his / her purpose.

Embodiment 2. FIG.
In the first embodiment, it is assumed that the application includes the condition input unit 60 and the user specifies the search condition. However, the present invention is not limited to the form in which the user specifies the condition and performs the search. The following is an example, not applicable.

FIG. 7 shows a functional block configuration of the video display system according to the second embodiment of the present invention, and is a diagram for explaining components and operation procedures of the video display system. Hereinafter, the configuration and operation of the video display system according to the present embodiment will be described with reference to this figure. Here, differences from the first embodiment will be mainly described.
The video display system according to the present embodiment does not include the condition input unit 60 as described in the first embodiment.
Also, as the processing of the metadata evaluation unit 70, instead of extracting only those that meet the given conditions as described in the first embodiment, each object is set to a predetermined condition for the metadata. Based on grouping. That is, each object (video period unit) is grouped according to the contents of a certain item in the metadata.
Further, the display unit 80 displays representative images in video period units associated with each object in group units. That is, representative images are listed for each grouped content.

  In this case, for example, the metadata evaluation unit 70 classifies the objects into three groups such as “an object moved rightward”, “an object moved leftward”, and “an object that does not move in the left / right direction”. Is transmitted to the display unit 80. In the display unit 80, as shown in the example of the screen display in FIG. 8, the display is performed by classifying into three groups of “move rightward”, “move leftward”, and “little leftward / rightward movement”. The method of creating the representative image in this case is the same as in FIG.

In the video display system of such a form as well as the one described in the first embodiment, the video can be handled in a unit of a meaningful period in which individual objects in the video are continuously captured, In addition, the basic effect of easily grasping the contents of the video can be obtained.
In addition to this, in the case of such a form, the video display system automatically analyzes the video content without the user performing an operation of inputting the search condition in particular, and is divided based on a predetermined condition. Since it is presented for each group, it is possible to easily narrow down a desired video portion.

Embodiment 3 FIG.
Further, as another example, not only a case where a list of representative images of a plurality of objects (video period units) is displayed, but one representative image corresponding to one object (video period units) is displayed. It may be a simple application that only displays.

FIG. 9 shows a functional block configuration of a video display system according to Embodiment 3 of the present invention, and is a diagram for explaining components and operation procedures of the video display system. Hereinafter, the configuration and operation of the video display system according to the present embodiment will be described with reference to this figure. Here, differences from the first embodiment will be mainly described.
The video display system according to the present embodiment does not include the condition input unit 60 and the metadata evaluation unit 70 as described in the first embodiment.

  For example, when the object processing unit 30 recognizes one object from the video and extracts the feature of the object as metadata, the representative image processing unit 50 generates one representative image. A simple operation of displaying on the display unit 80 can be considered. FIG. 10 shows an example of the screen display. The method of creating the representative image in this case is the same as in FIG.

In the video display system of such a form as well as the one described in the first embodiment, the video can be handled in a unit of a meaningful period in which individual objects in the video are continuously captured, In addition, the basic effect of easily grasping the contents of the video can be obtained.
In addition to this, in the case of such a form, there is also an effect that the video display system side automatically analyzes and presents the video content even if the user does not particularly perform an operation of inputting search conditions.

Embodiment 4 FIG.
In each of the embodiments described above, the target to be highlighted in the representative image is only the target object corresponding to the video period unit that is continuously captured from the beginning to the end of the video period unit. It was. However, except for the case where only one object is detected within that period, the object processing unit 30 separately forms other objects, and the feature amount is extracted as metadata. ing.
Therefore, in this embodiment, an object other than the target object (shown in a part of the video period within the video period unit is handled while handling the video in the video period unit corresponding to each object. Highlighting is also performed on objects associated with different video period units.

  FIG. 11 shows a display example of a representative image in the video display system according to the present embodiment. As a comparison, in FIG. 5 according to the first embodiment, as a representative image 1 indicating a stationary state, only a circumscribed square frame surrounding the region of the target object is superimposed on a single representative image base to emphasize the target object. Although an example has been shown, in FIG. 11, other objects (objects associated with a video period unit different from the video period unit) detected in addition to the target object also have a circumscribed rectangle surrounding the area. Emphasis is given by superimposing the frame.

  FIG. 12 shows another display example of the representative image in the video display system according to the present embodiment. For comparison, in FIG. 6 according to the first embodiment, a single representative image base is used as a background image, and corresponding object areas are cut out from a plurality of frames only for the target object, and are sequentially superimposed and displayed. As for an object other than the target object (an object associated with a video period unit different from the video period unit), a region captured within the range of the video period unit is cut out and sequentially superimposed.

  Further, regarding the presence / absence of emphasis expression for objects other than the object corresponding to each video period unit, a range from a method of emphasizing all objects under the same condition to a method of not emphasizing at all may be handled in a scalable manner. . For example, in FIG. 11, the left person as the target object is surrounded by a thick frame line, whereas the right person is surrounded by a thin frame line, and the degree of emphasis is changed. Further, in the example as shown in FIG. 12, it is possible to make a difference with respect to objects other than the target object by increasing the time interval of images to be superimposed and displayed.

  Alternatively, the handling between multiple objects in one video period unit is not only divided into the target object and the other two, but also whether it is near or far from the target object, It may be divided into stepwise groups according to the strength of the degree of association with the target object, such as whether or not there is an intersection, and differentiated in the way of emphasis. For example, the target object is displayed with the strongest emphasis, relatively weak emphasis is given only to the object of the highly related group, and no emphasis is given to the object of the less relevant group, In some cases, it may be differentiated such that a group of objects having a low degree of association is positively deleted from a background image that is a representative image base.

As in the present embodiment, by highlighting objects other than the target object, it is possible to make it easier to grasp the contents of the video when handling a video that includes a plurality of objects. can get.
In addition, when a plurality of objects other than the target object are captured, the degree of emphasis on these objects can be varied to pay attention to a plurality of surrounding objects according to the user's purpose. It is possible to obtain a display that emphasizes the behavior of the existing object.

Embodiment 5 FIG.
In Embodiment 1, when a user gives a search condition, it seems that the user is interested in matters related to the search condition in video data. Thus, in the present embodiment, information related to the search condition is highlighted without separately inputting the condition of the display method.

For example, it is assumed that the type of metadata possessed by an object includes an estimated age, sex, personal identification number, etc. of the person in addition to the movement trajectory information of the object.
In this metadata holding situation, it is assumed that, for example, a condition that “the average moving direction of the object is rightward” is input from the user as a search condition. In this case, since it is considered that the user is interested in information regarding the movement direction of the object, the search results are displayed by selecting only the objects whose average movement direction is rightward and displaying a list of representative images. Is displayed in such a manner that information on the movement trajectory is clearly indicated. That is, for example, processing such as displaying a polygonal line or curve of a movement trajectory in a representative image or displaying information such as a total movement amount or average movement speed as an icon or text information near the object area is performed. An example that is performed by the representative image processing unit 50 is given. Of course, information such as the total movement amount and the average movement speed may be additionally displayed around the representative image as icons or text information.

  On the other hand, in the metadata holding situation similar to the above, it is assumed that, for example, a condition that “age is a person in their 20s” is input from the user as a search condition. In this case, it is considered that the user is more interested in personal information about what kind of person the person is than information on the movement of the object. Accordingly, in this case, only the corresponding age group objects are selected to display a list of representative images, and for each representative image, display is performed so that the personal information of the target object is clearly indicated. That is, for example, the representative image processing unit 50 performs processing such as superimposing and displaying information such as estimated age, sex, and personal identification number near the object area using text and icons. Of course, information such as the estimated age, sex, and personal identification number of the object may be additionally displayed around the representative image using text and icons.

  As in the present embodiment, based on the search condition input by the user, the contents to be emphasized in the representative image are automatically determined and the representative image is generated, so that the user can display the search condition in addition to the search condition. There is an effect that it is possible to save time and effort to input the conditions regarding the method. In addition, since information related to the content that the user is interested in is also displayed, it is possible for the user who has searched for the purpose of searching for a desired video to understand the content more meaningfully. can get.

Embodiment 6 FIG.
In the fifth embodiment, an example is shown in which the feature information of the object related to the search-related condition given by the user is explicitly displayed. However, in the present embodiment, conversely, the input search condition is changed to the input search condition. Display for unrelated metadata, that is, metadata other than the input search condition. In the search result obtained by the search process, all the information displays related to the search condition input by this user should be satisfied. Therefore, when there are many features related to the object in the metadata, it is not necessary to display additional information about the search results. Rather, it may be more convenient to have other feature information displayed additionally. Conceivable.

For example, it is assumed that the type of metadata possessed by an object includes an estimated age, sex, personal identification number, etc. of the person in addition to the movement trajectory information of the object.
In this metadata holding situation, for example, when a condition related to the moving direction of an object is input as a search condition, all target objects in the representative image displayed as a result should satisfy a certain standard regarding the moving direction. Therefore, as other features related to the object, for example, processing such as superimposing and displaying personal information such as the estimated age, sex, and personal identification number of the object in the vicinity of the object area using text and icons is representative image processing. An example performed by the unit 50 can be considered. Of course, personal information such as the estimated age, sex, and personal identification number of the object may be additionally displayed around the representative image using text and icons.

  On the other hand, in the metadata holding situation similar to the above, it is assumed that, for example, a condition that “age is a person in their 20s” is input from the user as a search condition. In this case, all the target objects in the representative image displayed as a result should satisfy only the estimated age condition. Therefore, as other information related to the object, rather than superimposing information related to the movement trajectory or displaying information related to the personal information of the object, text and icons are limited to only information on the estimated gender and personal identification number other than the estimated age. An example in which the representative image processing unit 50 performs processing such as superimposing and displaying near the object area using the. Of course, an estimated gender other than the estimated age of the object, a personal identification number, and the like may be additionally displayed around the representative image using text and icons.

  When the object has many features as metadata by displaying the feature information of the objects other than those related to the search condition input by the user as in the present embodiment, a lot of information If you display, the screen may be difficult to see, but the content that is known by the search results can be reduced without displaying it, and more information can be displayed with a simpler display method. Is obtained.

Embodiment 7 FIG.
In each of the embodiments described above, when there are a plurality of objects to be displayed, the representative image is displayed in a list format such as a thumbnail display. On the other hand, in the present embodiment, one representative image in the case where the background of the captured image is basically unchanged, such as an image of a fixed camera that does not swing. A plurality of object area images are superimposed and displayed on the base.

FIG. 13 shows a functional block configuration of a video display system according to Embodiment 7 of the present invention, and is a diagram for explaining components and operation procedures of the video display system. Hereinafter, the configuration and operation of the video display system according to the present embodiment will be described with reference to this figure. Here, differences from the first embodiment will be mainly described.
The video display system according to the present embodiment is different in that the function of the display unit 80 is composed of a background image generation unit 81 and an object composition unit 82, and the operation is a representative image in addition to the display unit 80. There are some differences in processing in the processing unit 50.

  First, in the representative image processing unit 50, when generating representative images for a plurality of objects, not only an additional recording process such as superimposing an outer frame surrounding the object region on the representative image base, Extraction processing that cuts out a part of the image data is necessary. Further, the representative image-based still image data before the object region highlighting is added is also separately transferred to the display unit 80.

In the display unit 80, the background image generation unit 81 in the display unit 80 selects one of the plurality of representative images generated by the representative image processing unit 50, and further, the still image that is the representative image base of the representative image. Designated as a background image using image data.
Next, in the object composition unit 82 in the display unit 80, the highlight display of the object area for each object is sequentially added to the background image extracted by the background image generation unit 81, and this is added to a display device such as a display. Is displayed.

FIG. 14 shows an example of a screen display in the present embodiment. For certain video data, the user specifies a search time range, specifies a condition that the average moving direction of the object is rightward, and displays five objects that meet the condition. Represents a state.
Since all the images in which these objects are reflected are in the same background, one of the still images serving as the representative image base of each object is used as the background image. On the background image, as a feature of each object, an emphasis expression by clearly indicating movement trajectory information so that the movement of each object can be seen at a glance is superimposed.
Specifically, it may be a polaroid photographic expression as shown in FIG. 6 shown in the first embodiment, but here it is also considered that a large number of objects may be difficult to see when superimposed on the same screen. Thus, only one representative image is superimposed on each object area, and the remaining trajectory information is expressed by superimposing a curved arrow that connects the positions of the centroids that each object area has passed across multiple frames. ing.

  The example described above is described on the assumption that the camera is not fixed, but for example, even when the camera moves to the left or right, the change in the background position associated with the change in the shooting direction is associated, In the case of having a function for synthesizing an image such as a panoramic picture, it is possible to display the image similar to the above example using this as a common background image for each object. FIG. 15 shows an example of a representative image in this case.

When displaying representative images of a plurality of objects as in the present embodiment, they are displayed in a superimposed manner on a single background image instead of being displayed side by side as thumbnails, that is, a plurality of images having the same background. By displaying the representative images in the video period units so that the objects associated with the video period units are superimposed on the common background image, the representative images of the objects to be listed are displayed. When it does not fit within the screen area to be performed, the inconvenience that the remaining list must be displayed again using screen scrolling can be solved.
As another effect, by superimposing a plurality of objects on one background image, where many objects appear in a biased position, or positions and movement trajectories that are extremely different from many objects. It is possible to obtain a clearer understanding of the spatial positional relationship between multiple objects, such as whether there is no object with or which object and which object appeared close to each other. .

FIG. 3 is a diagram for explaining components and operation procedures of the video display system according to the first embodiment. FIG. 10 is a diagram illustrating an example of metadata according to the first embodiment. 3 is a flowchart for explaining the operation of the video display system according to the first embodiment. FIG. 3 is a diagram for explaining in more detail the video display system according to the first embodiment. FIG. 6 is a diagram illustrating an example of a screen display according to the first embodiment. FIG. 10 is a diagram illustrating another display example of a representative image according to the first embodiment. FIG. 10 is a diagram for explaining components and operation procedures of a video display system according to a second embodiment. FIG. 10 is a diagram illustrating an example of a screen display according to the second embodiment. FIG. 10 is a diagram for explaining components and operation procedures of a video display system according to a third embodiment. FIG. 16 is a diagram illustrating an example of a screen display according to the third embodiment. FIG. 16 is a diagram illustrating a display example of a representative image according to the fourth embodiment. FIG. 20 is a diagram illustrating another display example of a representative image according to the fourth embodiment. FIG. 10 is a diagram for explaining components and operation procedures of a video display system according to a seventh embodiment. FIG. 38 is a diagram illustrating an example of a screen display according to the seventh embodiment. FIG. 38 is a diagram illustrating another display example of a representative image according to the seventh embodiment.

Explanation of symbols

10 video input unit, 20 video storage unit, 30 object processing unit, 31 object extraction unit, 32 metadata extraction unit, 40 metadata storage unit, 50 representative image processing unit, 51 representative image base selection unit, 52 object enhancement unit, 60 condition input unit, 70 metadata evaluation unit, 80 display unit, 81 background image generation unit, 82 object composition unit.

Claims (11)

  1. A video input unit for inputting video data;
    A video storage unit for storing input video data;
    Individual objects appearing in the video data are detected, and the period in which the same object is shown over a plurality of image frames is recognized and regarded as one video period unit. An object processing unit that associates a series of identical objects as one object with the video period unit, and extracts features of the object as metadata;
    In each video period unit, extracting at least one image frame satisfying a predetermined standard from the video data stored in the video storage unit based on the metadata as a representative image base in the video period unit, A representative image processing unit that processes the representative image base so that an object associated with the video period unit is emphasized, and generates a representative image in the video period unit;
    A video display system comprising: a display unit that displays the representative image.
  2. In the video display system according to claim 1,
    A metadata evaluation unit that performs grouping on the metadata of each object based on a predetermined condition,
    The video display system, wherein the display unit displays a representative image in a video period unit associated with each object in the group unit.
  3. In the video display system according to claim 1,
    A search condition input part for inputting a search condition;
    A metadata evaluation unit that extracts an object having metadata that matches the input search condition,
    The video display system, wherein the display unit displays a representative image in a video period unit associated with the extracted object.
  4. In the video display system according to any one of claims 1 to 3,
    The representative image processing unit generates a representative image so that an object associated with a video period unit different from the video period unit is emphasized although it is captured in a part of the video period within the video period unit. A video display system characterized by that.
  5. In the video display system according to any one of claims 1 to 4,
    The representative image processing unit extracts a plurality of image frames as a representative image base in a video period unit, and uses these image frames to display a representative image indicating a temporal change of an object associated with the video period unit. A video display system characterized by generating.
  6. In the video display system according to claim 5,
    The generation of the representative image indicating the temporal change of the object is performed by cutting out a part including at least a part of the object region associated with the video period unit in each image frame, and cutting the cut out object part along the time axis. An image display system characterized by performing processing so as to display sequentially.
  7. In the video display system according to claim 5,
    The representative image processing unit extracts a single image frame satisfying a predetermined criterion as a representative image base in the video period unit, and performs a process of enhancing the object associated with the video period unit. And a representative image showing a temporal change of the object according to claim 5,
    The video display system, wherein the display unit displays both a representative image showing the stationary state and a representative image showing a temporal change of the object.
  8. In the video display system according to any one of claims 1 to 7,
    Provided with a display condition input part for inputting conditions related to the display method,
    The video display system, wherein the representative image processing unit generates a representative image in accordance with the input condition relating to the display method.
  9. In the video display system according to any one of claims 3 to 8,
    The video image display system characterized in that the representative image processing unit superimposes metadata related to an input search condition when processing the representative image base so that the object is emphasized.
  10. In the video display system according to any one of claims 3 to 8,
    The video display system, wherein the representative image processing unit superimposes metadata other than the input search condition when processing the representative image base so that the object is emphasized.
  11. In the video display system according to any one of claims 1 to 10,
    The display unit displays a representative image in a plurality of video period units having the same background so that each object associated with each video period unit is superimposed on one background image. Characteristic video display system.
JP2004016774A 2004-01-26 2004-01-26 Video display system Expired - Fee Related JP4168940B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2004016774A JP4168940B2 (en) 2004-01-26 2004-01-26 Video display system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2004016774A JP4168940B2 (en) 2004-01-26 2004-01-26 Video display system

Publications (2)

Publication Number Publication Date
JP2005210573A JP2005210573A (en) 2005-08-04
JP4168940B2 true JP4168940B2 (en) 2008-10-22

Family

ID=34901820

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2004016774A Expired - Fee Related JP4168940B2 (en) 2004-01-26 2004-01-26 Video display system

Country Status (1)

Country Link
JP (1) JP4168940B2 (en)

Families Citing this family (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3465760B2 (en) * 1994-08-03 2003-11-10 新日本石油株式会社 Refrigerator oil composition and refrigeration fluid composition
JP2007052626A (en) * 2005-08-18 2007-03-01 Matsushita Electric Ind Co Ltd Metadata input device and content processor
US8949235B2 (en) 2005-11-15 2015-02-03 Yissum Research Development Company Of The Hebrew University Of Jerusalem Ltd. Methods and systems for producing a video synopsis using clustering
EP1955205B1 (en) 2005-11-15 2012-08-29 Yissum Research Development Company Of The Hebrew University Of Jerusalem Method and system for producing a video synopsis
JP2007184853A (en) * 2006-01-10 2007-07-19 Mitsubishi Electric Corp Video recording system
JP2007207191A (en) * 2006-02-06 2007-08-16 Asv:Kk Delay click mode in image reproduction system, and player device and server device used for mode thereof
JPWO2008041629A1 (en) * 2006-09-29 2010-02-04 ソニー株式会社 Reproduction device and method, information generation device and method, data storage medium, data structure, program storage medium, and program
JP4755575B2 (en) * 2006-12-12 2011-08-24 ヤフー株式会社 Video representative image extraction method, apparatus, and computer program
JP4920395B2 (en) * 2006-12-12 2012-04-18 ヤフー株式会社 Video summary automatic creation apparatus, method, and computer program
JP5063098B2 (en) * 2006-12-12 2012-10-31 ヤフー株式会社 Information providing apparatus, information providing method, and computer program
JP5355422B2 (en) * 2007-02-01 2013-11-27 イッサム・リサーチ・デヴェロップメント・カンパニー・オヴ・ザ・ヘブルー・ユニヴァーシティ・オヴ・イェルサレム Method and system for video indexing and video synopsis
US20080198159A1 (en) * 2007-02-16 2008-08-21 Matsushita Electric Industrial Co., Ltd. Method and apparatus for efficient and flexible surveillance visualization with context sensitive privacy preserving and power lens data mining
BRPI0815023A2 (en) * 2007-08-09 2015-03-10 Koninkl Philips Electronics Nv Method and device for creating a modified video from an input video, and, camcorder.
JP4996404B2 (en) * 2007-09-20 2012-08-08 技研トラステム株式会社 Human behavior search device
FR2925705A1 (en) * 2007-12-20 2009-06-26 Thomson Licensing Sas Image capture assisting device
JP4842919B2 (en) * 2007-12-20 2011-12-21 富士フイルム株式会社 Display device, photographing device, and display method
JP5117424B2 (en) * 2008-03-31 2013-01-16 富士フイルム株式会社 Image processing system, image processing method, and program
JP4909315B2 (en) * 2008-06-05 2012-04-04 日本電信電話株式会社 Video processing apparatus and method, program, and computer-readable recording medium
JP5106271B2 (en) * 2008-06-27 2012-12-26 キヤノン株式会社 Image processing apparatus, image processing method, and computer program
JP5066037B2 (en) 2008-09-02 2012-11-07 株式会社日立製作所 Information processing device
JP5096299B2 (en) * 2008-12-04 2012-12-12 トヨタ車体株式会社 Dust floating locus information creation method, creation program, and creation system
JP5213123B2 (en) * 2009-01-15 2013-06-19 株式会社日立製作所 Video output method and video output device
US9405970B2 (en) * 2009-02-02 2016-08-02 Eyesight Mobile Technologies Ltd. System and method for object recognition and tracking in a video stream
US8253564B2 (en) * 2009-02-19 2012-08-28 Panasonic Corporation Predicting a future location of a moving object observed by a surveillance device
JP4748250B2 (en) 2009-02-27 2011-08-17 ソニー株式会社 Image processing apparatus, image processing system, camera apparatus, image processing method, and program
JP4640515B2 (en) * 2009-03-25 2011-03-02 日本ビクター株式会社 Thumbnail generation device, thumbnail generation method, and thumbnail generation program
US8818025B2 (en) * 2010-08-23 2014-08-26 Nokia Corporation Method and apparatus for recognizing objects in media content
JP5863195B2 (en) * 2012-11-14 2016-02-16 日本電信電話株式会社 Video playback device, video playback method, and video playback program
JP6234164B2 (en) * 2013-10-25 2017-11-22 キヤノン株式会社 Image reproducing apparatus, method and program
JP6257337B2 (en) * 2014-01-14 2018-01-10 日本放送協会 Image selection device
EP3023987B1 (en) * 2014-11-20 2017-03-22 Axis AB Method and apparatus for visualizing information of a digital video stream
KR20180056656A (en) * 2015-09-25 2018-05-29 퀄컴 인코포레이티드 Systems and methods for video processing
WO2019078164A1 (en) * 2017-10-16 2019-04-25 日本電気株式会社 Search device, terminal device, analysis device, search method, method for operating terminal device, analysis method, and program

Also Published As

Publication number Publication date
JP2005210573A (en) 2005-08-04

Similar Documents

Publication Publication Date Title
Bobick et al. The recognition of human movement using temporal templates
Flickner et al. Query by image and video content: The QBIC system
Santella et al. Gaze-based interaction for semi-automatic photo cropping
US6829432B2 (en) Face image photographic apparatus and face image photographic method
US8009921B2 (en) Context dependent intelligent thumbnail images
US9818024B2 (en) Identifying facial expressions in acquired digital images
US8903200B2 (en) Image processing device, image processing method, and image processing program
CN101510257B (en) Human face similarity degree matching method and device
EP3086206B1 (en) Method, apparatus and computer program product for providing gesture analysis
US8189927B2 (en) Face categorization and annotation of a mobile phone contact list
US9336456B2 (en) Systems, methods and computer program products for identifying objects in video data
US8358902B2 (en) Apparatus, method and program for image search
US8768006B2 (en) Hand gesture recognition
US20150067600A1 (en) Classification And Organization Of Consumer Digital Images Using Workflow, And Face Detection And Recognition
US8311277B2 (en) Method and system for video indexing and video synopsis
JP5050075B2 (en) Image discrimination method
US9495583B2 (en) Organizing images by correlating faces
JP2010520542A (en) Illumination detection using a classifier chain
JP5318115B2 (en) Image classification by location
JP2010045819A (en) Face detecting camera and method
JP4765732B2 (en) Movie editing device
JP5081922B2 (en) Apparatus and method for generating photorealistic image thumbnails
JP3615657B2 (en) Video search method and apparatus, and recording medium
JP2011154687A (en) Method and apparatus for navigating image data set, and program
JP2007074731A (en) System, method, and program for supporting monitoring of three-dimensional multi-camera video

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20061012

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20080715

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20080728

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110815

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110815

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110815

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120815

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120815

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130815

Year of fee payment: 5

LAPS Cancellation because of no payment of annual fees