US8971633B2 - Content retrieval to facilitate recognizing content details of a moving image - Google Patents

Content retrieval to facilitate recognizing content details of a moving image Download PDF

Info

Publication number
US8971633B2
US8971633B2 US13/639,012 US201113639012A US8971633B2 US 8971633 B2 US8971633 B2 US 8971633B2 US 201113639012 A US201113639012 A US 201113639012A US 8971633 B2 US8971633 B2 US 8971633B2
Authority
US
United States
Prior art keywords
content
retrieval
image
images
timeline
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US13/639,012
Other languages
English (en)
Other versions
US20130170753A1 (en
Inventor
Kazumasa Tanaka
Kenji Tanaka
Yoshihiro Takahashi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Assigned to SONY CORPORATION reassignment SONY CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TAKAHASHI, YOSHIHIRO, TANAKA, KAZUMASA, TANAKA, KENJI
Publication of US20130170753A1 publication Critical patent/US20130170753A1/en
Application granted granted Critical
Publication of US8971633B2 publication Critical patent/US8971633B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/102—Programmed access in sequence to addressed parts of tracks of operating record carriers
    • G11B27/105—Programmed access in sequence to addressed parts of tracks of operating record carriers of operating discs
    • G06F17/30244—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5854—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73—Querying
    • G06F16/738—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
    • G06F16/785—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content using colour or luminescence
    • G06F17/30256—
    • G06F17/30802—
    • G06F17/3084—
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34—Indicating arrangements 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/482—End-user interface for program selection
    • H04N21/4828—End-user interface for program selection for searching program descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
    • H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/91—Television signal processing therefor
    • H04N5/93—Regeneration of the television signal or of selected parts thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/16—Analogue secrecy systems; Analogue subscription systems
    • H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309—Transmission or handling of upstream communications
    • H04N7/17318—Direct or substantially direct transmission and handling of requests

Definitions

  • the present invention relates to an apparatus and a method of content retrieval and a program, and in particular, an apparatus and a method of content retrieval and a program with which content details of a moving image can be easily recognized by displaying a retrieval result for each of a plurality of images.
  • retrieval of content can be performed as an individual image being an image to be retrieved, it is not possible to perform retrieval of a part where a plurality of images is sequentially displayed, comparison of a temporal ratio of displaying a desired image in the content with a temporal ratio of displaying another image, and the like.
  • the present invention is devised to facilitate recognition of content details of a moving image by displaying retrieval results respectively for a plurality of images.
  • An aspect of the present invention is a content retrieval apparatus, including: retrieval image input means which receives input of a plurality of retrieval images being still images; image feature quantity extracting means which extracts image feature quantity of each of the plurality of retrieval images with a predetermined method; similarity degree calculation means which calculates a similarity degree of content of a moving image between the image feature quantity previously extracted from a still image of a frame structuring the content and the image feature quantity of each of the retrieval images; specifying means which specifies a frame of the still image being similar to each of the retrieval images by comparing the calculated similarity degree with a predetermined threshold value; and timeline displaying means which generates display data for performing timeline displaying by displaying the content on a time axis along with information indicating a temporal position of the frame of the still image being similar to each of the retrieval images based on the specified frame and metadata which is previously extracted from the content.
  • the timeline displaying means may generate display data for displaying a screen having a retrieval image display unit which displays a list of the plurality of retrieval images and a timeline display unit which displays the content on a timeline, the plurality of retrieval images may be displayed at the retrieval image display unit while being surrounded respectively by display frames in mutually different colors, and information indicating a temporal position of the frame of the still image being similar to each of the retrieval images may be displayed at the timeline display unit as a color bar in the same color as the color of the display frame.
  • the timeline displaying means may display each of a plurality of pieces of content with the timeline displaying at the timeline display unit on a single screen, and information indicating a temporal position of the frame of the still image being similar to each of the retrieval images for each of the plurality of pieces of content may be displayed as a color bar in the same color as the color of the display frame.
  • the timeline displaying means may generate display data for displaying a screen further having a moving image display unit which displays a moving image obtained by reproducing the content in a case that reproduction of the content displayed with the timeline displaying is instructed.
  • the timeline displaying means may specify scenes structuring the content based on information of scene change points of the content included in the metadata, and display the content on a timeline in such a manner that figures indicating the respective scenes are displayed while being associated with temporal length of the scenes, with the horizontal or vertical direction of a screen being the time axis.
  • the timeline displaying means may display the content on a timeline while the figures indicating the scenes are displayed respectively in representative color of the scenes specified with a predetermined method.
  • An aspect of the present invention is a content retrieval method, including the steps of: receiving input of a plurality of retrieval images being still images with retrieval image input means; extracting image feature quantity of each of the plurality of retrieval images with a predetermined method with image feature quantity extracting means; calculating a similarity degree of content of a moving image between the image feature quantity previously extracted from a still image of a frame structuring the content and the image feature quantity of each of the retrieval images with similarity degree calculation means; specifying a frame of the still image being similar to each of the retrieval images by comparing the calculated similarity degree with a predetermined threshold value with specifying means; and generating display data for performing timeline displaying by displaying the content on a time axis along with information indicating a temporal position of the frame of the still image being similar to each of the retrieval images based on the specified frame and metadata which is previously extracted from the content with timeline displaying means.
  • An aspect of the present invention is a program for causing a computer to function as a content retrieval apparatus, the apparatus including: retrieval image input means which receives input of a plurality of retrieval images being still images; image feature quantity extracting means which extracts image feature quantity of each of the plurality of retrieval images with a predetermined method; similarity degree calculation means which calculates a similarity degree of content of a moving image between the image feature quantity previously extracted from a still image of a frame structuring the content and the image feature quantity of each of the retrieval images; specifying means which specifies a frame of the still image being similar to each of the retrieval images by comparing the calculated similarity degree with a predetermined threshold value; and timeline displaying means which generates display data for performing timeline displaying by displaying the content on a time axis along with information indicating a temporal position of the frame of the still image being similar to each of the retrieval images based on the specified frame and metadata which is previously extracted from the content.
  • input of retrieval images being still images is received, image feature quantity of each of the plurality of retrieval images is extracted with a predetermined method, a similarity degree of content of a moving image between the image feature quantity previously extracted from a still image of a frame structuring the content and each image feature quantity of the retrieval images is calculated, a frame of the still image being similar to each of the retrieval images is specified by comparing the calculated similarity degree with a predetermined threshold value, and display data for performing timeline displaying by displaying the content on a time axis along with information indicating a temporal position of the frame of the still image being similar to each of the retrieval images is generated based on the specified frame and metadata which is previously extracted from the content.
  • FIG. 1 is a block diagram illustrating a structural example of a content processing apparatus according to an embodiment of the present invention.
  • FIG. 2 is a view illustrating an example of a screen displayed based on display data output from an output unit of FIG. 1 .
  • FIG. 3 is a view illustrating an example of a screen when content is reproduced at the screen illustrated in FIG. 2 .
  • FIG. 4 is a view illustrating an example of a screen after time further passes from the state illustrated in FIG. 3 .
  • FIG. 5 is a flowchart illustrating an example of a content displaying process.
  • FIG. 6 is a view illustrating an example of a case of performing retrieval at the screens of FIGS. 2 to 4 .
  • FIG. 7 is a view illustrating an example of a screen on which a retrieval result is displayed at the screen illustrated in FIG. 6 .
  • FIG. 8 is a view illustrating an example of a screen in a case of performing retrieval for a plurality of pieces of content.
  • FIG. 9 is a view illustrating an example of a screen on which a retrieval result is displayed at the screen illustrated in FIG. 8 .
  • FIG. 10 is a view illustrating an example of a screen when content is reproduced at the screen illustrated in FIG. 9 .
  • FIG. 11 is a flowchart illustrating a retrieval result displaying process.
  • FIG. 12 is a view illustrating another example of a screen on which a retrieval result is displayed.
  • FIG. 13 is a block diagram illustrating a structural example of a personal computer.
  • FIG. 1 is a block diagram illustrating a structural example of a content processing apparatus according to an embodiment of the present invention.
  • a content processing apparatus 10 performs timeline displaying of an input moving image content and performs timeline displaying of a retrieval result as retrieving a predetermined image in the content as required.
  • the content processing apparatus 10 is provided with a content input unit 21 , a metadata extracting unit 22 , an image feature quantity extracting unit 23 , a metadata database 24 , and an image feature quantity database 25 . Further, the content processing apparatus 10 is provided with a retrieval image input unit 26 , a retrieving unit 27 , and an output unit 28 .
  • the content input unit 21 receives input of content data.
  • the content is a moving image content and includes audio, a caption and the like as required.
  • the content is content which is edited as a part of a program to be broadcasted.
  • the metadata extracting unit 22 analyzes data of the content supplied from the content input unit 21 and extracts metadata from the content.
  • the metadata denotes information regarding scene change points, information regarding time to be required for performing later-mentioned timeline displaying, information of representative images and the like of the content.
  • the representative images of the content included in the metadata are set to be an image (still image) of a frame corresponding to a scene having a largest audio signal level among respective scenes structuring the content.
  • the representative image is to be extracted with a previously determined method.
  • Information such as data of the representative image and a frame number of the frame corresponding to the representative image is also extracted by the metadata extracting unit.
  • the metadata extracted by the metadata extracting unit is to be stored in the metadata database 24 as being associated with content identification information.
  • the image feature quantity extracting unit 23 is configured to analyze the content data supplied from the content input unit 21 and to extract image feature quantity from the content.
  • the image feature quantity denotes information to be used for obtaining a similarity degree to a retrieval image in a process of the retrieving unit 27 described later.
  • the image feature quantity denotes information and the like in which still images by the amount of one frame structuring the content are divided respectively into a plurality of predetermined areas as providing a representative color to each area.
  • the image feature quantity may be information of histogram of pixel values of still images by the amount of one frame, and the like.
  • the image feature quantity extracted by the image feature quantity extracting unit 23 is to be stored in the image feature quantity database 25 as being associated with the content identification information and the frame number, for example.
  • the image feature quantity extracting unit 23 is configured to extract image feature quantity similarly from the retrieval image input from the retrieval image input unit 26 .
  • the retrieval image input unit 26 is configured to receive input of a retrieval image being a still image.
  • the retrieval image is image which is arbitrarily selected by a user and is input for retrieving an image which is similar to the retrieval image out of the content input from the content input unit 21 .
  • the retrieving unit 27 compares the image feature quantity of the retrieval image extracted by the image feature quantity extracting unit 23 and the image feature quantity stored in the image feature quantity database 25 with a predetermined method. Accordingly, the similarity degree between the image feature quantity of the retrieval image and image feature quantity of each still image by the amount of one frame structuring the content stored in the image feature quantity database 25 is to be calculated as a numerical value.
  • the retrieving unit 27 specifies still images with image feature quantity respectively having a value of the similarity degree to the image feature quantity of the retrieval image being a predetermined threshold value or larger and supplies the frame number and the like of the still images to the output unit 28 .
  • the output unit 28 is configured to read metadata of the content from the metadata database 24 and to generate display data which is required for performing timeline displaying of the content.
  • the output unit 28 reads metadata of the content from the metadata database 24 and generates display data for performing timeline displaying of a retrieval result based on the frame number of the still image supplied from the retrieving unit 27 and the read metadata.
  • the display data output from the output unit 28 is supplied to a display (not illustrated) or the like and is displayed as an image as described later.
  • FIG. 2 is a view illustrating an example of a screen displayed on a display (not illustrated) or the like based on the display data output from the output unit 28 .
  • the screen illustrated in FIG. 2 includes a moving image display unit 51 .
  • the image of content input from the content input unit 21 is to be displayed at the moving image display unit 51 as a moving image.
  • the screen illustrated in FIG. 2 includes a representative image display unit 52 .
  • the screen illustrated in FIG. 2 includes a timeline display unit 53 .
  • the content is displayed on a timeline at the timeline display unit 53 so that the horizontal direction in the drawing corresponds to the time axis of the content. That is, the content is displayed as corresponding to the time axis as the left end of the timeline display unit 53 corresponding to a start time of the content and the right end of the timeline display unit 53 corresponding to an end time of the content.
  • the content timeline displaying denotes to display a specific content so that a user can visually recognize temporal length of the content and temporal positions of inner scenes, frames and the like of the content.
  • the content timeline displaying denotes displaying of the content with some method but not displaying of the content as a moving image or the like as being reproduced.
  • displaying of the content in the timeline displaying denotes that figures, waveforms, marks and the like which symbolize the content or a structural part of the content are to be displayed as being associated with the time axis.
  • the timeline display unit 53 includes a scene display area 71 .
  • Each scene of the content is displayed at the scene display area 71 as a rectangle having width (length) corresponding to temporal length of each scene. That is eight rectangles are displayed at the scene display area 71 , so that the content is recognized as being structured with eight scenes.
  • a start point and an end point of each scene are specified based on the information of the scene change points included in the metadata read from the metadata database 24 , so that the rectangles of the scene display area 71 are displayed.
  • the respective rectangles indicated at the scene display area 71 are to be displayed respectively in a representative color of each scene, for example.
  • the representative color of a scene is to be specified as a color which corresponds to the largest pixel value in the pixel values of the whole frame existing in the scene, or the like, for example.
  • the representative color of a scene may also be specified with a method other than the above. In short, it is only required that a color which is suitable for impression of a scene is set to be the representative color.
  • the representative images are displayed respectively above the scene display area 71 in the drawing. That is, the nine representative images displayed at the representative image display unit 52 are to be displayed at positions respectively corresponding to frame numbers in the content.
  • a representative image displayed at the first row and first column of the representative image display unit 52 is an image of a frame included in the first scene of the content and is displayed as a thumbnail 72 - 1 above the scene display area 71 .
  • broken lines which indicate the frame position are provided to the thumbnail 72 - 1 toward the leftmost rectangle of the scene display area 71 in the drawing.
  • a temporal position of a frame of the thumbnail 72 - 1 is indicated. That is, it is indicated that the representative image displayed at the first row and first col of the representative image display unit 52 is an image of the frame corresponding to the one slightly left center point of the leftmost rectangle of the scene display area 71 in the drawing on the time axis. In this manner, a user can easily recognize the temporal position of the representative image in the content.
  • thumbnail can be displayed as being associated with content displayed on a timeline so that a temporal position of a representative image in content can be recognized.
  • each thumbnail is generated based on data of a representative image frame or is included in metadata as being generated in advance.
  • a thumbnail 72 - 2 is recognized to indicate a representative image displayed at the second row and third column of the representative image display unit 52 and to be an image of a frame included in the second scene of the content.
  • a thumbnail 72 - 3 is recognized to indicate a representative image displayed at the third row and third column of the representative image display unit 52 and to be an image of a frame included in the third scene of the content.
  • thumbnails 72 - 4 to 72 - 9 are displayed and thumbnails respectively indicating nine representative images being displayed at the representative image display unit 52 are to be displayed above the scene display area 71 .
  • the respective thumbnails are displayed as being alternately superimposed.
  • the thumbnail 72 - 1 is superimposed to the thumbnail 72 - 2 and a part of the image is hidden.
  • the thumbnail superimposed above is displayed with 50% of transparence enabling a thumbnail superimposed below to be visible therethrough.
  • buttons 81 to 83 illustrated in FIG. 2 is structured as components of GUI. For example, owing to that the button 83 is operated, the thumbnails 72 - 1 to 72 - 9 are to be displayed respectively above the scene display area 71 . That is, the screen illustrated in FIG. 2 is displayed at first in a state that the thumbnails 72 - 1 to 72 - 9 are not displayed, and the thumbnails 72 - 1 to 72 - 9 are to be displayed respectively above the scene display area 71 when the button 83 is operated.
  • the button 81 is a button for displaying a moving image at the moving image display unit 51 as reproducing the content.
  • the button 82 is a button for stopping reproduction of the content.
  • a slider 91 is configured to indicate a position of a frame which is currently reproduced at the timeline display unit 53 .
  • FIG. 3 is a view illustrating an example of a screen after a predetermined time is passed at the screen illustrated in FIG. 2 as the content being reproduced with operation of the button 81 .
  • the slider 91 is denoted by a red rectangular frame which is displayed as being superimposed at the scene display area 71 and is displayed to have length in the horizontal direction in the drawing be increased with time.
  • the right end part of the slider 91 indicates the current reproducing position of the content. As illustrated in FIG. 3 , owing to passing of a specific time after the content is reproduced, the right end part of the slider 91 is moved to the right side on the scene display area 71 .
  • FIG. 4 is a view illustrating an example of a screen after further time passing from the state illustrated in FIG. 3 .
  • the right end part of the slider 91 is moved further to the right side on the scene display area 71 with time and is matched with a position of a frame of the representative image corresponding to the thumbnail 72 - 4 .
  • the thumbnail 72 - 4 indicates the representative image which is displayed at the second row and first column of the representative image display unit 52 and an image displayed at the moving image display unit 51 in FIG. 4 is matched with the representative image displayed at the second row and first column of the representative image display unit 52 .
  • the thumbnail 72 - 4 is displayed as being enlarged to have larger display area compared to other thumbnails. Further, at that time, the thumbnail 72 - 4 is displayed as being superimposed on the thumbnail 72 - 3 and the thumbnail 72 - 5 with 0% of transparence. That is, the displaying is performed so that the thumbnail 72 - 3 and the thumbnail 72 - 5 are invisible therethrough.
  • the thumbnail indicating the representative image is highlighted.
  • a display state of a thumbnail is varied with another method.
  • details of the content can be comprehensibly displayed on a timeline. Since the respective scenes of the content and thumbnails indicating the representative images are displayed along the time axis at the timeline display unit 53 , a user can previously recognize what kind of scene can be seen at what timing in a case that the content is reproduced.
  • thumbnails are sequentially highlighted in the timeline display unit 53 while the position of the right end part of the slider 91 is moved in a case that the content is reproduced, a user can evaluate quality of content editing as focusing attention on the representative images, for example.
  • step S 21 the content input unit 21 receives input of the data of the content.
  • step S 22 the metadata extracting unit 22 analyzes the data of the content supplied from the content input unit 21 and extracts metadata from the content.
  • metadata for example, information regarding scene change points, time-related information to be required for performing timeline displaying, information of representative image and the like are extracted as the metadata.
  • the representative images of the content included in the metadata are to be extracted with a previously determined method.
  • Information such as data of the representative images and frame numbers of frames corresponding to the representative images are also extracted by the metadata extracting unit.
  • step S 23 the metadata database 24 stores the metadata extracted in the process of step S 22 .
  • step S 24 the output unit 28 reads the metadata of the content from the metadata database 24 and generates display data which is required for displaying the content on a timeline.
  • the screen as described above with reference to FIG. 2 is displayed.
  • the screen illustrated in FIG. 2 is displayed at first in a state that the thumbnails 72 - 1 to 72 - 9 are not displayed.
  • step S 25 the output unit 28 determines whether or not that thumbnail displaying is instructed and waits until it is determined that the thumbnail displaying is instructed.
  • step S 25 it is determined in step S 25 that the thumbnail displaying is instructed and the processing proceeds to step S 26 .
  • step S 26 the output unit 28 generates display data required for displaying the thumbnails on the screen which corresponds to the display data generated in step S 24 . Accordingly, for example, the thumbnails 72 - 1 to 72 - 9 are displayed respectively above the scene display area 71 .
  • the output unit 28 generates display data for appropriately displaying the screens.
  • the retrieval image input unit 26 is configured to receive input of a retrieval image being a still image.
  • the retrieval image being an image which is arbitrarily selected by a user is input for retrieving an image which is similar to the retrieval image out of the content input from the content input unit 21 .
  • FIG. 6 illustrates an example of a case of performing retrieval at the screens described above with reference to FIGS. 2 to 4 .
  • the drawing illustrates an example of retrieving a retrieval image from the same content as the content in FIGS. 2 to 4 .
  • a retrieval image display area 54 is arranged and images 101 to 103 being retrieval images input via the retrieval image input unit 26 are displayed at the retrieval image display area 54 .
  • three representative images are selected out of the nine representative images displayed at the representative image display unit 52 to be the retrieval images.
  • the image 101 is an image displayed at the second row and second column among the images displayed at the representative image display unit 52 .
  • the image 102 is an image displayed at the first row and third column among the images displayed at the representative image display unit 52 .
  • the image 103 is an image displayed at the first row and first column among the images displayed at the representative image display unit 52 .
  • FIG. 7 is a view illustrating an example of a screen on which a retrieval result is displayed as the button 84 being operated at the screen illustrated in FIG. 6 .
  • outline boxes (display frames) of the images 101 to 103 displayed at the retrieval image display area 54 are displayed in predetermined colors.
  • the predetermined colors are displayed in the drawing with difference of hatching patterns.
  • the outline box of the image 101 is displayed in blue
  • the outline box of the image 102 is displayed in green
  • the outline box of the image 103 is displayed in red.
  • each color bar displayed in FIG. 7 indicates a plurality of frame positions to be a bar having specific width in the horizontal direction in the drawing. That is, in a case of retrieving a still image in a moving image content, a plurality of images having a high similarity degree is continuously detected in general. Accordingly, when the frame positions respectively having a high similarity degree are colored, the color bar is to be displayed.
  • color bars 111 - 1 to 111 - 4 indicate positions of frames respectively having a high similarity degree to the image 101 and is formed to be a color bar in the same color (e.g., blue) as the color of the outline box of the image 101 .
  • color bars 112 - 1 to 112 - 3 indicate positions of frames respectively having a high similarity degree to the image 102 and is formed to be a color bar in the same color (e.g., green) as the color of the outline box of the image 102 .
  • color bars 113 - 1 to 113 - 3 indicate positions of frames respectively having a high similarity degree to the image 103 and are formed to be color bars in the same color (e.g., red) as the color of the outline box of the image 103 .
  • a user can immediately recognize where and how long a part to display a targeted image (retrieval image) exists in the content. Further, it is possible to immediately recognize how respective parts to display a plurality of images (e.g., images 101 to 103 ) are combined in the content.
  • the images 101 to 103 are images of a missile.
  • the image 101 , the image 102 and the image 103 are images before missile launching, at missile launching and after missile launching, respectively.
  • images of the missile before launching, at launching and after launching are repeatedly displayed in many cases.
  • the first position being the first scene is a scene corresponding to the leftmost rectangle of the scene display area 71 . That is, since the images of the missile before launching, at launching and after launching are displayed at parts corresponding to the color bar 111 - 1 , the color bar 112 - 1 and the color bar 113 - 1 , it is recognized that the missile images are displayed at this position.
  • the second position being the fifth scene is a scene corresponding to the fifth rectangle of the scene display area 71 from the left. That is, since the images of the missile before launching, at launching and after launching are displayed at parts corresponding to the color bar 111 - 2 , the color bar 112 - 2 and the color bar 113 - 2 , it is recognized that the missile images are displayed at this position.
  • the third position being the seventh scene is a scene corresponding to the seventh rectangle of the scene display area 71 from the left. That is, since the image of the missile before launching is displayed at a part corresponding to the color bar 111 - 3 , it is recognized that the missile images are displayed at this position.
  • the fourth position being the eighth scene is a scene corresponding to the eighth rectangle of the scene display area 71 from the left. That is, since the images of the missile before launching, at launching and after launching are displayed at parts corresponding to the color bar 111 - 4 , the color bar 112 - 3 and the color bar 113 - 3 , it is recognized that the missile images are displayed at this position.
  • a user can immediately recognize details of the content and can evaluate an editing method of an editor of the content as well, for example.
  • description is performed on an example to perform image retrieval for one content. However, it is also possible to perform image retrieval for a plurality of pieces of content.
  • FIG. 8 is a view illustrating an example of a screen displayed with display data generated by the output unit 28 of the content processing apparatus 10 of FIG. 1 in a case that image retrieval is performed for a plurality of pieces of content.
  • a moving image display unit 151 , a timeline display unit 153 and a retrieval image display area 154 are arranged on a screen of FIG. 8 .
  • seven content are displayed in a timeline at the timeline display unit 153 .
  • the timeline display unit 153 includes scene display areas corresponding to the number of content to be targets of retrieval.
  • the timeline display unit 153 includes scene display areas 171 - 1 to 171 - 7 .
  • Each scene of each content is displayed as a rectangle having width (length) corresponding to temporal length of the each scene at each of the scene display areas 171 - 1 to 171 - 7 .
  • three rectangles are displayed at the scene display area 171 - 1 and the content is recognized as being structured with three scenes.
  • a start point and an end point of each scene are specified based on information of scene change points included in the metadata read from the metadata database 24 , so that rectangles of the scene display areas 171 - 1 to 171 - 7 are displayed.
  • the respective rectangles indicated at the scene display areas 171 - 1 to 171 - 7 are to be displayed respectively in a representative color of each scene (as being displayed in white altogether in the drawing for convenience), for example.
  • the representative color of a scene is to be specified as a color which corresponds to the largest pixel value in the pixel values of the whole frame existing in the scene, or the like, for example.
  • the representative color of a scene may also be specified with a method other than the above. In short, it is only required that a color which is suitable for impression of a scene is set to be the representative color.
  • still image display areas 175 - 1 to 175 - 7 are arranged respectively at the left side in the drawing of the scene display areas 171 - 1 to 171 - 7 .
  • Each of images to be displayed at the still image display areas 175 - 1 to 175 - 7 is to be an image of the top frame, a predetermined representative image or the like of each content, for example.
  • text strings described in the drawing above the images displayed at the still image display areas 175 - 1 to 175 - 7 indicate identification information of the respective content, for example.
  • a moving image obtained as reproducing content selected by a user out of the content which are displayed in a timeline at the timeline display unit 153 is displayed at the moving image display unit 151 .
  • a retrieval image having input thereof received by the retrieval image input unit 26 is to be displayed at the retrieval image display area 154 .
  • the retrieval image is not input yet and the retrieve image is not displayed at the retrieval image display area 154 .
  • the retrieval image being an image which is arbitrarily selected by a user is input for retrieving an image which is similar to the retrieval image out of the content displayed in a timeline at the timeline display unit 153 .
  • FIG. 9 is a view illustrating an example of a screen on which a retrieval result is displayed after images 201 to 203 is input as the retrieval images and the button 184 is operated on the screen illustrated in FIG. 8 .
  • outline boxes of the images 201 to 203 displayed at the retrieval image display area 154 are displayed in predetermined colors.
  • the predetermined colors are displayed with difference of hatching difference.
  • each color bar displayed in FIG. 9 indicates a plurality of frame positions to be a bar having specific width in the horizontal direction in the drawing. That is, in a case of retrieving a still image in a moving image content, a plurality of images having a high similarity degree is continuously detected in general. Accordingly, when the frame positions respectively having a high similarity degree are colored, the color bar is to be displayed.
  • a color bar 211 indicates a position of a frame having a high similarity degree to the image 201 and is formed to be a color bar in the same color (e.g., blue) as the color of the outline box of the image 201 .
  • a color bar 212 indicates a position of a frame having a high similarity degree to the image 202 and is formed to be a color bar in the same color (e.g., green) as the color of the outline box of the image 202 .
  • a color bar 213 indicates a position of a frame having a high similarity degree to the image 203 and is formed to be a color bar in the same color (e.g., red) as the color of the outline box of the image 203 .
  • a user can immediately recognize where and how long a part to display a targeted image (retrieve image) exists in the plurality of pieces of content. Further, it is possible to immediately recognize how respective parts to display a plurality of images (e.g., images 101 to 103 ) are combined in the content. Then, it becomes also possible to compare edited details of the respective content, for example, by displaying the retrieval results on a single screen.
  • a targeted image e.g., images 101 to 103
  • FIG. 10 is a view illustrating an example of a screen in a case that the moving image is displayed at the moving image display unit 151 .
  • the scene display area 171 - 3 is selected by a user by using a pointing device (not illustrated) or the like and the content is reproduced as operating the button 181 .
  • a button 181 structured as a component of GUI is a button for displaying a moving image at the moving image display unit 151 as reproducing the content.
  • a button 182 is a button for stopping reproduction of the content.
  • a slider is configured to indicate a position of a frame which is currently reproduced at the timeline display unit 153 .
  • a slider 191 - 3 is displayed.
  • the slider 191 - 3 is denoted by a red rectangular frame which is displayed as being superimposed at the scene display area 171 - 3 and is displayed to have length in the horizontal direction in the drawing be increased with time.
  • the right end part of the slider 191 - 3 indicates the current reproducing position of the content. Owing to passing of a specific time after the content is reproduced, the right end part of the slider 191 - 3 is moved to the right side on the scene display area 171 - 3 .
  • the slider 191 - 3 is displayed.
  • a slider is to be displayed on the scene display area of the content.
  • the retrieval image input unit 26 receives input of the retrieval image.
  • the retrieval image is an image (still image) which is arbitrarily selected by a user and is input for retrieving images similar to the retrieval image out of the content input from the content input unit 21 .
  • the images 101 to 103 of FIG. 6 are input as the retrieval images.
  • step S 52 the image feature quantity extracting unit 23 analyzes the retrieval image input in step S 51 .
  • step S 53 the image feature quantity extracting unit 23 extracts image feature quantity of the retrieval image as a result of the process of Fig. S 51 .
  • step S 54 the retrieving unit 27 compares the image feature quantity of the retrieval image extracted by the image feature quantity extracting unit 23 in the process of step S 53 and the image feature quantity stored in the image feature quantity database 25 with a predetermined method. Accordingly, the similarity degree between the image feature quantity of the retrieval image and image feature quantity of each still image by the amount of one frame structuring the content stored in the image feature quantity database 25 is to be calculated as a numerical value.
  • step S 55 for example, the retrieving unit 27 specifies a frame of a still image having a value of the similarity degree to the image feature quantity of the retrieval image being a predetermined threshold value or larger.
  • step S 56 the retrieving unit 27 notifies the retrieval result to the output unit 28 .
  • the frame number and the like of the still image which is specified as a result of the process of step S 55 are supplied to the output unit 28 .
  • step S 57 the output unit 28 reads metadata of the content from the metadata database 24 and generates display data which is required for displaying the retrieval result on a timeline based on the frame number of the still image supplied in the process of step S 56 and the read metadata. Accordingly, for example, the screen as described above with reference to FIG. 7 or FIG. 9 is displayed.
  • the output unit 28 appropriately generates display data in accordance with operation and the like of GUI on the screen.
  • FIG. 12 is a view illustrating an example of a screen which displays a result of retrieving retrieval images for a certain content.
  • content of a baseball broadcast program is retrieved and displayed on a timeline.
  • images 104 and 105 are displayed at the retrieval image display area 54 .
  • two representative images are selected out of the nine representative images displayed at the representative image display unit 52 to be the retrieval images.
  • outline boxes of the images 104 and 105 displayed at the retrieval image display area 54 are displayed in predetermined colors.
  • the outline box of the image 104 is displayed in blue and the outline box of the image 105 is displayed in green.
  • the predetermined colors are indicated in the drawing with difference of hatching patterns.
  • color bars indicating positions of frames of images which are similar respectively to the images 104 and 105 are displayed as being superimposed to rectangles which correspond to the respective scenes at the scene display area 71 .
  • color bars illustrated in FIG. 12 are formed to be color bars respectively in the same color as the color of the outline box of the image 104 or as the color of the outline box of the image 105 .
  • the image 104 is an image of a pitcher pitching and the image 105 is an image of an outfielder running (jumping to a ball). Accordingly, at positions of the scene display area 71 where the color bars indicating frames of images which are similar to the image 104 are displayed and the color bars indicating frames of images which are similar to the image 105 at the right side thereof, it is recognized that a batted ball flies to an outfield.
  • the content is displayed on a timeline as the horizontal direction in the drawing being associated with the time axis on the display screen due to the content processing apparatus 10 .
  • the abovementioned series of processes may be performed with hardware or may be performed with software.
  • programs configuring the software are installed from a network or a recording medium to a computer which is assembled in dedicated hardware. Further, owing to installing a variety of programs, installing is performed from a network or a recording medium to a general-purpose personal computer 700 capable of performing various functions, for example, as illustrated in FIG. 13 .
  • a central processing unit (CPU) 701 performs a variety of processes in accordance with a program stored in a read only memory (ROM) 702 or a program loaded from a storage unit 708 to a random access memory (RAM) 703 . Further, data and the like necessary for the CPU 701 to perform a variety of processes are appropriately stored in the RAM 703 .
  • ROM read only memory
  • RAM random access memory
  • the CPU 701 , the ROM 702 and the RAM 703 are mutually connected via a bus 704 .
  • An input/output interface 705 is connected to the bus 704 as well.
  • An input unit 706 including a keyboard, a mouse and the like and an output unit 707 including a display such as a liquid crystal display (LCD), a speaker and the like are connected to the input/output interface 705 .
  • the storage unit 708 including a hard disk and the like, a communication unit 709 including a network interface card such as a modem and a LAN card, and the like are connected to the input/output interface 705 .
  • the communication unit 709 performs a communication process via a network including the Internet.
  • a drive 710 is connected to the input/output interface 705 as required and a removal medium 711 such as a magnetic disc, an optical disc, a magnetic-optical disk or a semiconductor memory is appropriately attached thereto. Then, a computer program read from the removal media is installed to the storage unit 708 as required.
  • a removal medium 711 such as a magnetic disc, an optical disc, a magnetic-optical disk or a semiconductor memory is appropriately attached thereto.
  • the programs configuring the software are installed from a network such as the Internet and recording media such as the removal medium 711 .
  • the recording media include the program-recorded ROM 702 and a medium configured with a hard disk provided in the storage unit 708 which are delivered to the user in a state of being previously assembled to the apparatus body.
  • the abovementioned series of processes in the present application includes a process which is performed in parallel or separately not necessarily being sequentially performed as well as a process which is sequentially performed along the described order.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Library & Information Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Television Signal Processing For Recording (AREA)
US13/639,012 2010-04-09 2011-03-30 Content retrieval to facilitate recognizing content details of a moving image Expired - Fee Related US8971633B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2010-090610 2010-04-09
JP2010090610A JP5471749B2 (ja) 2010-04-09 2010-04-09 コンテンツ検索装置および方法、並びにプログラム
PCT/JP2011/058022 WO2011125744A1 (ja) 2010-04-09 2011-03-30 コンテンツ検索装置および方法、並びにプログラム

Publications (2)

Publication Number Publication Date
US20130170753A1 US20130170753A1 (en) 2013-07-04
US8971633B2 true US8971633B2 (en) 2015-03-03

Family

ID=44762686

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/639,012 Expired - Fee Related US8971633B2 (en) 2010-04-09 2011-03-30 Content retrieval to facilitate recognizing content details of a moving image

Country Status (6)

Country Link
US (1) US8971633B2 (zh)
EP (1) EP2560379A4 (zh)
JP (1) JP5471749B2 (zh)
KR (1) KR20130045251A (zh)
CN (1) CN102822827A (zh)
WO (1) WO2011125744A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10318573B2 (en) * 2016-06-22 2019-06-11 Oath Inc. Generic card feature extraction based on card rendering as an image

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102959551B (zh) * 2011-04-25 2017-02-08 松下电器（美国）知识产权公司 图像处理装置
US9762967B2 (en) 2011-06-14 2017-09-12 Comcast Cable Communications, Llc System and method for presenting content with time based metadata
US20130191745A1 (en) * 2012-01-10 2013-07-25 Zane Vella Interface for displaying supplemental dynamic timeline content
KR102184987B1 (ko) 2013-11-15 2020-12-01 엘지전자 주식회사 영상 표시 장치 및 그 동작 방법
US9794599B2 (en) * 2014-04-10 2017-10-17 Telibrahma Convergent Communications Private Limited Method and system for auditing multimedia content
KR102373460B1 (ko) * 2014-09-15 2022-03-11 삼성전자주식회사 디바이스에 객체를 표시하는 방법 및 그 디바이스
JP5818050B1 (ja) * 2015-01-28 2015-11-18 ビックリック株式会社 ステータス判定システム
JP2017016275A (ja) * 2015-06-29 2017-01-19 ヤマハ株式会社 制御方法
US9906820B2 (en) * 2015-07-06 2018-02-27 Korea Advanced Institute Of Science And Technology Method and system for providing video content based on image
JP2018125702A (ja) * 2017-02-01 2018-08-09 富士ゼロックス株式会社 映像制御システム及びプログラム
KR20210108691A (ko) * 2020-02-26 2021-09-03 한화테크윈 주식회사 영상 검색 장치 및 이를 포함하는 네트워크 감시 카메라 시스템
JP7428855B2 (ja) * 2020-03-17 2024-02-07 Ｌｉｎｅヤフー株式会社 動画解析システム、動画解析装置、動画解析方法、およびプログラム
CN112182276A (zh) * 2020-10-12 2021-01-05 云南财经大学 基于图像内容和元数据的天文海量观测数据混合检索方法

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5537528A (en) 1992-05-28 1996-07-16 International Business Machines Corporation System and method for inputting scene information
JP2000251450A (ja) 1999-03-02 2000-09-14 Hitachi Denshi Ltd 画像検出方法
JP2006039753A (ja) 2004-07-23 2006-02-09 Canon Inc 画像処理装置、画像処理方法
JP2006510240A (ja) 2002-11-29 2006-03-23 ソニー・ユナイテッド・キングダム・リミテッド メディア処理システム
US20070043763A1 (en) 2005-08-16 2007-02-22 Fuji Xerox Co., Ltd. Information processing system and information processing method
US20070074244A1 (en) * 2003-11-19 2007-03-29 National Institute Of Information And Communicatio Ns Technology, Independent Administrative Agency Method and apparatus for presenting content of images
JP2007081858A (ja) 2005-09-14 2007-03-29 Fujitsu Ltd スミア補正方法及びスミア補正回路
US20070237360A1 (en) 2006-04-06 2007-10-11 Atsushi Irie Moving image editing apparatus
JP2007323319A (ja) 2006-05-31 2007-12-13 Nippon Telegr & Teleph Corp <Ntt> 類似検索処理方法及び装置及びプログラム
JP2008185626A (ja) 2007-01-26 2008-08-14 Toshiba Corp ハイライトシーン検出装置
US20080256450A1 (en) * 2007-04-12 2008-10-16 Sony Corporation Information presenting apparatus, information presenting method, and computer program
US20080263448A1 (en) * 2007-04-23 2008-10-23 Digital Fountain, Inc. Apparatus and method for low bandwidth play position previewing of video content
US20090083814A1 (en) * 2007-09-25 2009-03-26 Kabushiki Kaisha Toshiba Apparatus and method for outputting video Imagrs, and purchasing system
JP2009088602A (ja) 2007-09-27 2009-04-23 Toshiba Corp 電子機器および表示処理方法
JP2009163644A (ja) 2008-01-09 2009-07-23 Sony Corp 映像検索装置、編集装置、映像検索方法およびプログラム

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3235917B2 (ja) 1993-08-23 2001-12-04 株式会社リコー 画像記録再生装置
JP3683462B2 (ja) 2000-02-21 2005-08-17 シャープ株式会社 動画表示装置

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5537528A (en) 1992-05-28 1996-07-16 International Business Machines Corporation System and method for inputting scene information
JP2000251450A (ja) 1999-03-02 2000-09-14 Hitachi Denshi Ltd 画像検出方法
JP2006510240A (ja) 2002-11-29 2006-03-23 ソニー・ユナイテッド・キングダム・リミテッド メディア処理システム
US20070074244A1 (en) * 2003-11-19 2007-03-29 National Institute Of Information And Communicatio Ns Technology, Independent Administrative Agency Method and apparatus for presenting content of images
JP2006039753A (ja) 2004-07-23 2006-02-09 Canon Inc 画像処理装置、画像処理方法
US20070043763A1 (en) 2005-08-16 2007-02-22 Fuji Xerox Co., Ltd. Information processing system and information processing method
JP2007052564A (ja) 2005-08-16 2007-03-01 Fuji Xerox Co Ltd 情報処理システムおよび情報処理方法
JP2007081858A (ja) 2005-09-14 2007-03-29 Fujitsu Ltd スミア補正方法及びスミア補正回路
US20070237360A1 (en) 2006-04-06 2007-10-11 Atsushi Irie Moving image editing apparatus
JP2007281858A (ja) 2006-04-06 2007-10-25 Omron Corp 動画編集装置
JP2007323319A (ja) 2006-05-31 2007-12-13 Nippon Telegr & Teleph Corp <Ntt> 類似検索処理方法及び装置及びプログラム
JP2008185626A (ja) 2007-01-26 2008-08-14 Toshiba Corp ハイライトシーン検出装置
US20080256450A1 (en) * 2007-04-12 2008-10-16 Sony Corporation Information presenting apparatus, information presenting method, and computer program
US20080263448A1 (en) * 2007-04-23 2008-10-23 Digital Fountain, Inc. Apparatus and method for low bandwidth play position previewing of video content
US20090083814A1 (en) * 2007-09-25 2009-03-26 Kabushiki Kaisha Toshiba Apparatus and method for outputting video Imagrs, and purchasing system
JP2009088602A (ja) 2007-09-27 2009-04-23 Toshiba Corp 電子機器および表示処理方法
JP2009163644A (ja) 2008-01-09 2009-07-23 Sony Corp 映像検索装置、編集装置、映像検索方法およびプログラム

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Office Action issued in corresponding Chinese application No. 2011800173185 issued Oct. 15, 2014 and the English Translation.
Search Report issued in corresponding European application No. 11765644.7 dated Feb. 7, 2014.

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10318573B2 (en) * 2016-06-22 2019-06-11 Oath Inc. Generic card feature extraction based on card rendering as an image
US10878023B2 (en) 2016-06-22 2020-12-29 Oath Inc. Generic card feature extraction based on card rendering as an image

Also Published As

Publication number Publication date
CN102822827A (zh) 2012-12-12
EP2560379A1 (en) 2013-02-20
KR20130045251A (ko) 2013-05-03
JP2011223327A (ja) 2011-11-04
JP5471749B2 (ja) 2014-04-16
US20130170753A1 (en) 2013-07-04
WO2011125744A1 (ja) 2011-10-13
EP2560379A4 (en) 2014-03-12

Similar Documents

Publication Publication Date Title
US8971633B2 (en) Content retrieval to facilitate recognizing content details of a moving image
US9325946B2 (en) Content processing apparatus and method, and program
AU2024202313B2 (en) Video Processing For Embedded Information Card Localization And Content Extraction
US9881215B2 (en) Apparatus and method for identifying a still image contained in moving image contents
CN112740713B (zh) 用于提供多媒体内容中的关键时刻的方法及其电子设备
US20160261929A1 (en) Broadcast receiving apparatus and method and controller for providing summary content service
US20080175486A1 (en) Video-attribute-information output apparatus, video digest forming apparatus, computer program product, and video-attribute-information output method
KR20000009742A (ko) 동영상에서의 특정인물 등장구간 검색 시스템
US20100002137A1 (en) Method and apparatus for generating a summary of a video data stream
KR20080007424A (ko) 콘텐츠 처리 장치 및 콘텐츠 처리 방법, 및 컴퓨터프로그램
US8300894B2 (en) Method for decomposition and rendering of video content and user interface for operating the method thereof
JP4398994B2 (ja) 映像処理装置及び方法
KR102151668B1 (ko) 하이라이트 추출 방법 및 하이라이트 추출 장치
JP2007164253A (ja) 映像中文字認識装置、映像中文字認識方法、プログラム、及び記録媒体
Coimbra et al. The shape of the game
US11417100B2 (en) Device and method of generating video synopsis of sports game
Halin et al. Automatic overlaid text detection, extraction and recognition for high level event/concept identification in soccer videos
JP5600557B2 (ja) コンテンツ紹介映像作成装置およびそのプログラム
Teixeira et al. News browsing system: multimodal analysis
JP2007201815A (ja) 表示装置、再生装置、方法、及びプログラム
JP2009302780A (ja) 特定シーン抽出装置及び特定シーン抽出プログラム
Simoes et al. Automatic key-frame extraction from broadcast soccer videos
El-Saban Automatic Soccer Video Summarization
Bhojne et al. News Video Indexing and Retrieval using Overlay Text

Legal Events

Date Code Title Description
AS Assignment

Owner name: SONY CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TANAKA, KAZUMASA;TANAKA, KENJI;TAKAHASHI, YOSHIHIRO;REEL/FRAME:029063/0001

Effective date: 20120913

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20230303