WO2001003005A1 - Appareil d'enregistrement d'informations de recherche d'images dynamiques et dispositif de recherche d'images dynamiques - Google Patents

Appareil d'enregistrement d'informations de recherche d'images dynamiques et dispositif de recherche d'images dynamiques Download PDF

Info

Publication number
WO2001003005A1
WO2001003005A1 PCT/JP2000/004299 JP0004299W WO0103005A1 WO 2001003005 A1 WO2001003005 A1 WO 2001003005A1 JP 0004299 W JP0004299 W JP 0004299W WO 0103005 A1 WO0103005 A1 WO 0103005A1
Authority
WO
WIPO (PCT)
Prior art keywords
moving image
search
information
images
partial
Prior art date
Application number
PCT/JP2000/004299
Other languages
English (en)
Japanese (ja)
Inventor
Shuichi Watanabe
Original Assignee
Sharp Kabushiki Kaisha
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sharp Kabushiki Kaisha filed Critical Sharp Kabushiki Kaisha
Priority to JP2001508725A priority Critical patent/JP4027662B2/ja
Priority to DE60036288T priority patent/DE60036288T2/de
Priority to CN008121761A priority patent/CN1371502B/zh
Priority to EP00940880A priority patent/EP1195692B1/fr
Priority to US10/019,747 priority patent/US7092040B1/en
Publication of WO2001003005A1 publication Critical patent/WO2001003005A1/fr
Priority to US11/387,772 priority patent/US7884884B2/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/11—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information not detectable on the record carrier
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73—Querying
    • G06F16/738—Presentation of query results
    • G06F16/739—Presentation of query results in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
    • G06F16/786—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content using motion, e.g. object motion or camera motion
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal
    • H04N9/8227—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal the additional signal being at least another television signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/78—Television signal recording using magnetic recording
    • H04N5/782—Television signal recording using magnetic recording on tape
    • H04N5/783—Adaptations for reproducing at a rate different from the recording rate

Definitions

  • the present invention relates to a device for recording information for retrieving a moving image (moving image retrieval information recording device) and a device for retrieving a moving image (moving image retrieval device).
  • the present invention relates to a moving image search information recording device and a moving image search device capable of efficiently searching and extracting a desired scene from a plurality of scenes.
  • the present invention relates to an improvement in such a moving image search technique.
  • ⁇ moving image '' refers to unprocessed moving image data composed of RGB (Red, Green, Blue) signals, and MPEG (Moving Picture Experts Group). It includes both moving image data that has been digitally encoded and encoded according to a predetermined moving image compression method.
  • scene is a unit of video that consists of a series of consecutive image frames.For example, a video obtained from a single shot from the start to the end of recording, or an edited video A series of moving images in which both ends are separated by two editing points in a moving image. This includes cases where the edited part contains special effects such as fades and dissolves.
  • One scene may include a plurality of scenes.
  • the “scene change point” refers to the position on the time axis at which two scenes switch, and the first frame of the scene immediately after the scene is switched is called the “scene change frame”.
  • key frame refers to a characteristic frame used in a search or the like that does not represent each scene or the entire moving image.
  • the moving image data is content composed of image information that changes along a time axis. All of the moving image data can be searched. Therefore, the simplest method for searching for a desired moving image is to look at the entire moving image and select a desired moving image. However, it takes some time to see the entire moving image. Therefore, as the number of moving images registered in the database increases, the time required to extract a desired moving image becomes extremely long. Therefore, this method is not practical.
  • search information representing the content of each moving image are prepared in advance for each moving image. To search for a desired moving image.
  • the information used as the search information includes information added to the image (additional information), information indicating characteristics of the image (characteristic information), and information indicating the outline of the image (summary information).
  • the additional information includes a title or a description prepared for the moving image.
  • the characteristic information includes information representing characteristics of an image, such as frequency of luminance values derived from an image signal or motion information between frames.
  • the summary information includes characteristic information of an image or a scene change point or a key frame determined by an external input.
  • characteristic information is particularly convenient and important.
  • a key frame representing a moving image can be selected using characteristic information. It is easy to digitize the characteristic information, and if the characteristics of a desired moving image are quantified in the form of characteristic information, a moving image having characteristics matching the required characteristic information can be easily extracted.
  • search information refers to this characteristic information.
  • a moving image includes a plurality of scenes.
  • a searcher searches for a moving image in order to find a desired frame or a desired scene included in the moving image from one or more moving images to be searched. Move using search information
  • the basic method for finding a desired frame or scene from an image is to search and record search information for each frame or search information for each scene in advance, and select a frame or scene based on these. It is to do.
  • Japanese Patent Application Laid-Open No. Hei 9-284 7702 discloses that the feature amount of a frame for detecting a scene change point includes a sum of motion vectors for each frame and a plurality of frames constituting the frame. Apparatus and methods for using intra-frame and inter-frame continuity of individual sub-regions are disclosed.
  • Japanese Patent Application Laid-Open No. 7-38842 discloses that the average value of the magnitude of the motion vector in each scene and the color histogram of the image are used as the feature amount of the scene, A control method for displaying a representative image as an image index is disclosed.
  • a device for recording information for searching for moving images (moving image search information recording device) and a device for searching for moving images (moving image search device) were created. Then, it would be as shown below.
  • the assumed conventional moving image retrieval information recording apparatus analyzes input moving image data and divides the data into scenes.
  • An analysis unit 601 for outputting information representing the structure (moving image structure information); and an output unit connected to receive the output of the analysis unit 601 and the input moving image data.
  • image summarization information such as the sum of the motion vectors corresponding to each frame in the scene is generated as search information and output frame units Connected to receive the output of the frame-based search information generation unit 602 and the frame-based search information generation unit 602, Based on a whole scene Scene information search unit 603 for generating and outputting summary information such as the average value of the size of the moving vector as search information, and moving image structure information output from the analysis unit 601
  • the search information for each frame output from the frame-based search information generating unit 602 and the search information for the entire scene output from the scene-based search information generating unit 603 are stored in a predetermined format.
  • one frame can be considered as a unit obtained by temporally dividing a moving image. Therefore, the frame can be called an example of a time division block.
  • an assumed conventional image search device includes a reading unit 701 for reading moving image search information and moving image data from recording medium 606, and a reading unit 701 for reading.
  • a search information management unit 70 that holds each of the supplied moving image search information and receives the request for the search information and outputs the search information in a scene unit or the search information in a frame unit together with the moving image structure information.
  • 2 in response to a scene-based search request, requests the scene-based search information and the moving image structure information from the search information management unit 720, and receives the received scene-based search information and the moving image structure information.
  • a scene-based search execution section 703 for outputting a scene corresponding to the search request based on the search request and outputting information on the detected scene.
  • a search request is provided to the search execution unit 703 in scene units or the search execution unit 704 in frame units. It includes a search control unit 706 for repeating the process of displaying moving image data provided from the management unit 705 as a search result and outputting the search result when the search is completed.
  • the device shown in FIG. 1 operates as follows.
  • the analyzing unit 601 divides the moving image data into scene units.
  • the analyzing unit 600 also outputs moving image structure information.
  • the frame-based search information generation unit 602 generates summary information corresponding to a frame in a scene based on the output moving image structure information and the moving image data.
  • the frame-based search information generation unit 602 further outputs the summary information thus generated to the scene-based search information generation unit 603 and the moving image search information generation unit 604 as search information.
  • the scene-based search information generator 603 generates summary information for the entire scene based on the frame-based summary information provided from the frame-based search information generator 602.
  • the scene-based search information generator 603 further supplies the summary information generated in this way to the scene-based search information generator 603 as search information.
  • the moving image search information generation unit 604 generates the moving image structure information from the analysis unit 601, the frame-based search information generation unit 602 from the frame-unit search information generation unit, and the scene-unit search information.
  • the search information for the entire scene is received from the unit 603, and is arranged in a predetermined format to generate search information corresponding to a moving image.
  • the moving image search information generating unit 604 gives the thus generated search information corresponding to the moving image to the recording unit 605.
  • the recording unit 605 records the original moving image data and the search information provided from the moving image search information generation unit 604 on the recording medium 606. At this time, the recording unit 605 uses a force to add information indicating that the recorded moving image data corresponds to the search information to the moving image data or the search information, to one or both of them, Alternatively, it is recorded separately from moving image data and search information.
  • the conventional search device operates as follows.
  • the operator gives a search instruction to the search control unit 706.
  • the search control unit 706 selects either a scene-based search or a frame-based search in response to the search instruction.
  • the search control unit 706 gives a search request to the search execution unit 703 in scene units.
  • the search execution unit 703 in scene units requests the search information management unit 702 for search information and moving image structure information in scene units. It receives the output search information and moving image information for each scene.
  • the scene-based search execution unit 703 detects a scene that matches the search request given from the search control unit 706 based on the scene-based search information and the moving image structure information. Then, the scene-based search execution unit 703 outputs information on the detected scene to the data management unit 705.
  • the data management unit 705 reads out the corresponding image data from the recording medium 606 via the readout unit 701 based on the information on the scene given from the scene-based search execution unit 703, and performs the search. This is given to the control unit 706.
  • the search control unit 706 displays the image data. The operator looks at the displayed image data, determines the next search policy, and gives the next search instruction to the search control unit 706. Hereinafter, the same processing is repeated.
  • the search ends when the image that the operator wants to retrieve is found.
  • the search result is output from the search control unit 706.
  • search information regarding a scene is generated based on search information in frame units included in the scene. This is because the smallest unit that composes a scene is generally recognized as a frame.
  • the spatial features in each frame obtained by dividing each frame into small areas (screen division block unit) Search information) can be used as search information.
  • these conventional devices have a problem that the spatial characteristics of an image cannot be obtained over the entire scene having a certain length on the time axis.
  • the aerial features of an image over the entire scene are referred to as ⁇ ⁇ search information J in units of space division blocks.
  • the present invention has been made in view of the above-described problems of the related art, and it is an object of the present invention to efficiently perform a moving image search using a new index that accurately represents a feature of a scene. It is an object of the present invention to provide a device for recording moving image search information and a device for searching for a moving image, which enable the search for a moving image.
  • An object of the present invention is to provide a moving image search information recording device and a moving image search device that enable efficient search of a moving image by extracting features. Still another object of the present invention is to efficiently search for a moving image using a new index that accurately represents a feature of a scene, and to reduce the amount of information for search. An object of the present invention is to provide a moving image search information recording device and a moving image search device. Disclosure of the invention
  • a moving image search information recording device includes moving image data and moving image structure information for dividing a moving image represented by the moving image data into one or more partial moving images on a time axis.
  • a search information generating unit that generates search information corresponding to each of one or more partial moving images based on the search information and the moving image structure information together with the corresponding moving image data on a recording medium.
  • the search information generating unit is configured to generate first summary information for generating summary information of at least one spatial partial moving image obtained by spatially dividing each of the one or more partial moving images on the time axis. Including a generating unit.
  • Summarization information is generated from the spatial partial moving images obtained by spatially dividing each of the partial moving images on the axis B.
  • This summary information is newly introduced in the present invention, and is not obtained from a temporal part of a partial moving image on the time axis.
  • This summary information is obtained from the spatial partial video that covers only a part of the image spatially, but over the partial video in time. Therefore, this summary information represents the spatial characteristics of a partial video over the entire time. Using this new index, it is possible to efficiently search for partial moving images that focus on spatial features rather than time.
  • a moving image search device provides a moving image search device for searching for a desired image using search information corresponding to each of one or more partial moving images constituting the moving image.
  • An image search device Search information is associated with the moving image data representing the moving image, and the search information is obtained by spatially dividing each of the one or more partial moving images on the time axis. Includes the summary information of the spatial partial moving image.
  • the moving image search device is a unit for searching for a partial moving image on the time axis which is connected to the information extracting unit for reading and managing the search information and is provided from the outside.
  • a search is performed for searching for a partial moving image on the time axis that matches the first search request using the summary information of the spatial partial moving images included in the search information. It includes a first partial moving image search unit.
  • the summary information obtained from the spatial partial moving images obtained by spatially dividing each of the partial moving images on the time axis is newly introduced in the present invention. Is obtained from a spatial partial video that covers only a part of the video, but on the time axis. Therefore, this summary information represents the spatial characteristics of a partial moving image over the entire time of the partial moving image. Therefore, according to the moving picture search device according to the present invention, it is possible to efficiently perform a search of partial moving pictures, focusing on spatial characteristics rather than time, using the new index.
  • FIG. 1 is a schematic block diagram of an assumed conventional moving image search information recording apparatus.
  • FIG. 2 is a schematic block diagram of an assumed conventional moving image search device.
  • FIG. 3 is a block [ ⁇ ] of the moving image search information recording device according to the embodiment of the present invention.
  • FIG. 4 is a diagram showing a process of decomposing a moving image into scenes in the moving image search information recording device according to one embodiment of the present invention.
  • FIG. 5 is a diagram showing a process of generating search information in units of time-division blocks in the moving image search information recording device according to the embodiment of the present invention.
  • FIG. 6 is a diagram showing a process of generating search information in units of space division blocks in the moving image search information recording device according to one embodiment of the present invention.
  • FIG. 7 is a diagram for explaining the concept of scene-based search information in the moving image search information recording device according to one embodiment of the present invention.
  • FIG. 8 is a diagram for explaining the concept of search information in units of space division blocks in the moving image search information recording device according to one embodiment of the present invention.
  • FIG. 9 is a diagram for explaining the concept of search information in units of space division blocks excluding redundancy in the moving image search information recording device according to one embodiment of the present invention.
  • FIG. 10 is a block diagram of a moving image search device according to one embodiment of the present invention.
  • m 11 is a diagram showing a reference scene in the similarity search using the scene unit search information.
  • FIG. 12 is a diagram showing one to three scenes in the similarity search using the scene unit search information.
  • FIG. 13 is a diagram showing another scene in the similarity search using the scene unit search information.
  • FIG. 14 is a diagram showing a reference scene in a similarity search using space division block unit search information.
  • FIG. 15 is a diagram showing one scene in the similarity search by the space division block unit search information.
  • FIG. 16 is a diagram showing another scene in the similarity search using the space division block unit search information.
  • FIG. 17 is a diagram showing the concept of recording scene-based search information and selectively recording search information in space-divided pronk units in the moving image search information recording device according to one embodiment of the present invention. is there.
  • FIG. 18 is a diagram showing the meaning of the value of the designated division number flag in a table format.
  • FIG. 19 shows the concept of recording the search information in scene units and the hierarchical recording of the search information in space division blocks in the moving image search information recording device according to one embodiment of the present invention.
  • 2 0 is a diagram showing an example of division when dividing a full I ⁇ Ichimu 2 2.
  • Figure 2 1 is a diagram showing an example of division when dividing a frame into two three.
  • FIG. 22 is a diagram showing an example of division when dividing a frame into two.
  • FIG. 23 is a diagram showing an example of division when dividing a frame into 25 frames.
  • FIG. 24 is a diagram showing an example of division when dividing a frame into 2 ⁇ frames.
  • FIG. 25 is a diagram showing a moving image database in which moving image data is divided and stored in scene units.
  • FIG. 3 shows a block diagram of the moving picture search information recording device 40 according to the present embodiment.
  • this moving image search information recording device 40 receives moving image data, analyzes the structure of the moving image, divides the scene into scenes, and outputs moving image structure information such as scene change points.
  • a first search information generation unit 102 for generating and outputting summary information (in units of divided brons), and moving image structure information and moving image data output from the analysis unit 101 are received.
  • a second search information generation unit 103 for generating and outputting summary information of a space division block composed of image data at the same division block position of all frames in the scene as search information regarding the scene; 1 search information Summary information for each frame (time-divided block) in the scene output from the generator 102, and Z or summary for each space-division block in the scene output from the second search information generator 103 With the information as input, summary information for the entire scene is obtained based on one or both of the summary information from the first search information generation unit 102 and / or the summary information from the second search information generation unit 103.
  • a third search information generation unit 104 for generating and outputting, summary information output from the first search information generation unit 102, and a summary information output from the second search information generation unit 103
  • the search information generation unit 105 receives the search information for the moving image and the original moving image data output from the fourth search information generation unit 105, and one or both of them receive the search information.
  • a recording unit 106 for recording search information, correspondence information with moving image data, and the like on a recording medium 107, by adding information corresponding to moving image data or the like or as separate data.
  • the moving image data handled by the apparatus of the present invention is an original image signal such as an RGB signal, MPE Any of image data encoded using a moving picture compression method such as G (Moving Picture Experts Group) may be used. In the present embodiment, for simplicity of description, it is assumed that moving image data is coded.
  • the analysis unit 101 divides a moving image into small temporal units (scenes). For this purpose, the analyzing unit 101 detects a scene change point in the image data. In this example, the analysis unit 101 detects a scene change point by using the frequency information in the frame in the prediction mode of the coded block included in the coded image data.
  • the analysis unit 101 includes a first scene change frame 130, a second scene change frame 132, and a third scene change frame 130 in the moving image 120.
  • the scene change frame 1 3 4 and the fourth scene change frame 1 3 6 are detected.
  • the analyzing unit 101 includes the first scene change frame 130 including the image data between the first scene change frame 130 and the second scene change frame 132).
  • the second scene change frame 132 including the image data between the second scene change frame 132 and the third scene change frame 134)
  • a third scene change frame 13 4 and a third scene change frame 13 4 including image data between the third scene change frame 13 4 and the fourth scene change frame 13 6.
  • Scene 1 4 4 and the fourth scene consisting of the image data from the fourth scene change frame 1 36 to the end of the moving image 120 (including the fourth scene change frame 1 36)
  • the moving image 1 2 0 is divided into 1 4 6.
  • the analysis unit 101 divides the moving image 120 into four scenes ⁇ 40, 144, 144, and 146 in this example.
  • the prayer club 101 is composed of the first scene change frame 130, the second scene change frame 132, the third scene change frame 134, and the fourth scene change frame. It has a function to output information indicating the position of the scene change frame such as 36 in the image data as moving image structure information.
  • first search information generating section 102 includes a certain scene obtained by dividing moving image data based on moving image structure information output from analyzing section 101. The following processing is performed on 160, and each file included in scene 160 is processed. Create frame summary information.
  • Scene 16 includes frames 16A to 16N.
  • the first search information generation unit 102 obtains, for each of the frames 16 2 A to 16 N, the frequency information 1666 A to 16 N of the motion vector in the frame. In addition, the first search information generation unit 102 obtains 1668N from the average value of the motion vectors 1668A in each of the frames 16A to 16N.
  • the frequency information 166, the average value information 168, and the power obtained in this manner constitute the summary information 170 of each frame.
  • Summary information is given as a function of frame number ⁇ (ie, a function of time).
  • This summary information is search information obtained by compressing information distributed in the space of each frame included in the scene 160.
  • Each frame is a unit obtained by temporally dividing an image, and is a type of time-divided block. Therefore, the summary information 170 can be said to be search information in time-division block units.
  • the second search information generating unit 103 receives the moving image structure information (information of the scene change position) output from the analyzing unit 101 and the moving image data.
  • the second search information generation unit 103 generates summary information for the space division block as follows using the moving image structure information and the moving image data.
  • the empty question division block means the following.
  • Each of all the frames in a scene is spatially divided into a plurality of division prounks in the same manner.
  • the whole of the divided blocks at the corresponding positions in all of these frames is called a spatial divided block of the scene. Therefore, if a scene contains n frames and each frame is divided into m divided blocks, the scene is divided into m spatial divided blocks, and each spatial divided block is divided into n spatial divided blocks. Includes divided blocks.
  • second search information generating section 03 performs the following processing on scene 16. As described above, it is assumed that the scene 16 includes a plurality of frames 16 2 A to 16 2. Each of these frames 16 2 A to 16 2 ⁇ is spatially divided into ⁇ divided blocks.
  • the division block refers to each small area when the image plane is divided into a plurality.
  • the frame 16 2 ⁇ shown in FIG. 6 is divided into a plurality of divided blocks 18 OA 1 to 180 AM. Cracked. The same applies to the other frames 16 2 B to 16 2 N.
  • one spatial division bron of the scene 160 is formed by the division block at the same position in each of the plurality of frames 16A to 16N.
  • a set of M-th divided blocks of each of N frames (the number of elements of which is equal to N,...) Forms a spatially-divided block 18 2 -M.
  • Other space division blocks are formed similarly.
  • the summary information 188 is summary information for each space division block, and is a function ⁇ (direct) for the position (coordinates) within the frame of the division block included in each space division block.
  • summary information 188 includes frequency information 184 and average value information 186.
  • frequency information 184 frequency information in a space division block of a prediction mode of each encoding process or frequency information in a space division block of a motion vector at the time of motion compensation prediction encoding can be used.
  • the frequency division block 182-1-1 can obtain frequency information 1884A
  • the space division block 182-2-2 can obtain frequency information 1884B, and so on.
  • the space division block 1 82 2—M force gives frequency information 1 84 M.
  • the average value information 186 the sum total of the motion vectors of the space division block II or the average value can be used.
  • the average value information 1886 A is obtained from the space division block 182-1
  • the average value information 1886B is obtained from the space division block 182-2.
  • the average value information 186 M is obtained from the space division block 182 M.
  • the standard deviation of the motion vector in the space division block can be used as the summary information.
  • the third search information generation unit 104 receives the summary information output from the first search information generation unit 102 or the summary information output from the second search information generation unit 103, Based on one or both of them, summary information for the entire scene is generated and output.
  • the summary information for the entire scene is used when searching for a scene. Search information.
  • frequency information, an average value, a standard deviation, and the like for the entire scene such as a motion vector, a luminance value, or a chrominance value, can be used.
  • the fourth search information generation unit 105 includes the summary information output from the first search information generation unit 102, the summary information output from the second search information generation unit 103, and the third By arranging, in a predetermined format, the summary information output from the search information generation unit 104 and the moving image structure information output from the analysis unit 101 in a predetermined format, the search information corresponding to the moving image is obtained. 3 to provide the generated output to the recording unit 1 0 6
  • the recording unit 106 receives the search information corresponding to the moving image and the original moving image data output from the fourth search information generating unit 105, and receives only the search information or the search information. And the moving image data are recorded on the recording medium 107. At this time, the recording unit 106 adds information indicating that the moving image data corresponds to the search information to either or both of the moving image data and the search information, or Record as separate data from search information.
  • the search information used in the moving image search information recording device 40 according to the present embodiment includes a motion vector included in motion compensated predictive encoded moving image data.
  • the information on the magnitude of the motion obtained based on the information is used.
  • search information in units of time-division blocks (frames) is obtained for each frame in the scene by using the motion vector of the entire frame obtained from the motion vector added to each encoded block. It is calculated as the sum of absolute values.
  • the other can be easily obtained from one of the sum of the absolute values of the motion vectors and the average thereof.
  • the search information for each space division block the sum of the absolute values of the motion vectors in each space division block is used. Again, the average of the motion vectors can be used as search information instead of the sum. If each of the divided blocks that compose the space division block matches the coding block, the corresponding The number of motion vectors is 1. When each divided block includes a plurality of coding blocks, the number of motion vectors corresponding to each divided block is equal to the number of coding blocks included in the divided block.
  • the search information for each space division block may be a value obtained by dividing the total sum of the absolute values of the motion vectors in each space division block by the number of division blocks constituting the space division block.
  • the number of divided blocks in this case is equal to the number of frames constituting the scene.
  • the search information for each space division block is obtained by calculating the sum of the absolute values of the motion vectors in each space division block by the number of coded blocks included in the division blocks constituting the space division block. It can be a value divided by the sum.
  • the sum of the absolute values of the motion vectors is used as the search information for the time-division block or the space-division block
  • the total sum of the absolute values of all the motion vectors included in the scene can be calculated.
  • the sum thus obtained is used as search information in scene units.
  • a scene 16 is composed of frames 16A to 16N. From these sums of the absolute values of the motion vectors of the frames 16A to 16N, the scene unit search information I of the scene 160 can be obtained.
  • each of the frames 162A to 162N is divided into 2 ⁇ 2 divided blocks. That is, each of the frames 16 2 A to 16 2 N is divided into four divided blocks. For example, frame 16 2 A is divided into four divided blocks 19 O A 1 through 190 A 4. The same applies to the following frames, although not shown.
  • frame 162B is divided into frame 190B4 and frame 190B4. Therefore, as can be seen from the description of the space division block above, the scene 160 is divided into four space division blocks, and the motion vectors in each space division block are assigned to these four space division blocks. From the sum of the absolute values of, the search information I and, respectively, for each space division block can be obtained as real values.
  • scene search information for scene 160 for example, It is possible to use ⁇ I, I I 2 , I a. I 4 ⁇ obtained by sequentially arranging the rope information I, I I 2 , I 3, I 4 . This search information may be recorded together with the original image data.
  • the above search information includes redundancy.
  • the scene unit search information I is equal to the sum of the search information in space division block units.
  • the scene unit retrieval information I can from the search information I space division Proc Unit I 4 KaraMotome Mel. Or may be from any three and each scene search information I search information I, from among the 1 4 space division blocks, calculates a remaining one of the search information of the spatial division Proc units.
  • the apparatus of the present embodiment by about 1 4 is not included in the search information Unisuru.
  • the value of each space division block unit search information with respect to the scene unit search information is shown as a percentage instead of the value of each space division block unit search information as it is.
  • the first to third space division blanking opening Tsu retrieval information click of, if P have P 2 and P 3 the ratio retrieval information I of scene unit retrieval information of spatial split block units at this time is 9 Is determined as shown in That is, the search information Pi of the first space division block is calculated from the first space division block and the scene unit search information. From the second space division block and the scene unit search information, search information P2 of the second space division trunk is calculated.
  • the search information of the space division block is recorded as a percentage with respect to the scene unit search information
  • the following advantages can be obtained.
  • the dynamic range of the signal for recording the search information is limited, so that the search information can be more efficiently recorded. That is, the amount of information to be recorded is reduced as compared with the case where it is not.
  • the value of the search information is normalized, it is possible to compare the search information attached to different scenes with each other.
  • the moving image search information is generated by the fourth search information generation unit 105, and is recorded on the recording medium 107 via the recording unit 106.
  • moving image search device 50 includes a reading unit 501 for reading moving image data recorded on recording medium 107 and its search information.
  • the search information of the moving image read by the reading unit 501 is held, and the search information of the scene unit, the search unit of the space division block, or the search information of the time division block is stored according to the search request.
  • a request is made to the search information management unit 502 to output the information together with the information, and the entire scene is sent from the search information management unit 502 to the search information management unit 502.
  • a first search execution unit 5-3 for detecting a scene that matches a search request for each scene, and outputting information about the detected scene, Respond to inter-block search request Then, a request is made to the search information management unit 502, and the search is performed based on the summary information for each space division block in the scene received from the search information management unit 502 and the moving image structure information.
  • a second search execution unit 504 for detecting scenes matching the request and outputting information about the detected scenes, and a search information management unit 502 in response to a search request in units of time division blocks. A scene or frame that matches the search request based on the summary information for each frame in the scene received from the search information management unit 502 and the moving image structure information.
  • a third search execution unit 505 for detecting information and outputting information on the detected scene or frame, and holding the moving image information read by the readout unit 501 to execute the first search Part 503, second search execution part 504 or third
  • a data management unit 506 for outputting image data of a scene or a frame detected based on the search result based on the search result output from the search execution unit 505;
  • a search request in scene units is sent to the first search execution unit 503, a search request in space division block units is sent to the second search execution unit 504, or a time division block unit.
  • a search control unit 507 for displaying the image data of the scene or frame output from the data management unit 506 and outputting the search result at the end of the search.
  • the moving image search device 50 operates as follows. At the start of the search, the operator gives a search instruction to the search control unit 507.
  • the search control unit 507 selects one of a scene unit search, a space division block unit search, and a time division block unit search according to the search instruction, and executes the first search.
  • a search request is issued to any one of the section 503, the second search execution section 504, or the third search execution section 505.
  • the one that receives the search request from the search control unit 507 is the search information management unit. ⁇ ⁇ ⁇ . Detects scenes or frames that match the search request, based on the video structure information provided from Part 502 and the summary information in scene units, space-division-broken units, or time-division block (frame) units Then, information about the detected scene or frame is provided to the data management unit 506.
  • the data management unit 506 corresponds to the scene or frame information received from the first search execution unit 503, the second search execution unit 504, or the third search execution unit 505.
  • An image of a scene or a frame to be extracted is extracted from the moving image data and provided to the search control unit 507.
  • the search control unit 507 displays the image provided from the data management unit 506. From the displayed image, the operator determines whether or not the image is a desired scene, and gives an instruction to the search control unit 507 to output a search result if the image is a desired scene. The search control unit 507 outputs a search result in response to this instruction. If the image displayed by the search control unit 507 does not represent a desired scene, the operator further gives the next search instruction to the search control unit 507. Hereinafter, the search control unit 507 performs the same operation as described above, and displays the scene of the search result. Hereinafter, this operation is repeated until the search is completed.
  • the search information is based on a motion vector included in motion-compensated prediction-coded video data.
  • the information about the magnitude of the movement obtained by using is used.
  • the desired scene or frame within the scene is retrieved as follows.
  • a scene in which motion is spatially ubiquitous over one scene and a scene in which motion is distributed relatively evenly in space can be selected.
  • search information in units of space division blocks which have not been considered in the past, is used for scene search. Therefore, the following effects can be obtained.
  • FIGS. 11 to 13 scene A shown in FIG. 12 (A) and scene B shown in FIG. 13 (A) are referred to as a reference scene shown in FIG. 11 (A).
  • a reference scene shown in FIG. 11 (A) For the case of calculating the similarity of With such “similarity” calculation, it is possible to detect a scene having motion characteristics similar to the reference scene.
  • FIGS. 12 to 13 show a case where only search information in scene units is used.
  • FIGS. 11 to 13 show scene unit search information for each image shown in (A).
  • scenes A and B have the same motion intensity as the entire scene with respect to the reference scene. Therefore, as shown in FIGS. 11 to 13 (B), the corresponding scene unit search information has the same value. That is, both scenes A and B are similar to the reference scene. Is determined to be.
  • FIG. 14 to FIG. 16 show the reference scenes, scenes A and B, respectively.
  • each frame of each scene includes four divided blocks.
  • the search information for each space division block is given to the space division block consisting of the first to third division blocks.
  • the information P, P 2 and P 3 have been calculated.
  • the scene-based search information for the reference scene, scenes A and B has substantially the same value.
  • the following facts can be understood by comparing each one with search information in units of space division blocks. That is, the scene A has a distribution similar to the distribution of the space-division block unit search information of the reference scene. This is because, in scene B, since the person is on the right side, the motion can be spatially biased. In both the reference scene and scene A, the person is almost in the center. Therefore, scene A is detected as a scene similar to the reference scene, but scene B is not detected. In other words, even if the motion of the entire scene is similar to the scenes of the scenes A and B, the scene can be selected based on whether or not the motion is ubiquitous.
  • search information in units of frames for example, particularly fast-moving frames in a scene, or frames in which motion stops, can be selected.
  • search information in units of scenes By combining the above-described search information in units of scenes, search information in units of space division blocks, and search information in units of time division blocks, it is possible to more precisely search for a desired scene.
  • Video search can be performed.
  • the analyzing unit 101 automatically finds a scene change point by performing signal analysis on moving image data.
  • the operator may look at the moving image, analyze the structure of the moving image based on the subjectivity of the operation, and attach a tag to the scene division position.
  • the analysis unit 101 analyzes the moving image data in the same manner, but at that time, divides the moving image into scenes by looking at the tags attached to the moving image data
  • the intra-frame frequency information of the motion vector at the time of motion compensation prediction encoding is used as the frequency information ⁇ 66 of the summary information of the frame.
  • the present invention is not limited to this, and intra-frame frequency information of the prediction mode of each encoded block can be used as frequency information.
  • the sum or standard deviation of the motion vectors in the frame, the average luminance value in the frame calculated using the original image signal or the decoded image signal, or the average color difference value, etc. May be used.
  • Search methods and procedures are not limited to the combinations shown here. It is possible to change the order in which the selected candidates are narrowed down. Also, the method and procedure for searching vary depending on the generated search information.
  • the number of divisions is not limited to four.
  • the space division blocks search information expressed as a percentage is a gamma 1 5 from P.
  • P 16 is obtained more subtracting the ⁇ !] Search information of other space division blocks 1 0 0%.
  • space division block unit search information expressed as a percentage is from P fi 3 .
  • P64 is also obtained by subtracting the sum of the search information of other space division blocks from 100%.
  • a space division block is formed by the division blocks divided according to the selected number (which can be called “resolution”), and search information is obtained for each of them. Furthermore, when recording search information, in addition to the search information in scene units and the search information in space division proxies based on the selected resolution, the division number designation is shown in Fig. 18 showing the correspondence between the value and the division number. Search flags Add to information. If the division number designation flag is “0”, ⁇ P is P 2 , P 3 ⁇ ,
  • a database that requires only a relatively simple search such as 2 ⁇ 2 is used for a database that requires only a relatively simple search, because it is necessary to know the approximate distribution of motion in the upper, lower, left, and right images.
  • the amount of search information recorded can be reduced.
  • High-performance search can be realized. Since high-resolution search information includes low-resolution search information, there is no need to duplicate low-resolution search information when low-resolution search is required.
  • the number of divisions in the vertical direction is equal to the number of divisions in the horizontal direction, such as 2 ⁇ 2, 4 ⁇ 4, and 8 ⁇ 8, and both are powers of two. If this was the number of divisions selectively used was, even between the resolution of the partition has been recorded different spatial division Proc unit search information database that Do possible correspondence between the retrieval information u also in different division resolution each other Can also be compared. Further, as shown in FIG. 19, search information in units of space division blocks having a plurality of resolutions may be recorded in a hierarchical manner.
  • the search information of a scene is search information 200 of a scene unit, search information 202 of a space division block unit by 2 ⁇ 2 division, search information 204 of a space division block unit by 4 ⁇ 4 division, and 8 ⁇ 8 minutes. And search information 206 in units of space division blocks by division.
  • search information in units of space division blocks in a certain layer is represented by search information in units of four spatial blocks with higher resolution.
  • each layer Te smell, without recording the one of retrieval information of four spatially split block units, 3 can be obtained all the information necessary for the search
  • the search information in the example shown in Fig. 19 is divided into 8 x 8 frames shown in Fig. 7 Search information having the same resolution as that for recording search information in units of a space division block formed as a unit.
  • the search information shown in FIG. 19 has an advantage that search information of another resolution can be easily obtained.
  • the frame dividing method is not limited to this.
  • the search information may be generated by dividing the frame in a manner as shown in FIGS. 20 to 24 and using the obtained space division block as a position.
  • search information for realizing a search function similar to that obtained by the apparatus of the above-described embodiment can be generated.
  • the summary information is obtained by using the frame 16 2 ⁇ as one time-divided pronk 16 4 ⁇ .
  • one time division block may include two or more frames.
  • each frame may be sub-sampled to generate a sub-frame, and one time division block may include one or a plurality of sub-frames.
  • frames in the scene may be appropriately thinned out, and the summary information for each thinned frame may be used.
  • summary information of the space division block is obtained using all the division blocks included in the space division block.
  • the invention is not limited to such an implementation. It is not always necessary to use the image data corresponding to all the divided blocks in the space division block, and the division information is appropriately thinned out, or the image data in the division block is appropriately thinned out, and the summary information of the space division block is obtained. May be generated.
  • search vectors may be generated by correcting the movement vector corresponding to the movement of the camera.
  • the moving image search information recording device 40 and the moving image search device 50 are separate devices.
  • the search information is passed from the moving image search information recording device 40 to the moving image search device 50 via the recording medium 107.
  • the search information may be passed directly to the search information management unit 502 in FIG. 10 together with the moving image structure information output from the analysis unit 101.
  • the search process in this case is called a so-called real-time search.
  • a divided block obtained by dividing a frame in the above-described embodiment may correspond to an encoded block.
  • a block of an appropriate size different from the coding block may be newly defined.
  • the divided blocks are symmetric, and the arrangement of the divided blocks is also symmetric.
  • the block does not need to be symmetric.
  • the arrangement of the blocks does not need to be symmetric. It is also possible to appropriately thin out the image plane. That is, an image may include a region or an area that does not include a divided block. Also, the divided blocks may partially overlap. That is, an image may include an area included in a plurality of divided blocks.
  • the third search information generation unit 104 shown in FIG. 3 includes the summary information and the second search information generation unit 100 output from the first search information generation unit 1 ⁇ 2. In response to the summary information output from step 3, summary information is generated for the entire scene.
  • the invention is not limited to such an implementation.
  • the third search information generation unit 104 may generate summary information on the entire scene directly from the input image data.
  • search information In the above-described embodiment, various types of summary information (search information) are physically defined. It seeks one defined scene as a unit. However, it is not always necessary to obtain search information in scenes. For example, if a scene is divided into
  • the search information may be obtained in units of ⁇ .
  • the word “scene” also means “subscene”.
  • search information can be obtained in units of a composite scene consisting of multiple scenes.
  • the word “scene” also means “a group of scenes”.
  • search information is generated, recorded, and searched for one piece of moving image data, but the present invention is not limited to such an example. That is, it is assumed that a plurality of pieces of moving image data are to be searched, search information is generated and recorded, and a desired scene or frame is searched from the plurality of pieces of moving image data.
  • each captured moving image that is, one scene from the start of one recording to the stop of the recording is transferred.
  • the moving image data to be represented is generated as independent data files. Therefore, an image database as shown in Fig. 25 can be easily recalled.
  • search information is generated by the moving image search information recording device 40 shown in FIG.
  • the input moving image data is a plurality of moving image data existing in the database. Since the moving image data has already been decomposed into scenes (one moving image data contains only one scene each), moving image structure information is unnecessary. Therefore, it is not necessary for the analysis unit 101 to analyze the structure of the moving image.
  • the first, second, and third search information generators 102, 104, and 104 generate summary information for frames (time-divided blocks), space-divided blocks, and the entire scene from moving image data, respectively. .
  • the fourth search information generation unit 105 arranges the summary information in a predetermined format to generate search information for a moving image.
  • the recording unit 106 receives each of the moving image data and the search information corresponding to each of the moving images, and as a force or separate data for adding information corresponding to each of the moving image data and the search information to one or both of them,
  • the search information and the corresponding information are recorded on the recording medium 1-7.
  • the recording medium 107 may be in the original image database. Further, each search information may be recorded independently or collectively as search information corresponding to the image database.
  • a search is executed from such an image database and the search information generated and recorded by the moving image search device shown in FIG.
  • what is read by the reading unit 501 and managed by the search information management device 502 is search information corresponding to a plurality of moving image data existing in the database, and a moving image corresponding to each moving image data. No image structure information is required.
  • the first, second, and third search execution units 503, 504, and 505 respectively use the corresponding search information to detect a scene or a frame that matches the search request, and detect the detected scene or frame. Outputs information about frames.
  • the data management unit 506 and the search control unit 507 operate in the same manner as in the above-described embodiment, and obtain search results.
  • the moving image search information recording apparatus and the moving image search apparatus according to the tree invention generate scene search information in units of spatially divided blocks obtained by dividing a scene spatially, not temporally, and perform the search. Since a scene is searched based on information, it is suitable for a system that efficiently searches for a desired scene in a moving image according to spatial features observed through the scene.

Abstract

Cette invention se rapporte à un appareil (40) d'enregistrement d'informations de recherche d'images dynamiques, servant à enregistrer des informations de recherche permettant de rechercher une image dynamique représentée par des données d'image dynamique et comprenant une ou plusieurs images dynamiques partielles (140, 142, 144, 146, 160) sur l'axe temporel. A cet effet, cet appareil utilise des sections génératrices d'informations de recherche (102, 103, 104, 105) destinées à générer des informations de recherche correspondant aux images dynamiques partielles et une section d'enregistrement (106) destinée à enregistrer les informations de recherche conjointement avec des informations de correspondance concernant la correspondance entre les informations de recherche et les données d'image dynamique. Ces sections génératrices d'informations de recherche (102, 103, 104, 105) comportent un premier générateur d'informations récapitulatives (103) destiné à générer des informations récapitulatives (188) sur une ou plusieurs images dynamiques partielles spéciales formées par division spatiale de chacune des images dynamiques partielles (140, 142, 144, 146, 160).
PCT/JP2000/004299 1999-06-30 2000-06-29 Appareil d'enregistrement d'informations de recherche d'images dynamiques et dispositif de recherche d'images dynamiques WO2001003005A1 (fr)

Priority Applications (6)

Application Number Priority Date Filing Date Title
JP2001508725A JP4027662B2 (ja) 1999-06-30 2000-06-29 動画像検索装置、および動画像検索情報管理装置
DE60036288T DE60036288T2 (de) 1999-06-30 2000-06-29 Dynamisches bildsuch-informations-aufzeichnungsgerät und dynamische bildsuchvorrichtung
CN008121761A CN1371502B (zh) 1999-06-30 2000-06-29 活动图像检索信息记录装置及活动图像检索装置
EP00940880A EP1195692B1 (fr) 1999-06-30 2000-06-29 Appareil d'enregistrement d'informations de recherche d'images dynamiques et dispositif de recherche d'images dynamiques
US10/019,747 US7092040B1 (en) 1999-06-30 2000-06-30 Dynamic image search information recording apparatus and dynamic image searching device
US11/387,772 US7884884B2 (en) 1999-06-30 2006-03-22 Dynamic image search information recording apparatus and dynamic image searching devices

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP18460699 1999-06-30
JP11/184606 1999-06-30
JP11/339019 1999-11-30
JP33901999 1999-11-30

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US10019747 A-371-Of-International 2000-06-29
US11/387,772 Continuation US7884884B2 (en) 1999-06-30 2006-03-22 Dynamic image search information recording apparatus and dynamic image searching devices

Publications (1)

Publication Number Publication Date
WO2001003005A1 true WO2001003005A1 (fr) 2001-01-11

Family

ID=26502587

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2000/004299 WO2001003005A1 (fr) 1999-06-30 2000-06-29 Appareil d'enregistrement d'informations de recherche d'images dynamiques et dispositif de recherche d'images dynamiques

Country Status (8)

Country Link
US (2) US7092040B1 (fr)
EP (2) EP1195692B1 (fr)
JP (2) JP4027662B2 (fr)
KR (1) KR100486047B1 (fr)
CN (1) CN1371502B (fr)
DE (1) DE60036288T2 (fr)
ES (1) ES2288859T3 (fr)
WO (1) WO2001003005A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005063418A (ja) * 2003-07-18 2005-03-10 Samsung Electronics Co Ltd ＧｏＦ／ＧｏＰの質感表現方法と、これを用いたＧｏＦ／ＧｏＰの検索方法及び装置
WO2009042340A2 (fr) * 2007-09-25 2009-04-02 Motorola, Inc. Procédé pour créer, consommer et partager de façon intelligente du contenu vidéo sur des dispositifs mobiles
US7884884B2 (en) 1999-06-30 2011-02-08 Sharp Kabushiki Kaisha Dynamic image search information recording apparatus and dynamic image searching devices
JP2013149110A (ja) * 2012-01-20 2013-08-01 Renesas Electronics Corp 移動量検出装置、移動体検出装置及び移動量検出方法
JP2013210845A (ja) * 2012-03-30 2013-10-10 Secom Co Ltd 移動物体照合装置

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7075683B1 (en) * 1999-02-15 2006-07-11 Canon Kabushiki Kaisha Dynamic image digest automatic editing system and dynamic image digest automatic editing method
US7996191B2 (en) * 2001-08-10 2011-08-09 Jon Milton Foster Property development utilizing structure depicted in a motion picture or television show
US7167574B2 (en) 2002-03-14 2007-01-23 Seiko Epson Corporation Method and apparatus for content-based image copy detection
JP2004234613A (ja) * 2002-12-02 2004-08-19 Nec Corp 映像記述システムおよび方法、映像識別システムおよび方法
US7532804B2 (en) 2003-06-23 2009-05-12 Seiko Epson Corporation Method and apparatus for video copy detection
US7856054B1 (en) * 2003-11-14 2010-12-21 Apple Inc. Scene change identification during encoding of compressed video
JP4699476B2 (ja) * 2005-10-27 2011-06-08 三菱電機株式会社 映像要約装置
US8009861B2 (en) * 2006-04-28 2011-08-30 Vobile, Inc. Method and system for fingerprinting digital video object based on multiresolution, multirate spatial and temporal signatures
US8174579B2 (en) * 2008-08-22 2012-05-08 Panasonic Corporation Related scene addition apparatus and related scene addition method
JP4725690B2 (ja) 2009-01-23 2011-07-13 日本電気株式会社 映像識別子抽出装置
US8731292B2 (en) * 2011-01-07 2014-05-20 Alcatel Lucent Method and apparatus for comparing videos
CN102595116B (zh) 2011-01-14 2014-03-12 华为技术有限公司 多图像块划分的编解码方法和装置
CN113632493A (zh) * 2019-03-13 2021-11-09 北京字节跳动网络技术有限公司 变换跳过模式中的子块变换
KR102364628B1 (ko) * 2019-07-03 2022-02-18 한국과학기술원 비디오 처리 방법 및 장치
US20220366538A1 (en) * 2019-07-03 2022-11-17 Korea Advanced Institute Of Science And Technology Video processing method and apparatus
US20230171328A1 (en) * 2021-12-01 2023-06-01 Tencent America LLC Method and apparatus for split rendering for lightfield/immersive media using proxy edge cloud architecture

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0546769A (ja) * 1991-08-12 1993-02-26 Nippon Telegr & Teleph Corp <Ntt> 動きベクトル算出方法
JPH06153146A (ja) * 1992-11-04 1994-05-31 Matsushita Electric Ind Co Ltd 動画像のシーンチェンジ検出装置および編集装置
JPH07152779A (ja) * 1993-11-29 1995-06-16 Toshiba Corp 動画インデックス検出処理方法及び動画インデックス検出処理機能を持つ動画処理装置
JPH0983864A (ja) * 1995-09-19 1997-03-28 Kokusai Denshin Denwa Co Ltd <Kdd> 動画像のカット点画像検出装置
US5719643A (en) * 1993-08-10 1998-02-17 Kokusai Denshin Denwa Kabushiki Kaisha Scene cut frame detector and scene cut frame group detector
EP0907147A2 (fr) * 1997-09-26 1999-04-07 Matsushita Electric Industrial Co., Ltd. Procédé d'affichage de séquences vidéo et dispositif d'affichage

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0738842A (ja) 1993-06-29 1995-02-07 Toshiba Corp 動画編集装置
US5805733A (en) * 1994-12-12 1998-09-08 Apple Computer, Inc. Method and system for detecting scenes and summarizing video sequences
US5485611A (en) * 1994-12-30 1996-01-16 Intel Corporation Video database indexing and method of presenting video database index to a user
US5779108A (en) * 1995-06-15 1998-07-14 Calmar Inc. Pressure venting trigger sprayer
US5819286A (en) * 1995-12-11 1998-10-06 Industrial Technology Research Institute Video database indexing and query method and system
JPH09182019A (ja) * 1995-12-26 1997-07-11 Sony Corp 映像信号記録装置及び再生装置
US5661524A (en) * 1996-03-08 1997-08-26 International Business Machines Corporation Method and apparatus for motion estimation using trajectory in a digital video encoder
JPH09284702A (ja) 1996-04-09 1997-10-31 Oki Electric Ind Co Ltd シーン変化フレーム検出方法および装置
DE19618625C1 (de) * 1996-05-09 1997-10-23 Daimler Benz Ag Flüssigkeitsgekühlter Kolben für Verbrennungsmotoren
US5778108A (en) * 1996-06-07 1998-07-07 Electronic Data Systems Corporation Method and system for detecting transitional markers such as uniform fields in a video signal
US6400853B1 (en) * 1997-03-19 2002-06-04 Canon Kabushiki Kaisha Image retrieval apparatus and method
US6427048B2 (en) * 1997-04-28 2002-07-30 Sony Corporation Apparatus and method for recording, reproducing and displaying a video signal received by a signal receiver
JP3780623B2 (ja) * 1997-05-16 2006-05-31 株式会社日立製作所 動画像の記述方法
US6125229A (en) * 1997-06-02 2000-09-26 Philips Electronics North America Corporation Visual indexing system
DE60036288T2 (de) 1999-06-30 2008-05-29 Sharp K.K. Dynamisches bildsuch-informations-aufzeichnungsgerät und dynamische bildsuchvorrichtung

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0546769A (ja) * 1991-08-12 1993-02-26 Nippon Telegr & Teleph Corp <Ntt> 動きベクトル算出方法
JPH06153146A (ja) * 1992-11-04 1994-05-31 Matsushita Electric Ind Co Ltd 動画像のシーンチェンジ検出装置および編集装置
US5719643A (en) * 1993-08-10 1998-02-17 Kokusai Denshin Denwa Kabushiki Kaisha Scene cut frame detector and scene cut frame group detector
JPH07152779A (ja) * 1993-11-29 1995-06-16 Toshiba Corp 動画インデックス検出処理方法及び動画インデックス検出処理機能を持つ動画処理装置
JPH0983864A (ja) * 1995-09-19 1997-03-28 Kokusai Denshin Denwa Co Ltd <Kdd> 動画像のカット点画像検出装置
EP0907147A2 (fr) * 1997-09-26 1999-04-07 Matsushita Electric Industrial Co., Ltd. Procédé d'affichage de séquences vidéo et dispositif d'affichage

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KOBLA V. & DOERMANN D.: "Indexing and retrieval of the MPEG compressed video", JOURNAL OF ELECTRONIC IMAGING,, vol. 7, no. 2, April 1998 (1998-04-01), pages 294 - 307, XP002930737 *
See also references of EP1195692A4 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7884884B2 (en) 1999-06-30 2011-02-08 Sharp Kabushiki Kaisha Dynamic image search information recording apparatus and dynamic image searching devices
JP2005063418A (ja) * 2003-07-18 2005-03-10 Samsung Electronics Co Ltd ＧｏＦ／ＧｏＰの質感表現方法と、これを用いたＧｏＦ／ＧｏＰの検索方法及び装置
WO2009042340A2 (fr) * 2007-09-25 2009-04-02 Motorola, Inc. Procédé pour créer, consommer et partager de façon intelligente du contenu vidéo sur des dispositifs mobiles
WO2009042340A3 (fr) * 2007-09-25 2009-05-22 Motorola Inc Procédé pour créer, consommer et partager de façon intelligente du contenu vidéo sur des dispositifs mobiles
JP2013149110A (ja) * 2012-01-20 2013-08-01 Renesas Electronics Corp 移動量検出装置、移動体検出装置及び移動量検出方法
JP2013210845A (ja) * 2012-03-30 2013-10-10 Secom Co Ltd 移動物体照合装置

Also Published As

Publication number Publication date
CN1371502A (zh) 2002-09-25
JP2004104836A (ja) 2004-04-02
JP4027662B2 (ja) 2007-12-26
EP1195692B1 (fr) 2007-09-05
KR100486047B1 (ko) 2005-05-03
JP4462922B2 (ja) 2010-05-12
ES2288859T3 (es) 2008-02-01
EP1195692A1 (fr) 2002-04-10
DE60036288D1 (de) 2007-10-18
US7884884B2 (en) 2011-02-08
EP1835421A2 (fr) 2007-09-19
US20060164560A1 (en) 2006-07-27
KR20020019482A (ko) 2002-03-12
CN1371502B (zh) 2010-05-05
US7092040B1 (en) 2006-08-15
EP1195692A4 (fr) 2004-04-21
DE60036288T2 (de) 2008-05-29
EP1835421A3 (fr) 2009-04-22

Similar Documents

Publication Publication Date Title
WO2001003005A1 (fr) Appareil d&#39;enregistrement d&#39;informations de recherche d&#39;images dynamiques et dispositif de recherche d&#39;images dynamiques
EP1024444B1 (fr) Méthodes de description d&#39;informations d&#39;images, de recherche et de reproduction de données vidéo et appareil de reproduction de données vidéo
KR100492437B1 (ko) 화상 검색 시스템 및 화상 검색 방법
US7889974B2 (en) Method and apparatus for editing images, and method and apparatus for reproducing the edited images
US7266771B1 (en) Video stream representation and navigation using inherent data
US20020051081A1 (en) Special reproduction control information describing method, special reproduction control information creating apparatus and method therefor, and video reproduction apparatus and method therefor
JPH04504920A (ja) デジタル・ビデオ・データを操作する方法及び装置
JP2001527304A (ja) ディジタル動画の階層的要約及び閲覧方法
JP4520994B2 (ja) 画像処理装置、画像処理方法、および画像処理プログラム
CN102682817B (zh) 剧本编辑装置、方法和系统以及图像拍摄装置及其控制方法
JPH11284948A (ja) サムネイル作成方法及びサムネイル表示方法
JP4770875B2 (ja) 画像特徴データ生成装置、画像特徴判定装置および画像検索システム
JP4574771B2 (ja) 画像情報記述方法
JP2008166895A (ja) 映像表示装置及びその制御方法、プログラム、記録媒体
EP1643764A1 (fr) Appareil de reproduction vidéo
JP4167245B2 (ja) デジタル映像処理方法及びその装置
JP2000194727A (ja) 動画像検索装置、動画像検索方法、及び動画像検索プログラムを記録した記録媒体
JP3524007B2 (ja) 動画場面転換部検出装置および方法、動画場面転換部検出制御プログラムを記憶した媒体、ならびに動画処理装置
KR20050025483A (ko) 내용 기반 검색을 위한 개선된 질의 방법 및 그 방법을구현하기 위한 프로그램을 저장한 기록매체
JP3581085B2 (ja) 二次コンテンツ生成システム及びその方法並びに二次コンテンツ生成プログラムを記録した記録媒体
KR0147568B1 (ko) 벡터 양자화에 의한 영상 압축 회로
JPH1070703A (ja) 画像情報管理用記録装置および画像情報管理用再生装置および画像情報管理用記録・再生装置
Boschetti et al. High Dynamic Range Images Coding: Embedded and Multiple Description
JPH10164435A (ja) 複数映像の時空間統合、管理方法及びその装置並びにそのプログラムを記録した記録媒体
CN113992937A (zh) 一种文化与科技融合的数字宣传系统

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CN JP KR SG US

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
ENP Entry into the national phase

Ref country code: JP

Ref document number: 2001 508725

Kind code of ref document: A

Format of ref document f/p: F

WWE Wipo information: entry into national phase

Ref document number: 2000940880

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 10019747

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 1020017016932

Country of ref document: KR

WWE Wipo information: entry into national phase

Ref document number: 008121761

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 1020017016932

Country of ref document: KR

WWP Wipo information: published in national office

Ref document number: 2000940880

Country of ref document: EP

WWG Wipo information: grant in national office

Ref document number: 1020017016932

Country of ref document: KR

WWG Wipo information: grant in national office

Ref document number: 2000940880

Country of ref document: EP