WO2010073905A1 - Appareil de visualisation d'images animées - Google Patents

Appareil de visualisation d'images animées Download PDF

Info

Publication number
WO2010073905A1
WO2010073905A1 PCT/JP2009/070566 JP2009070566W WO2010073905A1 WO 2010073905 A1 WO2010073905 A1 WO 2010073905A1 JP 2009070566 W JP2009070566 W JP 2009070566W WO 2010073905 A1 WO2010073905 A1 WO 2010073905A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
moving image
unit
still image
information
Prior art date
Application number
PCT/JP2009/070566
Other languages
English (en)
Japanese (ja)
Inventor
貴則 山田
Original Assignee
シャープ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by シャープ株式会社 filed Critical シャープ株式会社
Publication of WO2010073905A1 publication Critical patent/WO2010073905A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
    • H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
    • H04N1/00204—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a digital computer or a digital computer system, e.g. an internet server
    • H04N1/00209—Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
    • H04N1/00222—Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax details of image data generation or reproduction, e.g. scan-to-email or network printing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/482—End-user interface for program selection
    • H04N21/4828—End-user interface for program selection for searching program descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
    • H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/858—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • H04N21/8586—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot by using a URL
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording

Definitions

  • the present invention relates to a moving image viewing apparatus that searches and displays moving images, which are shot by a plurality of moving shooting devices and distributed in real time, using feature information calculated from still images.
  • the application will be used for the transmission and reception of high-quality moving images, and everyone will be able to deliver video in real time at any time.
  • a photographing device such as an in-vehicle camera or a mobile phone is taking a picture while moving from place to place
  • the content of the scene or delivered video changes from moment to moment.
  • viewers need to search and select videos that they want to view efficiently.
  • Patent Literature 1 in order for a viewer to select a moving image distributed from a plurality of photographing devices, a method of presenting and selecting a snap video acquired from each photographing device, or a position of the photographing device on a map is displayed. And suggesting a way to choose from.
  • Patent Document 2 it is possible to perform a search using an attribute list composed of text information directly attached to an image to be searched or feature information extracted from the image.
  • Patent Document 1 provides a means for intuitively selecting a scene to be viewed, when the number of photographing devices increases, how to search for a target moving image from among them is increased. There is no mention of what to do. Further, the method disclosed in Patent Document 2 has a problem of how to perform accurate attribute extraction and feature extraction in the case of a video that is updated in real time and whose contents change from moment to moment.
  • the present invention is a moving image viewing apparatus that enables a moving image captured by a moving image capturing apparatus and distributed via a network to be searched and viewed using a still image
  • a communication unit that communicates by connecting to a network, an input interface unit that receives input from a user, a display unit that presents search results to the user, and a moving image distributed from the network via the communication unit
  • a moving image frame acquisition unit that acquires frame data
  • a feature extraction unit that extracts feature information indicating image characteristics from the still image used for search and the frame data acquired by the moving image frame acquisition unit, and the feature extraction
  • a meta information storage unit for storing the meta information of the still image and the frame data including the feature information extracted from the unit
  • an image search control unit for controlling moving image search
  • the image search control unit searches the meta information of the still image of the meta information storage unit based on the search instruction input from the input interface unit, and the meta information of the still image corresponding to the search instruction exists , Calculating the similarity of the feature information of the frame data
  • the image search control unit may select meta information of frame data having feature information whose similarity is a certain value or more, and display the moving image corresponding to the selected frame data on the display unit The user may select a target moving image by the input interface.
  • the moving image viewing apparatus of the present invention further includes a still image acquisition unit that acquires a still image from a network via the communication unit, and the image search control unit receives a search instruction input from the input interface unit. Based on the above, the still image acquisition unit searches and acquires a still image from the network.
  • the moving image viewing apparatus of the present invention further includes a still image storage unit that stores a still image, and the image search control unit is configured to store the still image storage unit based on a search instruction input from the input interface unit. It is characterized in that a still image is retrieved and acquired from the image.
  • the image search control unit searches and acquires a still image when the meta information storage unit has a predetermined number or less of corresponding meta information.
  • the meta information of the still image includes a keyword for performing a search, and the image search control unit searches based on the keyword input from the input interface unit.
  • the meta information of the still image and the frame data includes position information that uniquely indicates a location of the image, and the image search control unit acquires an image based on the position information.
  • the moving image viewing apparatus of the present invention further includes a still image storage unit that stores a still image, and the position information of the still image is information that specifies a still image stored in the still image storage unit. It is characterized by that.
  • the moving image frame acquisition unit acquires an image at an arbitrarily specified time among the moving images distributed from the moving image capturing device. If the frame closest to the specified time is an intra frame, one moving image frame acquiring unit A frame is acquired, and when a frame closest to a specified time is a predicted frame, an intra frame and a frame including the predicted frame necessary for decoding the frame are acquired.
  • the moving image frame acquisition unit acquires images from a plurality of the moving image photographing devices specified in advance according to a specified time.
  • the feature extraction unit extracts luminance information of each pixel from the image, and outputs the number of pixels having the same luminance value arranged in ascending order, descending order, or predetermined order as feature information.
  • the feature extraction unit extracts color information of each pixel from the image, and outputs, as feature information, the number of pixels having the same color arranged in ascending order, descending order, or a predetermined order.
  • the present invention uses meta information of still images for search to search for moving images transmitted with contents that change scenes and subjects from moment to moment, and therefore uses still images as keywords or keys.
  • the scene you want to see can be searched efficiently and easily.
  • FIG. 1 is a system diagram showing an embodiment of a moving image viewing system. It is a block diagram which shows the structure of the function with which the moving image viewing apparatus of a moving image viewing system is provided. It is a flowchart which shows the procedure from searching a still image on a network based on the input keyword, extracting the characteristic of the acquired still image, and storing in the still image meta-information storage part. It is a figure which shows the example of the still image meta information stored in a still image meta information storage part. It is a flowchart which shows the procedure from acquiring a still image from a still image preservation
  • FIG. 1 is a system diagram showing an embodiment of a moving image viewing system.
  • Reference numerals 11 to 14 denote moving image photographing devices, which are equipped with a communication device and capable of transmitting the images in real time while photographing.
  • four devices are shown to indicate that a plurality of different devices are mixed, but the number of devices is not limited in actual use.
  • Reference numerals 15 and 16 denote moving image viewing apparatuses, such as a mobile phone, a personal computer, and a television.
  • Reference numeral 17 denotes a network in which the moving image capturing device and the moving image viewing device can communicate with each other, and includes, for example, communication using a circuit switching network, a mobile phone network, a wireless LAN, Bluetooth, or the like.
  • Reference numerals 18 and 19 denote servers connected to the network 17, which can store and distribute still images, moving images taken by the moving image shooting device, and the like.
  • the moving image capturing apparatus may provide a moving image to the moving image viewing apparatus via the network 17 without using the server.
  • the moving image viewing apparatuses 15 and 16 receive a search request from the moving image photographing apparatuses 11 to 14 that distribute moving images in real time (or via the servers 18 and 19) at regular intervals or from the viewer.
  • a snapshot of the image being distributed that is, frame data (still image) of the moving image is acquired.
  • the feature information is extracted and stored in the apparatus as meta information of the moving image frame together with the acquisition source information (URI).
  • URI acquisition source information
  • the meta information includes image feature information, acquisition source information (URI), acquisition time, position information indicating the location of the image, search keywords, copyright information, and the like. It is embedded in still image data, managed as separate data such as text or XML, and managed as a database.
  • the search keyword is a character string representing an object, a landscape, a person, a state, and the like.
  • the moving image viewing devices 15 and 16 extract feature information from still images in the devices or still images that can be acquired from the communication network 17 (still images in the servers 18 and 19), and search keywords, It is stored in the apparatus as meta information of a still image together with acquisition source information (information indicating whether a file in the apparatus exists or URI).
  • the search keyword includes a keyword included in metadata included in a still image in advance, a keyword used as a search word when acquiring a still image from the communication network 17, and a keyword directly input by a user from an input device of a terminal. Is used.
  • the meta information of the still image acquired by the same method is stored in the apparatus in advance.
  • the moving image viewing device extracts feature information of the corresponding still image, and features information of the moving image frame meta information The similarity is calculated, and it is determined that the high similarity is the moving image desired by the viewer, and the search result is presented.
  • the feature information of the extracted still image may be one or plural, and in the case where there are plural pieces, the result obtained by calculating the similarity with one moving image frame meta information is also plural. Therefore, the one with the highest similarity is adopted.
  • FIG. 2 is a block diagram showing a functional configuration of the moving image viewing apparatus of the moving image viewing system.
  • the moving image viewing apparatus includes a communication unit 20, a moving image frame acquisition unit 30, a still image acquisition unit 40, an image decoding unit 50, a feature extraction unit 60, a meta information storage unit 70, A still image storage unit 80, an image search control unit 90, an input interface unit 100, and a display unit 110 are provided.
  • the communication unit 20 exchanges data with the servers 18 and 19 and the moving image photographing apparatuses 11 to 14 via the communication network 17.
  • the moving image frame acquisition unit 30 is a snapshot (moving image frame) of a moving image that is distributed at a certain time interval or when a search request is received from a viewer via the communication unit 20, that is, Get a still image.
  • the still image acquisition unit 40 acquires a still image distributed to the communication network 17 via the communication unit 20.
  • the image decoding unit 50 has a function of decoding a moving image and a function of decoding a still image, and are called a moving image decoding unit 51 and a still image decoding unit 52, respectively.
  • the feature extraction unit 60 extracts feature information (details will be described later) from the decoded image.
  • the meta information storage unit 70 has a function capable of separately storing the feature information extracted from the still image and the feature information extracted from the moving image, and the storage destination is the still image meta information storage unit 71, This is called a moving image frame meta information storage unit 72.
  • the still image storage unit 80 stores still images.
  • the image search control unit 90 controls each unit when moving image search processing is performed.
  • the input interface unit 100 has a function of inputting a keyword for performing a search and a function of selecting a moving image to be viewed, and are referred to as a search key input unit 101 and a moving image designating unit 102, respectively.
  • the input interface unit 100 includes hardware such as an input device such as a keyboard, a mouse, a touch panel, and a button, and a pointing device.
  • the search key input unit 101 can input a scene that the viewer wants to see by directly inputting a keyword or the like, selecting from a keyword or genre registered in advance, or selecting an image close to the scene to be viewed.
  • the display unit 110 can present a search result obtained by inputting a search key to a viewer or display a moving image selected from the viewer, and these can be displayed as a search result display unit 111 and a moving image display, respectively. This is referred to as part 112.
  • FIG. 3 shows a procedure from when the moving image viewing apparatus searches for a still image on the network based on the input keyword, extracts the characteristics of the acquired still image, and stores them in the still image meta information storage unit 71. It is a flowchart which shows. This is a partial extraction of the processing from search to viewing according to the present invention.
  • step S11 based on the keyword input from the search key input unit 101 of the input interface unit 100, the image search control unit 90 captures a still image on the communication network 17 via the communication unit 20.
  • Search for The search method is not particularly limited. For example, in the Internet, when some keywords are input and transmitted to the Web server, there are some services that create a list of still images corresponding to the keywords from the database in the Web server. You can use this as well. Later, in order to further improve the accuracy of the search result of moving images, a plurality of images are acquired from the plurality of searched still images.
  • step S12 the image search control unit 90 confirms whether or not the acquired still image has already been acquired and the feature information has been extracted. , And remove it from the target of the subsequent processing. Whether or not it has been extracted can be determined by information indicating a still image acquisition source stored in the still image meta information storage unit 71, that is, by a URI. If the feature information has already been extracted from all of the acquired still images, the process shown in FIG. 3 ends here. If there is an unextracted still image, the process proceeds to the next process.
  • step S13 the image search control unit 90 causes the still image acquisition unit 40 to acquire a still image from which patent information has not been extracted, and sets the still image in a format that can extract feature information. Therefore, the still image decoding unit 52 performs decoding.
  • the feature extraction unit 60 extracts the feature information of the decoded still image.
  • the feature information is information that can be used to search for an original image or an image similar to the original image using the information as a clue.
  • the feature information extraction method is not limited in the present invention. For example, the method of recording the color distribution of the image, the method of extracting the feature point of the image, the texture in the image, the luminance information, or the color information There is a method of extracting statistical information, that is, a histogram.
  • the texture data histogram for example, for all the pixels in the image, compared to the brightness of the surrounding eight pixels, whether or not more than half of the surrounding pixels are brighter than the brightness of the pixel of interest. Is obtained, and the number of pixels is totaled for each luminance and arranged for each luminance.
  • the histogram of color information is obtained, for example, by extracting the degree of red, green, and blue colors for each pixel, and counting and arranging the number of pixels for each color.
  • Alignment may be in ascending order or descending order in terms of the number of pixels, or in ascending order or descending order in terms of luminance or color. Alternatively, a predetermined arrangement may be used.
  • step S15 the feature extraction unit 60 collects the keyword input in step S11, the information (URI) of the still image acquisition source, and the feature information extracted in step S14 into a still image meta information. Store in the storage unit 71.
  • steps S12 to S15 may be repeated for each still image, or the processing from steps S12 to S15 is performed for all the images. You may go in order.
  • FIG. 4 is a diagram illustrating an example of still image meta information stored in the still image meta information storage unit 71 as a result of the series of processes in FIG.
  • Still image meta information numbers 1 to 3 are still meta image information extracted by keywords inputted so far, and still image meta information number 4 is added with still image meta information obtained by newly inputted keywords. Indicates that In this way, if still image meta information corresponding to a keyword input in the past is accumulated, the search speed when the same keyword is input can be increased.
  • the image search control unit 90 does not make the determination in step S12, and once performs the processing up to step S14, compares the feature information, and does not newly store still image meta information if they match completely. It is also possible. In addition, if the feature information matches but the keyword does not match, only the keyword can be added to the keyword portion of the stored still image meta information. As a result, the same feature information has a different meaning key. Can also be used.
  • FIG. 5 is a flowchart showing a procedure from acquiring a still image from the still image storage unit 80 based on the input keyword, extracting features of the acquired still image, and storing the feature in the still image meta information storage unit 71. It is. This is a partial extraction of the processing from search to viewing according to the present invention.
  • the image search control unit 90 includes a still image that includes the keyword input from the search key input unit 101 as a part of the file name or has as file metadata. It is confirmed whether it exists in the preservation
  • step S ⁇ b> 22 the image search control unit 90 confirms whether any of the still images stored in the still image storage unit 80 has already been acquired and feature information has been extracted. However, those that have already been extracted are excluded from the target of subsequent processing. Whether or not it has been extracted can be determined by information (see FIG. 4) indicating a still image acquisition source stored in the still image meta information storage unit 71, that is, a file name. If the feature information has already been extracted from all of the stored still images, the processing shown in FIG. 5 ends here. If there is an unextracted still image, the process proceeds to the next process.
  • step S23 the image search control unit 90 sets the still image from which the patent information stored in the still image storage unit 80 has not been extracted into a format in which the feature information can be extracted.
  • Decoding is performed by the image decoding unit 52.
  • step S24 the feature extraction unit 60 extracts the feature information of the decoded still image.
  • step S25 the feature extraction unit 60 stores the keyword input in step S21, the still image file path, and the feature information extracted in S24 together in the still image meta information storage unit 71.
  • the saved information is saved as in the example shown in FIG.
  • An apparatus for viewing a moving image needs to have information on a moving image acquisition source, that is, a moving image photographing apparatus, in advance.
  • a moving image photographing apparatus For example, several photographing devices are registered in advance, information on each photographing device can be registered by a viewer, or information is acquired from a portal site that collects information on moving image photographing devices. There are methods.
  • the moving image frame acquisition unit 30 of the moving image viewing apparatus sequentially takes a snapshot of the moving image being distributed at a certain time interval or at the time when a search request is received from the viewer. (Moving image frame) is acquired. For example, an image at an arbitrarily specified time is acquired from the moving images distributed from the moving image capturing device, and if the frame closest to the specified time is an intra frame, one intra frame is acquired and specified. If the frame closest to the predetermined time is a predicted frame, a frame including an intra frame and a predicted frame necessary for decoding the frame is acquired.
  • the image acquired by the moving image frame acquisition unit 30 is decoded by the still image decoding unit 52 of the image decoding unit 50, and the feature extraction unit 60 extracts features by the same method as the feature extraction of the still image.
  • the feature extraction unit 60 stores the extracted feature information in the moving image frame meta information storage unit 72 together with information (URI) indicating the acquisition source. At this time, if moving image frame meta information having the same information indicating the acquisition source already exists in the moving image frame meta information storage unit 72, the feature extraction unit 60 overwrites and stores the information.
  • FIG. 6 is an example of moving picture frame meta information stored in the moving picture frame meta information storage unit.
  • step S31 first, the viewer is presented with a screen for searching for a moving image on the display unit 110.
  • the moving image to be viewed can be searched and selected.
  • one or a plurality of keywords are input from the search key input unit 101 of the input interface unit 100, selected from a keyword list, or a list of images that represent scenes to be viewed is displayed and selected from there. And so on.
  • step S32 it is determined whether or not a still image used in the search is newly acquired from the network.
  • the determination condition is whether or not the still image meta information corresponding to the keyword input in step S31 exists in the still image meta information storage unit 71. Normally, if this does not exist, a moving image cannot be searched unless a new still image is acquired. However, if it exists, it is not necessary to newly acquire it. However, since the search accuracy of moving images is improved by using meta information of a plurality of still images, for example, as a result of the search, the still image meta information storage unit 71 If only a certain number or less of still image meta information can be secured, a still image for search is acquired from the network. If it is expected in step S34 that the still image in the apparatus can be used for search purposes, or if it is not necessary to find one as a search result, there is no need to obtain it.
  • step S33 the series of processing shown in FIG. 3 is performed, and the meta information including the acquired still image feature information is stored in the still image meta information storage unit 71.
  • step S34 the image search control unit 90 determines whether to use a still image in the apparatus as a still image used in the search.
  • the determination condition is whether or not the still image meta information corresponding to the keyword input in step S31 exists in the still image meta information storage unit 71. Normally, if it exists, it is not necessary to use a still image in the apparatus as a search image, but it is desirable to use it in order to improve the accuracy of moving image search.
  • step S35 the series of processing shown in FIG. 5 is performed, and the feature information is stored in the still image meta information storage unit.
  • the determination conditions in steps S32 and S34 may be settings unique to the viewing device, or may be set freely by the viewer.
  • step S36 the image search control unit 90 inputs the keyword, the still image meta information stored in the still image meta information storage unit 71, and the moving image frame meta information stored in the moving image frame meta information storage unit 72. Then, the moving image search process is performed.
  • still image meta information whose keyword matches part or all of the input keyword is extracted.
  • the similarity between the extracted feature information of the still image meta information and the feature information of the moving image frame meta information stored in the moving image frame meta information storage unit 72 is calculated.
  • the similarity calculation method differs depending on the feature information format, and a plurality of calculation methods can be considered even if the feature feature format is the same. Therefore, although not particularly limited in the present invention, for example, as described above, the brightness of the image When the information histogram is used as the feature information, the distance between the values for each luminance in the feature information is calculated, and the sum of the calculated distances can be used as a scale representing the similarity. In this method, the similar value is smaller, and the value is larger if they are not similar.
  • the similarity will be different if the resolutions are different, so resizing so that the image size will be the same number of pixels before performing feature extraction. It is preferable to calculate the similarity after performing the feature extraction or by making the scales of the histograms equal before calculating the similarity, that is, by making the sum of all the values in the histogram the same.
  • the search result is displayed on the search result display unit 111 in step S37.
  • the search results may be displayed in order of URIs, or the corresponding snapshot images acquired earlier may be displayed instead of the URIs. If possible, a moving image may be received from each photographing apparatus and displayed.
  • the viewer designates a moving image desired to be viewed from the search results displayed on the search result display unit 111 by the moving image designating unit 102.
  • the image search control unit 90 issues a viewing start request to the URI specified via the communication unit 20.
  • the moving image decoding unit 51 decodes the moving image and the moving image display unit 112 displays the moving image.
  • the viewer simply enters the search keyword and selects the desired moving image from the search results, and the content of the scene or video to be distributed changes from moment to moment.
  • the present invention assumes a case where a moving image is distributed via a communication network.
  • the present invention is also applicable to moving image distribution by broadcasting.
  • the acquisition source information held as metadata is channel information or frequency information.
  • the search keyword can also be used when broadcast information includes character information such as program information.
  • the present invention assumes a moving image photographed by a photographing apparatus, it can also be applied to an animation moving image.
  • Moving image capturing devices 15 and 16 Moving image viewing device 17 Network 18 and 19 Server 20 Communication unit 30 Moving image frame acquisition unit 40 Still image acquisition unit 50 Image decoding unit 51 Moving image decoding unit 52 Still image decoding unit 60 Features Extraction unit 70 Meta information storage unit 71 Still image meta information storage unit 72 Moving image frame meta information storage unit 80 Still image storage unit 90 Image search control unit 100 Input interface unit 101 Search key input unit 102 Moving image designation unit 110 Display unit 111 Search result display unit 112 Moving image display unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

L'invention concerne un appareil de visualisation d'images animées pouvant rechercher et récupérer efficacement une scène souhaitée parmi une pluralité d'images animées fournies à partir d'une pluralité d'appareils d'acquisition d'images. S'il n'existe aucune méta-information d'images fixes, destinée à être utilisée pour une recherche, dans une unité de mémorisation de méta-informations d'images fixes (71), les méta-informations d'images fixes sont acquises à partir d'images fixes fournies sur un réseau ou à partir d'une unité de mémorisation d'images fixes (80) dans l'appareil de visualisation d'images animées, et des informations de caractéristiques d'images sont ensuite extraites par l'intermédiaire d'une unité d'extraction de caractéristiques (60), puis mémorisées dans l'unité de mémorisation de méta-informations d'images fixes (71). Les informations de caractéristiques sont extraites, par l'intermédiaire de l'unité d'extraction de caractéristiques (60), à partir des données de trames d'images animées fournies à partir des appareils d'acquisition d'images, puis mémorisées dans une unité de mémorisation de méta-informations de trames d'images animées (72). Une unité de commande de recherche d'images (90) exécute un procédé de recherche d'images animées à l'aide de mots-clés entrés, des méta-informations d'images fixes mémorisées dans l'unité de mémorisation de méta-informations d'images fixes (71), et des méta-informations de trames d'images animées mémorisées dans l'unité de mémorisation de méta-informations de trames d'images animées (72).
PCT/JP2009/070566 2008-12-25 2009-12-08 Appareil de visualisation d'images animées WO2010073905A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2008329672 2008-12-25
JP2008-329672 2008-12-25

Publications (1)

Publication Number Publication Date
WO2010073905A1 true WO2010073905A1 (fr) 2010-07-01

Family

ID=42287521

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2009/070566 WO2010073905A1 (fr) 2008-12-25 2009-12-08 Appareil de visualisation d'images animées

Country Status (1)

Country Link
WO (1) WO2010073905A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013035670A1 (fr) * 2011-09-09 2013-03-14 株式会社日立製作所 Système d'extraction d'objets et procédé d'extraction d'objets
JP2015527564A (ja) * 2012-06-07 2015-09-17 エフ・ホフマン−ラ・ロシュ・アクチェンゲゼルシャフト 自己免疫抗体
JP2016502194A (ja) * 2012-11-30 2016-01-21 トムソン ライセンシングＴｈｏｍｓｏｎ Ｌｉｃｅｎｓｉｎｇ ビデオ検索方法及び装置
JP2021100201A (ja) * 2019-12-23 2021-07-01 横河電機株式会社 装置、システム、方法およびプログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004072504A (ja) * 2002-08-07 2004-03-04 Sony Corp 画像表示装置、画像表示方法、プログラム、及び記録媒体、並びに画像表示システム
JP2006039753A (ja) * 2004-07-23 2006-02-09 Canon Inc 画像処理装置、画像処理方法
JP2006129519A (ja) * 2005-12-08 2006-05-18 Hitachi Ltd 画像記憶装置、監視システム、記憶媒体
JP2007251646A (ja) * 2006-03-16 2007-09-27 Mitsubishi Electric Corp 監視システム、集信装置および映像蓄積配信装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004072504A (ja) * 2002-08-07 2004-03-04 Sony Corp 画像表示装置、画像表示方法、プログラム、及び記録媒体、並びに画像表示システム
JP2006039753A (ja) * 2004-07-23 2006-02-09 Canon Inc 画像処理装置、画像処理方法
JP2006129519A (ja) * 2005-12-08 2006-05-18 Hitachi Ltd 画像記憶装置、監視システム、記憶媒体
JP2007251646A (ja) * 2006-03-16 2007-09-27 Mitsubishi Electric Corp 監視システム、集信装置および映像蓄積配信装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
MASAAKI SATO ET AL.: "Fukuso?Kao o Mochiita Jinbutsu Kensaku System", MATSUSHITA TECHNICAL JOURNAL, vol. 52, no. 3, 18 June 2006 (2006-06-18), pages 67 - 71 *
TOSHIHIKO HATA ET AL.: "Eizo Chikuseki?Kensaku?Hyoji Gijutsu", MITSUBISHI DENKI GIHO, vol. 78, no. 8, 25 August 2004 (2004-08-25), pages 47 - 50 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013035670A1 (fr) * 2011-09-09 2013-03-14 株式会社日立製作所 Système d'extraction d'objets et procédé d'extraction d'objets
JPWO2013035670A1 (ja) * 2011-09-09 2015-03-23 株式会社日立製作所 物体検索システムおよび物体検索方法
JP2015527564A (ja) * 2012-06-07 2015-09-17 エフ・ホフマン−ラ・ロシュ・アクチェンゲゼルシャフト 自己免疫抗体
JP2016502194A (ja) * 2012-11-30 2016-01-21 トムソン ライセンシングＴｈｏｍｓｏｎ Ｌｉｃｅｎｓｉｎｇ ビデオ検索方法及び装置
JP2021100201A (ja) * 2019-12-23 2021-07-01 横河電機株式会社 装置、システム、方法およびプログラム
JP7205457B2 (ja) 2019-12-23 2023-01-17 横河電機株式会社 装置、システム、方法およびプログラム

Similar Documents

Publication Publication Date Title
US8559516B2 (en) Video sequence ID by decimated scene signature
US20210343070A1 (en) Method, apparatus and electronic device for processing image
CN104012106B (zh) 使表示不同视点的视频对准
US20110289532A1 (en) System and method for interactive second screen
US20150227780A1 (en) Method and apparatus for determining identity and programing based on image features
US20090213270A1 (en) Video indexing and fingerprinting for video enhancement
KR102246305B1 (ko) 증강 미디어 서비스 제공 방법, 장치 및 시스템
US20120117599A1 (en) Thumbnail publication
JP4428424B2 (ja) 情報処理装置、情報処理方法、プログラムおよび記録媒体
US9538246B2 (en) Map your movie
WO2016192501A1 (fr) Procédé et appareil de recherche vidéo
US11190828B1 (en) Systems and methods for versatile video recording
US20160277808A1 (en) System and method for interactive second screen
JP2010086194A (ja) シェア画像閲覧方法および装置
JP2016532386A (ja) ビデオを表示する方法及びビデオを表示する装置
JP2006270869A (ja) 関連情報取得システム、管理装置、関連情報取得方法、関連情報送信プログラム。
WO2010073905A1 (fr) Appareil de visualisation d'images animées
US20140010521A1 (en) Video processing system, video processing method, video processing apparatus, control method of the apparatus, and storage medium storing control program of the apparatus
CN109495789B (zh) 媒体文件播放方法、设备及通信系统
KR101542416B1 (ko) 멀티앵글영상서비스 제공 방법 및 시스템
JP2006039753A (ja) 画像処理装置、画像処理方法
KR20130126203A (ko) 클라이언트 단말기를 이용한 동영상 콘텐츠 공유 서비스 제공 시스템 및 방법
CN110100445B (zh) 信息处理系统、信息处理装置和计算机可读介质
JP4853564B2 (ja) 情報処理装置、情報処理方法、プログラムおよび記録媒体
CN107431831B (zh) 使用视频帧来标识视频序列的装置和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09834708

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09834708

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP