JP2005107767A - Video retrieval device, video retrieval method and video retrieval program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide new video retrieval technology allowing a user to easily retrieve and reproduce a video portion which the user vaguely images and desires to reproduce. <P>SOLUTION: When the user previously scans outline information from all videos that are retrieval targets as characteristic information (can perform the scanning in each case without previously performing the scanning) and draws an outline of an object included in a desired video on a screen of a computer, the outline and the scanned outline information are collated, and a video most similar is exhibited to the user. As a retrieval result, only one retrieval result may be displayed, or a plurality of retrieval results can be displayed side by side in the similar order. Because it is possible to display even an unpredictable object, when displaying the retrieved result as it is, it is inconvenient to display of a video of a street or the like. Thus, only a video portion of the retrieved object is displayed in an area, wherein the user draws the outline of the object with the size thereof. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a video search apparatus and method for allowing a user to easily search for and play back a video portion desired to be played back vaguely imagined, and a video search program used for realizing the video search method.

  As a method for a user to search for and play back a video, a method of searching using a video name or keyword or narrowing down from a related genre is performed in a known search service.

  However, in these methods, it is difficult to search for a desired video from vague image storage.

Against this backdrop, input the pattern of the image or figure, the edge information of the pattern, and the color information around the pattern, the characteristics of the input pattern and the image data stored in the file Based on the data, a method of extracting image data as a search candidate from image data stored in a file and determining final search result image data from the image data is proposed (for example, , See Patent Document 1).
JP-A-8-194716

  Certainly, if the search method described in Patent Document 1 is used, it is possible to search for an arbitrary video by relying on the figure to be imaged.

  However, in the search method described in Patent Document 1, an image or graphic pattern, edge information of the pattern, and color information around the pattern must be input, and vague image storage is performed. It is insufficient for the request to search for a desired video from the list.

  Moreover, in the search method described in Patent Document 1, in response to the input search information, the entire area of the video having a video portion that matches the search information is displayed. This method is not suitable for displaying only the subject searched from the video.

  The present invention has been made in view of such circumstances, and it is an object of the present invention to provide a new video search technique that enables a user to easily search and play back a video portion desired to be played back vaguely imagined.

In order to achieve this object, the video search device of the present invention includes (a) acquisition means for acquiring information indicating the contour of the subject drawn by the user, and (b) contour information of the subject included in the video to be searched. By determining the similarity between the contour information stored in the storage unit that stores the contour information acquired by the acquisition unit and the contour information acquired by the acquisition unit, the profile information is similar to the contour information acquired by the acquisition unit. Search means for searching for one or a plurality of contour information; and (c) the video portion having the contour information searched by the search means so as to match the subject area defined by the contour information acquired by the acquisition means. And (d) display means for displaying the image portion adjusted by the adjustment means in a subject area defined by the outline information acquired by the acquisition means.

  Each of the processing means described above can be realized by a computer program, and this computer program can be provided by being recorded on a recording medium such as a semiconductor memory or via a network.

  In order to achieve this object, the video search device of the present invention includes (a) an acquisition unit that acquires information indicating a contour of a subject drawn by a user, and (b) a storage unit that accumulates video to be searched. By reading out the video from the image, and extracting the contour information of the subject included in the video, and (c) determining the similarity between the contour information extracted by the extraction unit and the contour information acquired by the acquisition unit, A search means for searching for one or a plurality of contour information similar to the contour information acquired by the acquisition means from the contour information extracted by the extraction means; and (d) a video portion having the contour information searched by the search means. Adjusting means for adjusting the video portion so as to match the subject area specified by the contour information acquired by the acquiring means; and (e) the subject specified by the contour information acquired by the acquiring means for the video portion adjusted by the adjusting means. region Configured to a display unit for displaying.

  Each of the processing means described above can be realized by a computer program, and this computer program can be provided by being recorded on a recording medium such as a semiconductor memory or via a network.

  In the present invention configured as described above, when a user draws the contour of a subject that the user wants to search in order to be able to search for the image by using the figure of the subject included in the image as a clue, from the image to be searched, A video part having an outline similar to the outline is searched, and the position and size of the video part obtained as a result are adjusted and processed so as to be displayed in the area drawn by the user.

  In this way, according to the present invention, even if the subject to be searched for is ambiguous, the user can search for an image of the subject simply by drawing the outline of the subject, and the subject Only the video part can be displayed in the area designated by the user.

  In order to search for a desired image imaged by the user, in the conventional technique, it is necessary to search by replacing the subject with a name, a keyword, a genre, or the like. In such a search, there is a difference between how to keyword and genre classification of videos set for searching as a database, and how to view keywords and genres to be selected for searching for desired videos. Is difficult to find intuitively.

  Therefore, a means for searching using the contour information of the video is effective. The desired image often has a clue to the subject reflected in the image.

  Therefore, in the present invention, contour information is scanned as feature information of a subject from all videos to be searched in advance (it is also possible to scan each time instead of scanning in advance), and the user can scan the computer. When the contour of the subject included in the desired video is drawn on the screen, the contour is compared with the scanned contour information, and the closest video is presented to the user. Note that only one search result may be displayed, one search result may be displayed one by one in order of closeness, or a plurality may be displayed in order of closeness. Moreover, you may add similarity.

  Here, if the search result is displayed as it is, even an unpredicted subject may be displayed, which is inconvenient for displaying images such as streets.

  Therefore, in the present invention, only the video portion of the searched subject is displayed at that position in the drawing area (the area in which the outline of the subject is drawn) used by the user.

  For example, assume that the user only remembers that a glass is shown in the video. At this time, as shown in FIG. 1, when the user draws a glass figure using a touch panel or the like, the contour information is extracted from the search target video as shown in FIG. 2, and the video as shown in FIG. The contour information similar to the drawn glass figure is scanned, for example, by reducing.

  Of the similarities obtained by such processing, the highest one is displayed as a search result. At that time, as shown in FIG. 4, only the video portion of the glass is displayed in the drawn glass area. By doing so, it is possible to prevent privacy from being displayed without displaying images other than glasses that can identify people and places.

  According to the present invention, even if the subject to be searched for is ambiguous memory, the user can search for the video of the subject simply by drawing the outline of the subject, and only the video portion of the subject can be searched. It can be displayed in the area specified by me.

  Hereinafter, the present invention will be described in detail according to embodiments.

[First Embodiment]
FIG. 5 illustrates a first embodiment of the present invention.

  As shown in this figure, the video search apparatus 1 of the present invention includes a video database 10, a contour information database 11, a subject contour extraction unit 12, a display 13 with a touch panel, a drawing information acquisition unit 14, and a similarity determination. Unit 15, similarity totaling unit 16, display area adjusting unit 17, and video reproduction unit 18.

  When the video search apparatus 1 according to the present invention is in accordance with the first embodiment, the video search apparatus 1 includes a video database 10 for storing videos to be searched, and in addition, a contour for storing contour information of videos to be searched. A configuration including the information database 11 is adopted.

  The subject contour extraction unit 12 is responsible for the process of constructing the contour information database 11. The subject contour extraction unit 12 acquires a video from the video database 10 and extracts the contour information of the subject included in the video. The contour information database 11 is constructed by registering it in the contour information database 11.

  For the extraction of the contour information by the subject contour extraction unit 12, for example, edge extraction using a neural network that uses luminance information and color information of a video is used. Instead of discriminating each subject in the video one by one, it may be binary information that is simply determined whether or not it is a contour, or only a portion having a strong contour may be extracted.

  Here, the registration processing of the contour information in the contour information database 11 may be performed in real time at the time of search, but it must be processed in advance in order to increase the search speed and reduce the processing load. Is desirable. In this case, when a video to be searched for is newly added, the contour information of the subject included in the video is extracted and added to the contour information database 11.

  Next, processing when the user searches for video will be described.

  When using the video search device 1 of the present invention, the user draws the contour of the subject to be searched on the display 13 with the touch panel.

  In response to this, the drawing information acquisition unit 14 acquires contour information (information on position, size, and shape) of the subject drawn by the user. Subsequently, the similarity determination unit 15 collates the contour information acquired by the drawing information acquisition unit 14 with each contour information stored in the contour information database 11 and measures the similarity.

  The similarity measured at this time may be any as long as it can determine the similarity between the two pieces of contour information. For example, by moving in the X-axis / Y-axis directions, You may make it measure the maximum value of the coincidence degree between outline information. At this time, it may be performed in combination with enlargement / reduction, or rotation may be considered.

  The similarity totaling unit 16 totals the similarities of the target videos obtained in this way and arranges them in the order of similarity. Here, the results in the order of similarity may be presented to the user, and the video to be played back may be selected from among them, or the video with the highest similarity may be determined as the playback target. Also good.

  Subsequently, the display area adjustment unit 17 measures the vertical and horizontal scales from the area drawn by the user and the outline size of the video determined as the reproduction target, and the area of the video to be reproduced in the user's drawing area. The image scale and coordinate information (X-axis / Y-axis coordinates) that match are acquired.

  Subsequently, the video playback unit 18 adjusts and plays back the video to be played back according to the scale and coordinate information of the video obtained by the display area adjustment unit 17, thereby drawing the user's drawing in the region drawn by the user. A video having contour information similar to the contour information is reproduced.

  FIG. 6 illustrates an example of a processing flow executed by the video search apparatus 1 of the present invention configured as described above.

  That is, in the video search device 1 of the present invention configured as described above, as shown in the processing flow of FIG. 6, first, in step 10, when the contour information drawn by the user is acquired, in the subsequent step 11, Unprocessed contour information is read from the contour information database 11.

  Subsequently, in step 12, the similarity between the contour information drawn by the user and the read contour information is determined, and in step 13 whether or not all the contour information stored in the contour information database 11 has been processed. If it is determined that all the contour information has not been processed, the process returns to step 11.

  When the similarity between the contour information drawn by the user and all the contour information stored in the contour information database 11 is determined by repeating the processing of step 11 to step 13 in this manner, the process proceeds to step 14. , Summing up their similarities.

  Subsequently, in step 15, a video with a high degree of similarity (subject video) is selected, and in subsequent step 16, the position and size of the selected video are adjusted to be displayed in the area drawn by the user.

Finally, in step 17, the adjusted video is reproduced in the area drawn by the user.

  In this way, the video search device 1 of the present invention searches for a video of a subject having the same contour as the contour drawn by the user, and searches the subject at the same position in the region drawn by the user. It is processed so that only the video portion of the image is reproduced.

  If it is desired to display the entire search result video, it may be performed in response to subsequent user operations. Further, although the video has been described here, a still image can be searched according to the same processing.

[Second Embodiment]
FIG. 7 illustrates a second embodiment of the present invention. In the figure, the same components as those described in FIG. 5 are indicated by the same symbols.

  When the video search device 1 of the present invention follows the first embodiment, the video information is extracted in advance and registered in the contour information database 11 and used. In the case of following the second embodiment, the subject contour extraction unit 12 acquires videos one by one from the video database 10 in response to an instruction from the similarity determination unit 15 without providing the contour information database 11. However, the configuration is such that the contour information is extracted.

  Next, processing when the user searches for video will be described.

  When using the video search device 1 of the present invention, the user draws the contour of the subject to be searched on the display 13 with the touch panel.

  In response to this, the drawing information acquisition unit 14 acquires contour information (information on position, size, and shape) of the subject drawn by the user.

  Subsequently, the similarity determination unit 15 instructs the subject contour extraction unit 12 to extract contour information, and in response to this, the subject contour extraction unit 12 reads an unprocessed video from the video database 10 and contours. Information is extracted and passed to the similarity determination unit 15. Upon receiving this contour information, the similarity determination unit 15 collates the contour information acquired by the drawing information acquisition unit 14 with the received contour information, and measures the similarity.

  Then, by repeating this series of processing, the similarity determination unit 15 collates the contour information acquired by the drawing information acquisition unit 14 with each contour information extracted from the video stored in the video database 10. The process of measuring the similarity is performed.

  Here, for the extraction of the contour information by the subject contour extraction unit 12, for example, edge extraction using a neural network using the luminance information and color information of the video is used. Instead of discriminating each subject in the video one by one, it may be binary information that is simply determined whether or not it is a contour, or only a portion having a strong contour may be extracted.

  Further, the similarity to be measured may be any as long as the similarity between the two pieces of contour information can be determined. For example, by moving in the X-axis / Y-axis directions, You may make it measure the maximum value of the coincidence degree between outline information. At this time, it may be performed in combination with enlargement / reduction, or rotation may be considered.

The similarity totaling unit 16 totals the similarities of the target videos obtained in this way and arranges them in the order of similarity. Here, the results in the order of similarity may be presented to the user, and the video to be played back may be selected from among them, or the video with the highest similarity may be determined as the playback target. Also good.

  Subsequently, the display area adjustment unit 17 measures the vertical and horizontal scales from the area drawn by the user and the outline size of the video determined as the reproduction target, and the area of the video to be reproduced in the user's drawing area. The image scale and coordinate information (X-axis / Y-axis coordinates) that match are acquired.

  Subsequently, the video playback unit 18 adjusts and plays back the video to be played back according to the scale and coordinate information of the video obtained by the display area adjustment unit 17, thereby drawing the user's drawing in the region drawn by the user. A video having contour information similar to the contour information is reproduced.

  FIG. 8 illustrates an example of a processing flow executed by the video search apparatus 1 of the present invention configured as described above.

  That is, as shown in the processing flow of FIG. 8, the video search device 1 of the present invention configured as described above first acquires the contour information drawn by the user in step 20. Unprocessed video is read from the video database 10.

  Subsequently, in step 22, contour information is extracted from the read video, and in step 23, the similarity between the contour information drawn by the user and the extracted contour information is determined. In step 24, the video database is extracted. If it is determined whether or not all the images stored in 10 have been processed and it is determined that all the images have not been processed, the process returns to step 21.

  In this way, by repeating the processing of step 21 to step 24, the similarity between the contour information drawn by the user and all the contour information extracted from the video stored in the video database 10 is determined. Proceed to, and totalize the similarities.

  Subsequently, in step 26, a video with a high degree of similarity (subject video) is selected, and in step 27, the position and size of the selected video are adjusted to be displayed in the area drawn by the user.

  Finally, in step 28, the adjusted video is reproduced in the area drawn by the user.

  In this way, the video search device 1 of the present invention searches for a video of a subject having the same contour as the contour drawn by the user, and searches the subject at the same position in the region drawn by the user. It is processed so that only the video portion of the image is reproduced.

  If it is desired to display the entire search result video, it may be performed in response to subsequent user operations. Further, although the video has been described here, a still image can be searched according to the same processing.

  According to the present invention, if the contour of the subject shown in the video is specified, a video having the same contour as the contour can be searched. Therefore, when providing various services using the video, the service is used. It can be used to build as easy.

It is a figure which shows an example of the outline of the to-be-photographed object which a user draws. It is a figure which shows an example of the outline information extracted from an image | video. It is a figure explaining collation of outline information. It is a figure which shows an example of the display of the search result in this invention. 1 is a first exemplary embodiment of the present invention. It is an example of the processing flow which this invention performs. It is a 2nd example of an embodiment of the present invention. It is an example of the processing flow which this invention performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image | video search apparatus 10 Image | video database 11 Outline information database 12 Subject outline extraction part 13 Display with a touch panel 14 Drawing information acquisition part 15 Similarity determination part 16 Similarity totalization part 17 Display area adjustment part 18 Video reproduction part

Claims (6)

A video search device for searching for a video based on a figure of a subject included in the video,
Acquisition means for acquiring information indicating the contour of the subject drawn by the user;
By determining the degree of similarity between the contour information stored in the storage means for storing the contour information of the subject included in the video to be searched and the contour information acquired by the acquisition means, Search means for searching for one or more contour information similar to the contour information acquired by the acquisition means;
Adjusting means for adjusting the video portion so that the video portion having the contour information searched by the search means matches the subject region defined by the contour information acquired by the acquisition means;
Display means for displaying the image portion adjusted by the adjusting means in a subject area defined by the contour information acquired by the acquiring means;
A featured video search device. A video search device for searching for a video based on a figure of a subject included in the video,
Acquisition means for acquiring information indicating the contour of the subject drawn by the user;
An extraction means for reading out the video from the storage means for storing the video to be searched and extracting the contour information of the subject included in the video;
By determining the similarity between the contour information extracted by the extracting means and the contour information acquired by the acquiring means, the contour information extracted by the extracting means is similar to the contour information acquired by the acquiring means. Search means for searching for one or more contour information;
Adjusting means for adjusting the video portion so that the video portion having the contour information searched by the search means matches the subject region defined by the contour information acquired by the acquisition means;
Display means for displaying the image portion adjusted by the adjusting means in a subject area defined by the contour information acquired by the acquiring means;
A featured video search device. An image search method for searching for an image based on a figure of a subject included in the image,
Acquiring information indicating the contour of the subject drawn by the user;
By determining the degree of similarity between the contour information stored in the storage means for storing the contour information of the subject included in the search target video and the acquired contour information, Searching for one or more contour information similar to the acquired contour information;
Adjusting the video portion so that the video portion having the searched contour information matches the subject area defined by the acquired contour information;
Displaying the adjusted video portion in a subject area defined by the acquired contour information,
A featured video search method. An image search method for searching for an image based on a figure of a subject included in the image,
Acquiring information indicating the contour of the subject drawn by the user;
A process of reading the video from the storage means for storing the video to be searched and extracting the contour information of the subject included in the video;
A process of searching for one or a plurality of contour information similar to the acquired contour information from the extracted contour information by determining a similarity between the extracted contour information and the acquired contour information; ,
Adjusting the video portion so that the video portion having the searched contour information matches the subject area defined by the acquired contour information;
Displaying the adjusted video portion in a subject area defined by the acquired contour information,
A featured video search method. A video search program for performing a process of searching for a video using a figure of a subject included in the video as a clue,
A process of acquiring information indicating the contour of the subject drawn by the user;
By determining the degree of similarity between the contour information stored in the storage means for storing the contour information of the subject included in the search target video and the acquired contour information, A process of searching for one or more contour information similar to the acquired contour information;
A process of adjusting the video portion so that the searched video portion having the contour information matches a subject area defined by the acquired contour information;
A video search program for causing a computer to execute processing for displaying the adjusted video portion in a subject area defined by the acquired contour information. A video search program for performing a process of searching for a video using a figure of a subject included in the video as a clue,
A process of acquiring information indicating the contour of the subject drawn by the user;
A process of reading the video from the storage means for storing the video to be searched and extracting the contour information of the subject included in the video;
A process of searching for one or a plurality of contour information similar to the acquired contour information from the extracted contour information by determining a similarity between the extracted contour information and the acquired contour information; ,
A process of adjusting the video portion so that the searched video portion having the contour information matches a subject area defined by the acquired contour information;
A video search program for causing a computer to execute processing for displaying the adjusted video portion in a subject area defined by the acquired contour information.

Images