JP2007047902A - Image retrieval device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image retrieval device, in which the summary of an image can be obtained to facilitate image retrieval. <P>SOLUTION: A characteristic information extraction part 7 extracts predetermined characteristic quantity data from the image. An extracted characteristic information storage part 9 stores the characteristic quantity data extracted by the extraction part 7 with extraction time data. A characteristic expression part 19 of a screen display part 11 generates, based on the characteristic quantity data and the extraction time data, a characteristic expression pattern expressing the magnitude of the characteristic concerned at each extraction time on the time axis of the image, and displays the characteristic expression pattern on a display device 13. The characteristic expression part 19 generates, as the characteristic expression pattern, a graph having a first axis showing the time axis and a second axis showing the magnitude of the characteristic quantity. Further, the expression part 19 adds a mark such as icon to a point of the characteristic expression pattern where the characteristic quantity is a predetermined threshold or more. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a video search apparatus used for searching a desired video portion from a video.

  Conventionally, a video search device is used, for example, to search for a desired video portion from a monitoring video taken by a monitoring camera. In the case of a surveillance system, the desired video part is the part to which the event to be monitored has moved. A conventional general video search apparatus displays a video together with the shooting date and time. The operator reproduces the video depending on the shooting date and searches for a desired video portion.

  Patent Document 1 discloses a video search device that facilitates video search. The video search apparatus of the document displays the appearance section for each character as a bar figure on the time axis. The operator can search for a desired video portion by selecting a desired appearance section of a desired character.

Also, the video search device of Patent Document 2 extracts still images from video at predetermined time intervals, arranges the still images, and displays the date and time of the time code together with each still image. The user can search for a desired video portion by selecting a still image and playing a video from the selected still image.
Japanese Patent Laying-Open No. 2004-127711 (for example, page 5-6, FIG. 1) JP-A-11-146325 (for example, page 9 and FIG. 18)

  However, in the conventional video search apparatus, the operator has to search for a desired video portion while playing back the video by relying on the date / time information. Therefore, there is a problem that the search is difficult and time-consuming and the search is not easy. It was.

  Further, in the techniques of Patent Document 1 and Patent Document 2, although the video search has been facilitated to some extent, it has not been easy to grasp the entire video such as where the characteristic part appears in the video. For example, when searching for surveillance video, it is desirable to intuitively understand the outline of where the featured part of the monitoring target is reflected, and if such an overall outline can be grasped, video search is possible. Will be even easier.

  The present invention has been made in order to solve the conventional problems, and an object of the present invention is to provide a video search apparatus that can grasp a video overview and facilitate video search.

  The video search apparatus of the present invention includes a video recording unit that records a captured video, a feature information extraction unit that extracts predetermined feature amount data from the video, and a feature amount data extracted by the feature information extraction unit. An extracted feature information storage unit that stores the extracted time point data together with the extracted feature amount data and the extracted time point data based on the extracted feature amount data and the extracted time point data. And a feature expression unit that generates a feature expression figure represented by a figure at the time of extraction and displays the feature expression figure on a display device. The feature representation graphic is a representation of the size of the feature amount data by a graphic diagram such as a point, a line, or a surface, and is, for example, a bar graph in an embodiment described later.

  With this configuration, feature value data is extracted from the video and stored together with the extracted time point data, and a feature representation graphic that represents the size of the corresponding feature value at each extraction time point on the video time axis is generated and displayed. Therefore, the outline of the video can be grasped from the displayed feature expression graphic, thereby facilitating video search.

  The video search device of the present invention further includes a trigger detection unit that detects the occurrence of an extraction trigger corresponding to a predetermined characteristic event, and the feature information extraction unit detects the feature when the occurrence of the extraction trigger is detected. Quantity data is extracted, and the feature expression unit generates the feature expression figure based on the feature quantity data extracted by the feature information extraction unit when the extraction trigger occurs.

  With this configuration, the feature amount when the extraction trigger corresponding to the feature event occurs is displayed at the location of the feature representation figure at the time of the corresponding extraction, so at what point the feature event occurred and how much feature amount at that time It is possible to display a feature expression figure that can easily grasp the outline of the video such as whether it was the size of the video.

  In the video search device of the present invention, the trigger detection unit includes a motion detection unit that detects a motion of a subject in the video as the extraction trigger. With this configuration, it is possible to appropriately detect occurrence of a characteristic event and extract feature amount data.

  In the video search device of the present invention, the feature expression unit generates a graph in which a first axis is a time axis and a second axis is the size of the feature amount as the feature expression figure. With this configuration, it is possible to provide a feature expression figure that makes it easy to grasp the outline of the video.

  In the video search device of the present invention, the feature information extraction unit extracts predetermined color representative value data from the video as the feature amount data. The color representative value data is, for example, a total value or an average value of specific color values of RGB. By using the color representative value data, the feature amount can be suitably expressed with a relatively simple process and a relatively small amount of data.

  In the video search device of the present invention, the feature expression unit adds a predetermined mark to a place where a feature amount equal to or greater than a predetermined feature threshold value is generated in the feature expression figure. The mark is, for example, an icon. With this configuration, it is possible to provide a feature expression figure that can easily grasp a portion having a large feature amount.

  In the video search device of the present invention, the extracted feature information storage unit stores a small still image at each extraction point together with the feature amount data and the extraction point data, and the feature expression unit includes the feature expression figure. In response to the temporary selection operation for the mark in the middle, the small still image corresponding to the selected mark is displayed on the display device. The small still image is, for example, a thumbnail image. The mark selection by the temporary selection operation is input from an input operation unit such as a pointing device. With this configuration, by retrieving and displaying a small still image corresponding to the selected mark, it is possible to help grasp the content of the video and facilitate the video search.

  Also, the video search apparatus of the present invention detects that the pointer figure of the pointing device is positioned on the mark as the temporary selection operation. With this configuration, a small still image can be displayed by moving the pointer graphic and positioning it on the mark, and the video content can be grasped by a simple operation.

  In addition, the video search apparatus of the present invention acquires video data corresponding to the mark from the video recording unit in response to an input of a selection determination operation on the mark and causes the display device to display the video data. For example, the video search device displays a moving image for a predetermined time at the time when the mark is located on the time axis. The selection determination operation is input from the input operation unit, and the selection determination instruction is, for example, a mouse click operation. With this configuration, by searching for and displaying a video corresponding to the selected mark, it is possible to help grasp the content of the video and facilitate video search.

  In the video search device of the present invention, the extracted feature information storage unit stores a small still image at each extraction point together with the feature amount data and the extraction point data, and the feature expression unit includes the feature expression figure. In response to the provisional selection operation for the feature amount data in the medium, the small still image corresponding to the selected feature amount data is displayed on the display device. The selection of the feature amount data by the temporary selection operation is input from an input operation unit such as a pointing device. The above-described mark may be displayed, and the feature amount data may be selected by a temporary selection operation for selecting the mark. The small still image is, for example, a thumbnail image. With this configuration, a small still image corresponding to the selected feature amount data is retrieved and displayed, so that it is possible to help grasp the video content and facilitate the video search.

  The video search device of the present invention obtains video data corresponding to the selected feature data from the video recording unit in response to an input of a selection determination operation on the feature data in the feature representation figure. Display on the display device. For example, a moving image of a predetermined time at a selected time point on the time axis is displayed. The selection determination operation is input from the input operation unit, and the selection determination instruction is, for example, a mouse click operation. The above-described mark may be displayed, and feature amount data may be selected by a mark selection / determination operation. With this configuration, since the video at the time corresponding to the selected feature data is searched and displayed, it is possible to help grasp the video content and facilitate video search.

  The video search device according to another aspect of the present invention includes an extracted feature information storage unit that stores predetermined feature amount data extracted from a captured video together with extraction time point data, and the feature read from the extracted feature information storage unit. Based on the quantity data and the extraction time point data, a feature expression figure in which the size of the extracted feature quantity data is represented by a figure at the extraction time point on the time axis of the video is generated, and the feature expression figure is displayed on the display device And a feature expression unit to be displayed. This configuration also provides the above-described advantages.

  According to another aspect of the present invention, there is provided an extracted feature information storage that stores a predetermined feature amount data extracted from a captured video together with extraction time data, which is a program executed by a computer constituting the video search device. The feature amount data and the extraction point data are read from the unit, and based on the feature amount data and the extraction point data, the size of the extracted feature amount data is graphically represented at the extraction point on the time axis of the video The computer is caused to execute processing for generating an expression graphic and displaying the characteristic expression graphic on a display device. This configuration also provides the above-described advantages.

  The present invention extracts feature quantity data from a video and stores it together with the extraction time point data, and generates and displays a feature expression figure that represents the size of the corresponding feature quantity at each extraction time point on the video time axis. By doing so, it is possible to provide a video search apparatus having an effect that the outline of the video can be grasped from the displayed feature expression graphic, and thereby the video search can be facilitated.

  Hereinafter, a video search apparatus according to an embodiment of the present invention will be described with reference to the drawings. In this embodiment, the video search device of the present invention is applied to a monitoring system and has a function of supporting a work for searching for a desired video portion from a monitoring video. The video search device is realized by a computer having an image processing function.

  A video search apparatus according to an embodiment of the present invention is shown in FIG. In FIG. 1, the video search device 1 includes a video recording unit 5 that records a video captured by the imaging unit 3, a feature information extraction unit 7 that extracts predetermined feature amount data from the video, and a feature information extraction unit 7. An extracted feature information storage unit 9 that stores the extracted feature data together with the extraction time data, a screen display unit 11 that displays a video on a display 13, and an operation unit 15 including a pointing device are provided. The screen display unit 11 has a feature representation unit 19 together with the video display unit 17, and the feature representation unit 17 generates a feature representation figure, which will be described later, using the feature amount data in the extracted feature information storage unit 9 and displays it on the display 13. Thus, the video search function of the present embodiment is realized. Hereinafter, the configuration of each part of the video search device 1 will be described.

  The imaging unit 3 is a surveillance camera and is installed at a surveillance location. The imaging unit 3 has an optical system such as a lens, an imaging element such as a CCD, and a configuration for processing an image signal generated by the imaging element. The imaging unit 3 captures a monitoring location and outputs moving image data as a monitoring video.

  The video recording unit 5 is a recording device such as a hard disk or a DVD, and records the video (moving image data) captured by the imaging unit 3. The video data is recorded together with a time code specifying the shooting date and time (shooting time) of each part in the video.

  The feature information extraction unit 7 is configured by a computer device having an image processing function. The feature information extraction unit 7 takes in the video imaged by the imaging unit 3 and extracts predetermined feature amount data from the video image. The feature information extraction unit 7 is configured to extract feature amount data when the trigger detection unit detects the occurrence of an extraction trigger corresponding to a predetermined feature event. In the present embodiment, the trigger detection unit is a motion detection unit that detects the movement of the subject in the video as an extraction trigger, and is provided in the feature information extraction unit 7 itself.

  In this embodiment, color representative value data is extracted as the feature amount data. The feature amount data is stored in the extracted feature information storage unit 9 together with the extraction time of the feature amount data. At this time, a thumbnail image is generated from the video frame at the time of extraction, and the thumbnail image is also recorded in the extracted feature information storage unit 9 in association with the feature amount data.

  FIG. 2 shows the configuration of the feature information extraction unit 7 together with the data stored in the extraction feature information storage unit 9. In the feature information extraction unit 7, the video input unit 21 inputs the video captured by the imaging unit 3 from the imaging unit 3 or the video recording unit 5.

  The motion detector 23 detects the occurrence of motion of the subject in the video. Each time a new frame image is input, the motion detection unit 23 compares the new frame image with the previous frame image and detects the motion that has occurred in the new frame. The motion detection unit 23 calculates a total difference value between two consecutive frame images in the video. The difference total value is the sum of the pixel value differences of the corresponding pixels in the two images. The motion detection unit 23 compares the difference total value with a predetermined motion detection threshold, and determines that a motion has occurred if the difference total value is equal to or greater than the motion detection threshold.

  When the motion detection unit 23 detects a motion, the color data extraction unit 25 extracts color representative value data as feature amount data from the image data of the frame in which the motion is detected. In the present embodiment, the color representative value data is the total value of each color of R (red), G (green), and B (blue) in one frame image at the time of extraction. Therefore, the three color representative value data Is calculated.

  The date and time acquisition unit 27 acquires date and time data when the color representative value data is extracted by the color data extraction unit 25, that is, when the motion detection unit 23 detects motion. This date / time data is referred to as extracted date / time data. The extraction date / time data is acquired from a clock built in the computer device constituting the feature information extraction unit 7. This extraction date / time data corresponds to the extraction time point data of the present invention.

  The thumbnail generation unit 29 generates a thumbnail image from the image data from which the color representative value data is extracted, that is, the image data of the frame when the motion detection unit 23 detects the motion. A thumbnail image is a typical example of a small still image.

  The feature information storage processing unit 31 extracts the color representative value data extracted by the color data extraction unit 25, the extracted date / time data acquired by the date / time acquisition unit 27, and the thumbnail image 29 generated by the thumbnail generation unit 29. Stored in section 9. The color representative value data, the extraction date / time data, and the thumbnail image are stored in association with each other as shown in FIG. These data are stored for each video file. The extracted feature information storage unit 9 is a storage device such as a hard disk or a DVD. The extracted feature information storage unit 9 and the video recording unit 5 may be physically the same storage device.

  More specifically, for example, the extraction feature information storage unit 9 is configured to store a table of extraction date / time data and color representative value data obtained from video, and to store thumbnail images in association with the extraction date / time data. ing. The feature information extracting unit 7 updates the table of the extracted feature information storage unit 9 to add the extracted date / time data and the color representative value data every time a motion in the video is detected, and the thumbnail image together with the extracted date / time data. Write to the extracted feature information storage unit 9.

  Next, the configuration of the screen display unit 11 will be described with reference to FIG. The screen display unit 11 is composed of a computer device having an image processing function. The screen display unit 11 and the feature information extraction unit 7 may be configured by the same computer or may be configured by separate computers.

  In the screen display unit 11, the video display unit 17 reads video data from the video recording unit 5 in accordance with an instruction input from the operation unit 15 and displays the video on the display 13. The operation unit 15 includes a keyboard and a pointing device, and the pointing device is, for example, a mouse.

  For example, the screen display unit 11 displays a video selection menu on the screen. The video selection menu screen is a menu screen for selecting a video to be displayed from a plurality of videos recorded in the video recording unit 9. Then, the operation unit 15 inputs the selection of the video, and the video display unit 17 reads the selected video from the video recording unit 5 and displays it on the display 13. When the video is encoded and recorded in the video recording unit 5, the video display unit 17 may have an appropriate decoding function. The video display unit 17 also displays function buttons such as play, stop, fast forward, rewind, and pause on the screen. When an instruction for pressing a function button is input from the operation unit 15, the video display unit 17 performs an operation corresponding to the pressed button.

  The feature expression unit 19 has a configuration that functions for video search in accordance with an instruction input from the operation unit 15 including a pointing device.

  In the feature expression unit 19, the feature information search unit 41 reads data corresponding to the video selected by the video display unit 17 from the extracted feature information storage unit 9. The data to be read out is feature amount data, extraction date / time data, and thumbnail images. All data related to the selected video is read out and held in the memory of the screen display unit 11. The feature search instruction unit 43 receives an operator instruction input from the operation unit 15 and transmits it to the feature information search unit 41. The feature display unit 45 is controlled by the feature information search unit 41 to generate an image for search and display it on the display 13.

  In the feature display unit 45, the feature expression graphic generation unit 51 generates a feature expression graphic. The feature expression graphic is a graphic generated based on the feature amount data and the extraction date / time data and representing the size of the corresponding feature amount at each extraction time on the video time axis. The size of the feature amount is represented by a figure such as a point, a line, or a surface. In the case of the present embodiment, the feature amount expression graphic is a bar graph.

  Further, in the present embodiment, the feature representation graphic generation unit 51 generates R (red), G (green), and B (blue) color representative value data when generating the feature representation graphic. Red) color representative value data is used as feature amount data. Then, the feature expression graphic generation unit 51 generates a characteristic expression graphic expressing the size of R (red) at each extraction time point.

  The mark generation unit 53 adds a predetermined mark to a place where a feature amount equal to or greater than a predetermined threshold value is generated in the feature expression graphic. For this process, a predetermined feature amount threshold value is stored in advance. The mark generation unit 53 compares each feature amount (R representative value data) with a feature amount threshold value. If the feature amount is equal to or greater than the threshold value, a mark is attached to the location of the corresponding feature amount. An icon is added as the mark.

  The thumbnail image display unit 55 displays the thumbnail image instructed from the feature information search unit 41 at a predetermined location on the screen of the display 13.

  FIG. 4 shows a screen of the display 13 displaying the feature expression graphic. In order to display this screen, the screen display unit 11 reads the stored GUI image resource, and further reads and activates a feature expression program for realizing the feature expression unit 19. Thereby, it becomes possible to display the feature expression graphic on the screen display unit 11, and the characteristic expression graphic is generated and displayed.

  The screen in FIG. 4 includes a video area 61, a feature expression area 63, and a thumbnail area 65. The video area 61 is an area for displaying a moving image as a recorded monitoring video. The video area 61 occupies most of the screen.

  The feature expression area 63 is an area for displaying a feature expression figure. The feature expression area 63 is an elongated area that is disposed below the screen and extends in the horizontal direction. In the present embodiment, the illustrated graph is displayed as the feature expression graphic. In this graph, the horizontal axis is the time axis, and the vertical axis is the size of the feature amount. As described above, R (red) representative value data is used as the feature amount. In the horizontal direction, the time including the entire range of the video is displayed. A bar-shaped diagram corresponding to the size of the feature amount is arranged at the location of the corresponding extraction date. Such a graph is generated by the feature expression graphic generation unit 51 and displayed in a predetermined feature expression area 61 of the display 13.

  Further, as shown in FIG. 4, a line L representing a feature amount threshold value is displayed on the graph of the feature expression figure. A mark M is added to the apex of the feature amount bar where the feature amount is equal to or greater than the threshold value. In the present embodiment, the mark M is a small icon.

  A thumbnail area 65 is provided on the right side of the feature expression area 63. In the thumbnail area 65, thumbnail images are displayed. The thumbnail image display unit 55 is configured to display a thumbnail image in the thumbnail area 65 in accordance with the following operation.

  The operator moves the pointer graphic of the pointing device on the screen and positions the pointer graphic on one mark M. This operation corresponds to the provisional selection operation of the present invention. The pointer graphic may stop on the mark M or pass through the mark M. When this operation is performed, the screen display unit 11 (feature expression unit 19) searches for and displays a thumbnail image corresponding to the selected mark M. At this time, in the screen display unit 11 of FIG. 3, the mark M temporary selection operation is input from the operation unit 15 and accepted by the feature search instruction unit 43 of the feature expression unit 19. The feature search instruction unit 43 issues a search instruction indicating the selected mark M to the feature information search unit 41. The feature information search unit 41 searches for a thumbnail image corresponding to the selected mark M. At this time, the feature information search unit 41 specifies the extraction date / time data of the feature amount data to which the selected mark M is attached, and further specifies a thumbnail image associated with the extraction date / time data. Then, the feature information search unit 41 instructs the thumbnail image display unit 55 to display the searched thumbnail image. The thumbnail image display unit 55 displays the thumbnail image in the thumbnail area 65 of the screen according to the instruction from the feature information search unit 41.

  Furthermore, the screen display unit 11 is configured to display the searched video according to the mark selection according to the following operation.

  It is assumed that the operator further performs a predetermined selection determination operation on the pointing device on one mark M. For example, the pointing device is a mouse, and a mouse button is clicked. Single click may be set to the selection determination operation, and double click may be set to the selection determination operation. This selection decision operation is input and accepted by the feature search instruction unit 43 of the feature expression unit 19. The feature search instruction unit 43 sends a search instruction indicating a selection determination operation together with the selected mark M to the feature information search unit 41. The feature information search unit 41 obtains the extraction date / time data of the feature amount data to which the selected mark M is attached, and informs the video display unit 17 of the extraction date / time. The video display unit 17 searches the video recording unit 9 for the video of the extraction date and time transmitted from the feature information search unit 41. At this time, the video display unit 17 may search for the video using the video time code. The video display unit 17 reads the searched video and displays it on the display 13. A video of a predetermined time at the extraction time (the time of the selection mark) is displayed. For example, an image of about 3 seconds is displayed. The video may be displayed longer. The extraction time point may be the start point of the video. Further, the video display unit 17 may display a video of a predetermined time before and after the extraction time point.

  The configuration of each part of the video search device 1 according to the present embodiment has been described above. Next, the operation of the video search device 1 will be described.

  FIG. 5 shows an operation of extracting feature amount data by the feature information extraction unit 7. As shown in FIG. 5, in the feature information extraction unit 7, the motion detection unit 23 performs a motion detection process every time one frame image is input by the video input unit 21 (S11). It is determined whether or not (S13). If no movement has occurred, the process returns to step S11.

  When a motion occurs, the feature information extraction unit 7 determines whether or not a predetermined time has elapsed since the previous feature amount extraction (S15). The predetermined time is set in order to avoid excessively frequent feature extraction. The predetermined time is, for example, 5 seconds. The predetermined time may be represented by the number of frames. If the determination in step S15 is No, feature amount extraction is not performed, and the process returns to step S11.

  If the determination in step S15 is Yes, the color data extraction unit 25 extracts color representative value data as feature amount data from the image data of the frame in which the motion is detected (S17). The color representative value data is a total value of each color of R (red), G (green), and B (blue). In addition, the date acquisition unit 27 acquires extracted date data that is the date of motion detection (S19). Further, the thumbnail generation unit 29 generates a thumbnail image from the image data of the frame at the time of detecting the motion (S21). The feature amount data (color representative value data), the extraction date / time data, and the thumbnail image are stored in the extracted feature information storage unit 9 (S23), and the process of the feature information extraction unit 7 returns to step S11.

  In this way, motion detection and feature amount extraction are repeated. The extracted feature information storage unit 9 accumulates feature amount data, extracted date / time data, and thumbnail images obtained from the video every time a motion occurs.

  Next, FIG. 6 shows the operation of the screen display unit 11. FIG. 6 mainly shows the operation of the feature expression unit 19 of the screen display unit 11.

  As shown in FIG. 6, first, one video file is selected from a plurality of video files recorded in the video recording unit 5 (S31). This video selection is input from the operation unit 15. The feature information search unit 41 of the feature expression unit 19 reads out and acquires all feature amount data, extraction date / time data, and thumbnail images corresponding to the selected video file from the extracted feature information storage unit 9 (S33). And the characteristic display part 45 produces | generates the graph of a feature expression figure from the acquired feature-value data and extraction date data (S35).

  At this time, the time axis in the horizontal direction is set so that the entire time of the video enters. Then, the feature representation graphic generation unit 51 of the feature display unit 45 uses the first pair of the feature amount data and the extraction date / time data to have a length corresponding to the size of the feature amount at the location of the extraction date / time. Place a bar figure. A representative value of R (red) is processed as the feature amount. In addition, the mark generation unit 53 compares the feature amount with a threshold value, and if the feature amount is equal to or greater than the threshold value, adds an icon mark M to the upper end of the feature amount bar graphic. Similar processing is repeated for all pairs of feature amount data and extraction date data. Further, the feature representation graphic generation unit 51 draws the mark addition threshold line L in the graph. The feature expression figure generated in this way is displayed on the display 13 by the feature display unit 45 (S37). As described with reference to FIG. 4, the feature representation graphic is displayed in a horizontally long feature representation region 63 provided below the screen avoiding the video region screen 61.

  Next, FIG. 7 shows the operation of displaying thumbnail images and search video. The operation of FIG. 7 is performed when the operator places the pointer figure of the pointing device on one mark M (icon) on the screen. This operation corresponds to a temporary selection operation.

  In FIG. 7, when the pointer graphic moves and is positioned on one mark M (S41), this operation is accepted by the feature search instruction unit 43, and the feature search instruction unit 43 searches for the selected mark M. An instruction is sent to the feature information search unit 41 (S43). The feature information search unit 41 searches for a thumbnail image corresponding to the selected mark M. At this time, the feature information search unit 41 obtains the extraction date / time data of the feature amount data with the mark M, identifies the thumbnail image associated with the extraction date / time data, and stores the corresponding thumbnail image in the thumbnail image display unit 55. Display. The thumbnail image display unit 55 displays the corresponding thumbnail image in a thumbnail area 65 provided beside the feature expression area 63 in FIG. 4 (S45).

  It is assumed that a predetermined selection determination operation is further input from the operation unit 15 and received by the feature search instruction unit 43 in a state where the pointer graphic is positioned on the mark (S47, Yes). As described above, the predetermined selection determination operation is a mouse click or the like. In this case, the feature information search unit 41 informs the video display unit 17 of the extraction date / time data corresponding to the target mark for the selection determination operation. The video display unit 17 searches the video recording unit 5 for a video corresponding to the extracted date and time data, and displays the searched video on the display 13 (S49). The corresponding video is read from the video recording unit 5 and displayed in the video area 61 of the display 13.

  Since the video search apparatus 1 performs the operation of FIG. 7, the operator can operate the video search apparatus 1 as follows. The operator grasps where the extraction trigger (movement of the subject) corresponding to the feature event occurred in the video by looking at the graph of the feature expression figure, and the level of the feature amount when each feature event occurs Is grasped from the graph, and thus the outline of the video is grasped. The operator moves the pointer graphic of the pointing device in the time axis direction (left-right direction) and moves the pointer graphic so as to pass through the marks M displayed in the graph one after another. Each time the pointer graphic passes through the mark M, a thumbnail image corresponding to the mark M is displayed. When a thumbnail image to be noticed is displayed, the operator performs a predetermined selection determination operation such as a mouse click in a state where the pointer graphic is on the mark M. This selection determination operation is input as an instruction to display a search video, and a video corresponding to the point in time when the mark M is arranged on the time axis is searched and displayed.

  The operation of the video search device 1 has been described above. Next, a modification of the video search device 1 will be described. First, in the above embodiment, the video input unit 21 of the feature information extraction unit 7 detects a motion by comparing the difference between frames with a threshold value. However, the image processing for motion detection is not limited to this.

  The motion detector is a form of the trigger detector. However, the trigger detection unit may have other configurations. The trigger detection unit may be configured by a sensor such as a sound sensor, an optical sensor, or an object detection sensor. The trigger detection unit may be an open / close sensor for a door at a monitoring place.

  In the present embodiment, the feature amount data is R (red), G (green), and B (blue) color representative value data, and R representative value data is used to create a feature representation figure. Used. As a modification, only R representative value data may be extracted. As another modification, other color representative value data such as G or B may be used. The color representative value data used for drawing may be switchable between R, G, and B. Further, the feature amount data may be data other than color as long as it is data representing the feature of the video. For example, the feature amount data may be video motion amount data such as a motion vector. The feature information extraction unit 7 is configured to have an extraction function according to feature amount data.

  In addition, regarding the feature expression graphic, in the present embodiment, a graph of the entire time of the video is displayed at a time. However, a partial graph of the video may appear on the screen, and the graph may be movable in the time axis direction by scrolling.

  Further, the scales of the horizontal axis and the vertical axis of the feature representation figure may be switchable. For example, the scale of the feature amount on the vertical axis is suitably switched between a daytime video and a nighttime video.

  Further, the feature expression graphic may be a graphic other than the bar graph. For example, the feature expression graphic may be a line graph. The size of the feature amount may be expressed by the difference in color, and this is also included in the feature expression graphic. However, the bar graph of the present embodiment is particularly advantageous in that the operator can intuitively and easily grasp the outline of the video.

  The video search device 1 according to the embodiment of the present invention has been described above. The video search device 1 of the present embodiment extracts feature quantity data from a video, stores it together with the extraction time data, and represents a feature expression figure that represents the size of the corresponding feature quantity at each extraction time on the video time axis. Is generated and displayed, so that an outline of the video can be grasped from the displayed feature expression graphic, thereby facilitating video search.

  In addition, according to the present embodiment, since the feature amount when the extraction trigger corresponding to the feature event occurs is displayed at the location of the feature expression graphic at the corresponding extraction time point, It is possible to display a feature expression figure that can easily grasp the outline of the video such as how large the feature amount was.

  Further, according to the present embodiment, the feature information extraction unit detects the movement of the subject in the video as an extraction trigger, so that it is possible to appropriately detect the occurrence of the feature event and extract the feature amount data.

  Further, according to the present embodiment, the feature expression unit 19 generates a graph in which the first axis is the time axis and the second axis is the size of the feature amount as the feature expression figure. Therefore, it is possible to provide a feature expression figure that makes it easy to grasp the outline of the video.

  Further, according to the present embodiment, the feature information extraction unit 7 extracts predetermined color representative value data from the video as the feature amount data. By using the color representative value data, the feature amount can be suitably expressed with a relatively simple process and a relatively small amount of data.

  Further, according to the present embodiment, the feature expression unit 19 adds a predetermined mark such as an icon to a place where a feature amount equal to or greater than a predetermined feature threshold value occurs in the feature expression figure. With this configuration, it is possible to provide a feature expression figure that can easily grasp a portion having a large feature amount.

  Further, according to the present embodiment, the small still image at each extraction time is stored together with the feature amount data and the extraction time data. In response to the provisional selection operation for the mark, a small still image corresponding to the selected mark is displayed. With this configuration, by retrieving and displaying a small still image corresponding to the selected mark, it is possible to help grasp the content of the video and facilitate the video search.

  Further, according to the present embodiment, the video search apparatus 1 detects that the pointer graphic of the pointing device is positioned on the mark as a temporary selection operation. With this configuration, a small still image can be displayed by moving the pointer graphic and positioning it on the mark, and the video content can be grasped by a simple operation.

  Further, according to the present embodiment, the video search apparatus 1 acquires video data corresponding to a mark from the video recording unit in response to an input of a selection determination operation for the mark and causes the display device to display the video data. With this configuration, by searching for and displaying a video corresponding to the selected mark, it is possible to help grasp the content of the video and facilitate video search.

  From another point of view, the video search apparatus 1 realizes a temporary selection operation and a selection determination operation of feature data of interest by a temporary mark selection operation and a selection determination operation. In this embodiment, since a small still image corresponding to the feature amount data selected by the provisional selection operation is retrieved and displayed, it is possible to help grasp the contents of the video and facilitate the video search. Further, in the present embodiment, since the video at the time corresponding to the feature amount data selected by the selection determination operation is searched and displayed, it is possible to help grasp the video content and facilitate the video search.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments, and it goes without saying that those skilled in the art can modify the above-described embodiments within the scope of the present invention.

  As described above, the video search device according to the present invention extracts feature data from a video and stores it together with the extraction time data, and expresses the size of the corresponding feature data at each extraction time on the video time axis. By generating and displaying the feature expression graphic, it is possible to grasp the outline of the video from the displayed characteristic expression graphic, thereby having an effect of facilitating the video search, which is useful as a monitoring system or the like.

Block diagram of video search apparatus in an embodiment of the present invention Block diagram showing the configuration of the feature information extraction unit Block diagram showing the configuration of the screen display Schematic diagram showing a screen display example Flow chart showing operation of video search device Flow chart showing operation of video search device Flow chart showing operation of video search device

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image | video search device 3 Image pick-up part 5 Image | video recording part 7 Feature information extraction part 9 Extraction feature information storage part 11 Screen display part 13 Display 15 Operation part 17 Video display part 19 Feature expression part 21 Video input part 23 Motion detection part 25 Color data Extraction unit 27 Date and time acquisition unit 29 Thumbnail generation unit 31 Feature information storage processing unit 41 Feature information search unit 43 Feature search instruction unit 45 Feature display unit 51 Feature expression graphic generation unit 53 Mark generation unit 55 Thumbnail image display unit

Claims (13)

A video recording unit for recording the captured video;
A feature information extraction unit for extracting predetermined feature data from the video;
An extracted feature information storage unit that stores feature amount data extracted by the feature information extraction unit together with extraction time point data;
Based on the feature value data read from the extracted feature information storage unit and the extraction time point data, a feature expression figure in which the size of the extracted feature amount data is represented as a graphic at the extraction time point on the time axis of the video Generating and displaying the feature representation graphic on a display device;
A video search apparatus comprising: A trigger detection unit that detects the occurrence of an extraction trigger corresponding to a predetermined characteristic event,
The feature information extraction unit extracts the feature amount data when occurrence of the extraction trigger is detected,
The video search apparatus according to claim 1, wherein the feature expression unit generates the feature expression figure based on the feature amount data extracted by the feature information extraction unit when the extraction trigger occurs. .   The video search apparatus according to claim 2, wherein the trigger detection unit includes a motion detection unit that detects a motion of a subject in the video as the extraction trigger.   The feature expression unit generates, as the feature expression figure, a graph in which a first axis is a time axis and a second axis is the size of the feature amount. The video search device according to any one of the above.   5. The video search apparatus according to claim 1, wherein the feature information extraction unit extracts predetermined color representative value data from the video as the feature amount data.   The video according to claim 1, wherein the feature expression unit adds a predetermined mark to a place where a feature amount equal to or greater than a predetermined feature threshold is generated in the feature expression graphic. Search device. The extracted feature information storage unit stores a small still image at each extraction point together with the feature amount data and the extraction point data,
The feature expression unit displays the small still image corresponding to the selected mark on the display device in response to a provisional selection operation on the mark in the feature expression figure. The video search device described in 1.   The video search apparatus according to claim 7, wherein the provisional selection operation detects that a pointer graphic of a pointing device is positioned on the mark.   The video data corresponding to the mark is acquired from the video recording unit and displayed on the display device in response to an input of a selection determination operation on the mark. Video search device. The extracted feature information storage unit stores a small still image at each extraction point together with the feature amount data and the extraction point data,
The feature representation unit displays the small still image corresponding to the selected feature amount data on the display device in response to a provisional selection operation on the feature amount data in the feature representation figure. The video search device according to claim 1.   In response to an input of a selection determination operation on feature quantity data in the feature representation figure, video data corresponding to the selected feature quantity data is acquired from the video recording unit and displayed on the display device. The video search device according to claim 1. An extracted feature information storage unit that stores predetermined feature amount data extracted from the captured video together with extraction time point data;
Based on the feature value data read from the extracted feature information storage unit and the extraction time point data, a feature expression figure in which the size of the extracted feature amount data is represented as a graphic at the extraction time point on the time axis of the video Generating and displaying the feature representation graphic on a display device;
A video search apparatus comprising: A program executed by a computer constituting the video search device,
The feature data and the extraction time data are read from the extracted feature information storage unit that stores the predetermined feature data extracted from the captured video together with the extraction time data, and extracted based on the feature data and the extraction time data. Generating a feature representation figure in which the size of the feature amount data is represented by a graphic at the time of extraction on the time axis of the video, and causing the computer to execute a process of displaying the feature representation figure on a display device Program.

Images