JP2006080947A - Video searching apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video searching apparatus whereby a user can search an event occurrence scene in a comparatively short time and/or which supports the user in a way of alleviating a load of visual confirmation imposed on the user. <P>SOLUTION: The video searching apparatus 6 for searching the event occurrence scene on the basis of some of event occurrence scenes and video data recording before and after the event occurrence scenes includes: a video storage section 8 for storing the video data; a display section 10 for displaying an image related to a time when the event occurrence scene is to be searched; an entry section 12 whereby the user inputs discrimination information as to whether a first time, displayed on the display apparatus and when the event occurrence scene is to be searched, is before or after the occurrence of the event on the basis of an image related to the first time; and a searching time discrimination section 18 that determines whether a direction wherein the occurrence of the event is to be searched next is toward the future or the past on the basis of the discrimination information entered via the entry section and also determines a second time, when the event occurrence scene is to be searched next, within a searching period between the first time and a third time, when the event occurrence scene has been searched and which is closest to the first time with respect to the determined direction wherein the occurrence of the event is to be searched. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a video search apparatus for searching for an event occurrence scene from video data.

  Currently, when an event occurs (for example, an accident or a disaster), a user confirms the occurrence of the event using video data recorded using a surveillance camera or the like. In particular, in the case where the time of occurrence of an event is not known, there is a need for a device that assists the user so that the user can quickly search in order to reduce the burden of the user searching for an event occurrence scene from a huge amount of video data.

For example, in Patent Document 1, prior to reproduction of a moving image (video data), a change in a certain parameter (color component, luminance component, etc.) between two consecutive image frames of the video data in time series is a standard. An apparatus is disclosed that performs image processing for acquiring a newer image frame as an index image in time series when the difference value is equal to or greater than the difference value, and displays the index image. Therefore, the user can visually find a desired video scene by referring to the plurality of displayed index images.
JP-A-6-38164

  However, in the above-described conventional apparatus, if the parameter selection and the reference difference value cannot be set so as to suit the event, the event occurrence scene cannot be searched because the index image does not include the event occurrence scene. In addition, even if parameters are selected so that an event occurrence scene is included in the index image, it is difficult to acquire only the event occurrence scene as an index image, and many other index images are acquired. Increase the burden of Furthermore, if the search range is long, it takes time for image processing, and the user cannot quickly search for an event occurrence scene.

  Therefore, an object of the present invention is to provide a video search apparatus that supports a user so that the user can search for an event occurrence scene in a relatively short time and / or reduce the burden of visual confirmation by the user.

In order to achieve the above object, one aspect of the video search apparatus according to the present invention is:
In the video search device for searching for the event occurrence scene from the video data recording the event occurrence scene and its surroundings,
A video storage unit for storing video data;
A display unit for displaying an image related to a time to be searched based on video data stored in the video storage unit;
An input unit for causing the user to input determination information on whether the first time is before or after the occurrence of an event from an image related to the first time to be searched displayed on the display unit;
A search in which the next direction to be searched is determined to be the future direction or the past direction based on the determination information input via the input unit, and the second time to be searched next is determined as the first time A search time determination unit for determining within a search section between the searched third time closest to the first time with respect to the direction to be;
It is characterized by providing.
Here, the image related to the time refers to a still image at the time and / or a video (moving image) of the time including the time. In addition, the time to be searched and the time that has already been searched indicate the time at which it is determined whether the event has occurred or after, and the time at which the determination has already been completed.

  According to an aspect of the present invention, the user visually determines whether the first time is before or after the occurrence of an event. If the first time is after the occurrence of an event, the already searched third one that is closest to the first time from one or more already searched times that are chronologically older than the first time. The second time to be searched next is determined from the search section between the first time and the third time. Conversely, if the first time is before the occurrence of the event, the already searched first time that is closest to the first time from one or more already searched times that are newer in time series than the first time. 3 is obtained, and the second time to be searched next is determined from the search section between the first time and the third time. By repeating this, the search section including the event occurrence scene is narrowed down. Therefore, the user can find the event occurrence scene in a relatively short time by observing the video of the search section after the search section is sufficiently narrowed down.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. Below, the example which applied the video search device concerning the present invention to a surveillance system is explained. However, the present invention is not limited to the form in which the event occurrence scene is searched from the monitoring video. For example, the form in which the event occurrence scene (for example, an accident) is searched from the video data captured by the video camera for the purpose other than the monitoring. , A form that improves the work efficiency of editing processing by extracting video data including a specific scene (corresponding to an event occurrence scene) from video content data (for example, unedited video of a movie) (so-called video content production / editing) This can be applied to various fields.

Embodiment 1 FIG.
FIG. 1 shows a monitoring system provided with Embodiment 1 of a video search apparatus according to the present invention. The system 2 includes a video camera 4 disposed at a plurality of points, and a video search device (server) 6 connected to the video camera 4 via a network N (for example, a LAN or the Internet).

  As will be described below, the video search device 6 according to the present embodiment finally narrows down the time interval including the event occurrence scene while allowing the user to determine whether the event has occurred before or after the occurrence of a plurality of times. In order to enable the user to search for an event occurrence scene. As shown in FIG. 2, the video search device 6 includes a video storage unit 8 that stores video data captured by each video camera 4 in association with time, one or more monitors, a VRAM, and the like. The input unit 12 includes a display unit 10 that displays the real-time video of the camera 4 and the stored video stored in the video storage unit 8 and a mouse that moves a cursor on the display screen of the monitor and receives various commands from the user. And controlling the entire video search device 6 to perform a predetermined calculation necessary for searching the event occurrence scene desired by the user (monitoring by the monitoring system 2) from the video data stored in the video storage unit 8. Unit 14. In the following description, it is assumed that an image related to one video camera 4 is displayed on one monitor.

  The monitoring system 2 includes a monitoring mode [see FIG. 3A] for displaying a real-time monitoring image of each video camera 4 on a corresponding monitor of the display unit 10 and a monitor selected by the user via the input unit 12. There is a search mode for displaying the video stored in the video storage unit 8 (in this mode, the monitoring video is still displayed on a monitor that is not selected) [see FIG. 3B]. When the user clicks “scene search” at the lower right of the screen of FIG. 3A in the monitoring mode, the display screen [FIG. 3B] of the search mode is displayed on the display unit 10, and the user in FIG. When “live monitoring” in the lower right of the screen of (b) is clicked, the display unit 10 displays a monitoring mode display screen [FIG. 3A]. The monitor mode display screen [FIG. 3A] also displays a real-time video in the center, and the number of the monitoring camera, the current time, etc. in the lower part. The search mode display screen [FIG. 3 (b)] has a still image at a predetermined time in the center, “after occurrence”, “before occurrence”, “determination impossible”, “video playback” buttons at the bottom, The above time, etc. are displayed on the screen. The predetermined time is a time determined by a search time determination unit described later.

  “After occurrence” and “Before occurrence” are for the user to instruct whether the displayed still image is an image after the occurrence of the event or before the occurrence of the event. “Undeterminable” is for the user to instruct that the corresponding time cannot be determined from the displayed still image before or after the occurrence of the event. When “Video Playback” is clicked, the screen is switched from the screen for displaying a still image to the screen of FIG. 3C so that the video can be played back. In the display screen shown in FIG. 3 (c), a portion for displaying an image at the center, a button for instructing normal playback, fast forward playback, rewind playback, etc., a "search" button, etc. are displayed at the bottom. It has become. When “Search” is clicked, the screen is returned to the display screen of FIG.

  The control unit 14 includes an event pre- and post-setting unit 16, a search time determination unit 18, and a display control unit 20.

  When the user clicks “after occurrence” or “before occurrence” via the input unit 12 while visually checking the still image displayed on the display unit 10 in the search mode, the event pre- and post-setting unit 16 displays the time Is set as the time after the occurrence of the event or the time before the occurrence of the event. In addition, when the user selects the search mode in the middle of the monitoring mode, the pre-event setting unit 16 sets the time at the time of selection as the time after the occurrence of the event.

  If the time set by the event pre- and post-setting unit 16 is the time after the occurrence of the event, the search time determining unit 18 sets the time before the predetermined time from the time after the occurrence of the event and the time set by the event pre- and post-setting unit 16. If the time is before the occurrence of the event, the time after the predetermined time from the time before the occurrence of the event is determined as the time of the still image to be presented to the user (in other words, the time to be searched next). Yes.

  In the present application, the time relationship between two search times (the time to be searched next, the time during the current search, and the two times that have already been searched) to be continuously performed is represented by a direction, and this direction is represented as a search direction. That's it. Specifically, if the specific search time is past the time searched immediately before, the search direction with respect to the time searched immediately before the specific search time is “past direction”, and the specific search time is searched immediately before. If it is in the future with respect to the determined time, the search direction with respect to the time searched immediately before the specific search time is set as the “future direction”. When the search direction is always the past direction with respect to the currently searched time group and the previously searched time group, the direction to the time during the current search of the time to be searched next is the past direction (that is, the current search) When the still image at the middle time is an image after the occurrence of an event), the search time determination unit 18 is prepared as a predetermined time (default value) from the time that was the search target at the present time, but the user can change it appropriately. The previous time is determined as the time to be searched next. When the search direction is always the past direction with respect to the current search time (first time) and the previously searched time group, the next search direction is the future direction (that is, the current search time When the still image is an image before the occurrence of the event), the time when the event occurred is the searched time (the third time) closest to the future direction from the time (first time) that is the current search target ) In the time interval (search interval) between the search time determination unit 18 and the time (first time) at which the next search time (second time) is currently the search target. Between the searched time (third time) closest to the future direction. As an example, the center of the search section (which is prepared as a default value but may be changed by the user as appropriate) is determined as the next search time. When the search direction includes the future direction with respect to the current search time (first time) and the previously searched time group, the next search direction is the past direction (that is, the time of the current search time is stationary) If the image is an image after the occurrence of the event), the time when the event occurred is the time that has been searched for at the present time (first time) and the searched time (third time) closest to the past direction. Since there is a time interval (search interval) between them, the search time determination unit 18 determines the time (second time) to be searched next from the time (first time) as the search target at the present time in the past direction. Between the searched times (third time) closest to.

  The search time determination unit 18 displays a still image at the time (second time) to be searched next on the display unit 10, and then an event occurs at a corresponding time from the screen displayed on the display unit 10 by the user. If it is determined that it is not possible to determine after or before, and “determination impossible” is clicked via the input unit 12, the time searched immediately before (first time) and the searched time (first time) closest to the past or future direction 3), another time within the time interval (search interval) is reset as the time to be searched next. As a result, the event occurrence scene can be searched also from the video data including a part that cannot be determined before and after the event occurrence. However, when the time at which determination is impossible is the first time to be searched in the search mode, the search time is not defined, so the search time determination unit 18 selects another time as appropriate.

  The display control unit 20 is for controlling the display unit 10 to display a moving image (video) and a predetermined still image. Based on the video data stored in the video storage unit 8, the video and the search time are displayed. A still image related to the time determined by the determination unit 18 is displayed on the display unit 10.

  Next, an example of search processing by the video search device 6 will be described with reference to the flowcharts of FIGS. In the following, after the user confirms the event occurrence with the live video obtained by the video camera in the surveillance area based on the report that the event (for example, theft etc.) has occurred in a certain surveillance area, A description will be given on the assumption that the stored image is searched.

First, in step S1, in the monitoring mode, after the user selects a video camera and confirms the occurrence of an event with live video (for example, the loss of an object due to theft), the user displays a display screen [FIG. 3 (a)]. When “scene search” is clicked, the event pre- and post-setting unit 16 sets the time t ₀ at the selected time as the time after the occurrence of the event. In step S2, the index i is set to 1. In step S <b> 3, the search time determination unit 18 determines a time t _i that allows the user to determine whether the event is before or after the occurrence of the event. In step S4, the display control unit 20 displays a still image relating to the time _{t i} to the display unit 10 [FIG. 3 (b)]. In step S5, it is determined whether or not the user has clicked “determination impossible” on the display screen. When the user does not click “determination impossible”, the process proceeds to step S6, and it is determined whether or not the user clicks “video playback” on the display screen.

  If it is determined in step S6 that the user has not clicked “video playback” on the display screen, the process advances to step S7 to determine whether the user clicked “before occurrence” or “after occurrence” on the display screen.

In step S7, if the user clicks on the "front generation" (i.e., determines that the user is an event at time t _i has not yet occurred), in step S8, time series than the time t _i and the time t _i The time in the section (search section) between the newest and the nearest searched time is determined as the next time t _{i + 1} to be searched. After that, after incrementing the index i in step S9, the flow returns to step S4, and an image is displayed at the time t _{i + 1} to be searched next. In step S7, if the user clicks on the "post-generation" (i.e., determines that the user event at time t _i has already occurred), in step S10, time series older than the time t _i and the time t _i The time in the section (search section) between the closest searched times is determined as the time t _{i + 1} to be searched next. Thereafter, the flow proceeds to step S9. Note that the video search device 6 is configured to terminate the search process by instructing via the input unit 12 if the user can visually check the image at time t _i and confirm the event occurrence scene. May be.

By repeating steps S4 to S9, the time interval including the event occurrence scene is successively narrowed down while allowing the user to determine whether or not the event has occurred before or after the occurrence of a plurality of times. An example showing the state of narrowing is shown in FIG. First, after the time _{t 0} after the event occurrence is set [Figure 5 (a)], the time _{t 1} is set to be searched FIG 5 (b)]. In the figure, the section including the event occurrence scene is indicated by hatching. When the user determines that the time t ₁ is the time before the occurrence of the event, the time t ₂ in the search section [t ₀ , t ₁ ] is determined as the time to be searched next [FIG. 5 (c). ]. When the user determines that the time t ₂ is the time after the occurrence of the event, the time t ₃ in the search section [t ₁ , t ₂ ] is determined as the time to be searched next [FIG. 5 (d). ]. When the time t ₃ by the user is determined to be a time prior event occurrence, the search interval [t _{2, t} 3] _(time t ₀ as previously searched-time in the future direction with respect to the time t ₃ and t ₂ there are closest time t ₂ to time t ₃ is time t ₄ in the closest) to the time t ₃ with respect to the future direction is determined as a next time to be searched FIG 5 (e)]. Thereafter, when the user determines that the time t ₄ is the time after the occurrence of the event, the time in the search section [t ₃ , t ₄ ] is determined as the time to be searched next.

In step S5, if the user clicks "indeterminate", the process returns to step S3 as an event at time t _i can not be determined as before or after that occurred, the search time determination unit 18, a different time from the time t _i Is again determined as the time to be searched.

When the user clicks “playback video” in step S6, the process proceeds to step S11, and the display control unit 20 displays the video of the search section between the time t _i to be searched and the searched time closest to the search direction. Is displayed on the display unit 10 [FIG. 3 (c)]. Since the video of this search section includes an event occurrence scene, if the user plays the video after narrowing down the search section, there is no need to search in another section, so the user can easily find the event occurrence scene. And you can search quickly. Until the search section is narrowed down within a predetermined time, playback of the video may be prohibited even if “video playback” is clicked. Alternatively, instead of step S6, video playback may be started automatically when the search interval becomes a predetermined value or less (this is described in detail in the second embodiment). .

In this flowchart, although the time you select a search mode time t ₀ after the event occurs in step S1, the user may select as appropriate. If known further event occurs prior to the time, the user may be allowed to set the time (i.e., at step S3, the time t ₁ user setting.)

  In the following, referring to FIGS. 6 to 8, after the user confirms the abandoned object with the live video obtained by the video camera in the monitoring area based on the report that there is a flaw that seems to be abandoned in a certain monitoring area, A specific example of searching for an image (an event occurrence scene) from a stored image in which a wrinkle has been placed will be described.

  First, when the user selects a corresponding video camera in the monitoring mode and clicks “scene search” displayed on the display screen [FIG. 6A], the monitoring system 2 displays the search mode display screen [FIG. )]. At this time, the event pre- and post-setting unit 16 sets the selected time (18:00) as the time after the event (leaving) occurs. The search time determination unit 18 determines 14:00 four hours before as a time when the user should determine whether the event is before or after the occurrence of the event, and as a result, a 14:00 still image is displayed on the display unit 10. In the upper part of the display screen, the state of narrowing down the search section is displayed with numerical values or figures so that the user can easily search. In the example of the figure, the search target time is shown, and the section including the event occurrence scene is indicated by hatching.

  When the user confirms that there is a defect at 14:00 and clicks “after occurrence”, the search time determination unit 18 further determines 10:00 four hours ago as the next search time, and as a result, 10 A still image of 0:00 is displayed on the display unit 10 (FIG. 6C). When the user confirms that there is no defect at 10:00 and clicks “before”, the search time determination unit 18 is set to determine the center time of the search section as the next search time. Then, 12:00 between the search sections [10:00, 14:00] is determined as the next search time, and as a result, a 12:00 still image is displayed on the display unit 10 [FIG. ]]. When the user confirms that there is no defect at 12:00 and clicks “before”, the search time determination unit 18 sets 13:00 in the section as a search section [12:00, 14:00]. The next search time is determined, and as a result, a 13:00 still image is displayed on the display unit 10 [FIG. 7 (e)].

  In the 13:00 still image, the position where the eyelid is left is shielded by the car, and the user cannot confirm whether the event occurred before or after the event occurrence, so click “cannot determine”. As a result, the search time determination unit 18 again determines another time 12:40 in the search section [12:00, 14:00] as the time to be searched. As a result, a 12:40 still image is displayed on the display unit 10 [FIG. 8 (f)]. When the user confirms that there is a defect at 12:40 and clicks “after generation”, the search time determination unit 18 searches 12:20 in the search section [12:00, 12:40] for the next search. As a result, a 12:20 still image is displayed on the display unit 10. It is narrowed down to a search interval of 5 minutes ([12:00, 12:05 in the example in the figure)] by determining whether the event occurs twice before or after. After that, when the user clicks on “video playback”, the screen shown in FIG. 3C is displayed, and the video in the search section [12:00, 12:05] is played normally, fast forward, and rewind. Observe while confirming that the man left the heel [Fig. 8 (g)].

  In this example, it is narrowed down to a search section of 5 minutes by 8 event pre- and post-event determinations (including the case where determination is impossible except for determination when selecting a search mode), 30 seconds for one determination, and the last neglect Assuming that it took 1 minute to confirm the scene, the search can be performed in 5 minutes in total. The video search device 6 according to the present embodiment can search in a shorter time compared to the case where the event occurrence scene is searched by image processing as in the conventional device described above.

  Note that, for example, 16 thumbnail images are displayed by dividing the video of 10:00 to 18:00 at equal intervals, and two images immediately after the occurrence of the event and immediately before the event are visually checked, and two images are displayed. Considering a method of determining a section between times as a search section and further displaying, for example, 16 thumbnail images and determining a new search section, the search section can be narrowed down to two minutes in two stages. is there. However, there are 16 × 2 = 32 thumbnail images to be displayed, and the number of images to be confirmed by the user increases. On the other hand, the video search device 6 according to the present embodiment has an advantage that the number of images to be visually confirmed is small, and therefore the burden on the user is small.

  In the present embodiment, the search time determination unit 18 determines the time to be searched next, but instead the display unit displays the current image being searched and determines whether the event is occurring before or after the occurrence of the event. 10 displays a search section including an event occurrence scene [section shown in the upper part of the display screen in FIG. 3B], and the user refers to the display screen and determines the next time to be searched via the input unit 12. An instruction may be given. You may make it show a search area to a user using another means (for example, audio | voice) instead of the display part 10. FIG. In this case, instead of the search time determination unit 18, the control unit 14 determines whether the next direction to be searched is the future direction or the past direction, and also relates to the time that is the current search target and the determined direction. A search interval determination unit that determines a search interval between the closest search time and the search time.

  In addition, an area related to an event on the display screen (for example, an area where an abandoned object exists) can be enlarged according to a user instruction at the start of the search or in the middle of the search. For example, brightness correction or integration processing may be performed to make it easier to see an image at a time that allows the user to determine whether the event has occurred before or after.

Furthermore, in the present embodiment, the search section is narrowed down based on the time t ₀ after the occurrence of the event, but instead, the time before the occurrence of the event may be used as a reference. In this case, the time determination by the search time determination unit 18 is as follows. When the search direction is always the future direction with respect to the current search time and the previously searched time group, the direction to the current search time of the time to be searched next is the future direction (that is, the current search) When the still image at the middle time is an image before the occurrence of the event), the search time determination unit 18 determines the time after the predetermined time as the time to be searched next. If the search direction is always the future direction with respect to the current search time (first time) and the previously searched time group, the next search direction is the past direction (that is, the current search time When the still image is an image after the occurrence of the event), the time when the event occurred is the searched time (third time) closest to the past direction from the time (first time) that is the current search target ) In the time interval (search interval) between the search time determination unit 18 and the time (first time) at which the next search time (second time) is currently the search target. To the past searched time (third time) closest to the past direction.

Embodiment 2. FIG.
FIG. 9 shows Embodiment 2 of the video storage apparatus according to the present invention. In the following description, constituent elements that are the same as or similar to those in the first embodiment are represented by the same reference numerals or the same reference numerals with appropriate subscripts. The video storage device 6A according to the present embodiment is similar to the video storage device 6 according to the first embodiment. However, the control unit 14A determines whether the still image at the target time is the occurrence of an event based on a user instruction. Instead of the event pre- and post-setting unit 16 for setting whether it relates to the after, an event pre- and post-determination unit 22 that automatically determines whether the still image is related to before or after the occurrence of the event by image processing is provided.

  Specifically, the event pre- and post-determination unit 22 determines predetermined parameters (for example, a color component, a luminance component, etc.) between an image frame after the occurrence of an event and an image frame (still image) at a time to be searched determined by the search time determination unit 18. ) Is obtained, and based on the difference, it is determined whether the time to be searched is before or after the occurrence of the event. As the image frame after the occurrence of the event, for example, a frame related to the time when the user of the monitoring system (see FIG. 1) switches from the monitoring mode to the search mode is used. However, for example, when the abandoned object has already been removed from the scene and the feature indicating the event occurrence is not included in the video at the time when the user confirms in the monitoring mode, another time is used as the image frame after the event occurrence. You may use about. In order to easily and quickly determine before and after the occurrence of the event by the event pre- and post-determination unit 22, the user sets the event type (for example, incidents such as neglect, theft, disasters such as landslides) (via the input unit 12). Parameters that are suitable for the event to be searched, such as the user setting a partial area of the image frame (for example, an area where an abandoned object is placed in the image frame) and obtaining a difference only for the area. It is preferable to allow the user to set (type of event, region in image frame, etc.).

  For example, when the abandoned object is shielded at the search time, the event pre- and post-determination unit 22 may not be able to determine. In this case, as in the first embodiment, the search time determination unit 18 determines a time interval (search interval) between the time when the other time was searched immediately before and the searched time closest to the past direction or the future direction. Another time is reset as the time to be searched. As a result, the event occurrence scene can be searched also from the video data including a part that cannot be determined before and after the event occurrence. However, when the current time that cannot be determined is the first time to be searched in the search mode, the search section is not defined, so the search time determination unit 18 selects another time as appropriate.

  The control unit 14A includes a search section narrowing determination unit 24 that determines whether or not the search section is sufficiently narrowed by comparing the search section with a predetermined value (for example, 5 minutes). The search section narrowing determination unit 24 is configured to send a predetermined signal to the display control unit 20 when the search section is sufficiently narrowed down.

  In the present embodiment, it is not necessary to indicate that the time to be searched for by the user is before the event occurrence, after the event occurrence, or cannot be determined at the time, and accordingly, the display unit 10 displays, for example, FIG. The display control unit 20 receives a signal from the search segment narrowing determination unit 24 when the search segment is sufficiently narrowed down, and displays a screen as shown in FIG. This allows the user to play the video. Alternatively, the display control unit 20 may automatically start video playback after sufficiently narrowing down the search section.

  Next, an example of search processing by the video search device 6A will be described with reference to the flowcharts of FIGS.

First, in step S1001, the user specifies parameters (for example, the type of event and the area in the image frame) suitable for the event to be searched via the input unit 12. In step S <b> 1002, the user sets time t ₀ after the occurrence of the event via the input unit 12. For example, the time when the user clicks “scene search” on the display screen [FIG. 3A] in the monitoring mode is set as the time t ₀ after the occurrence of the event. In step S1003, the index i is set to 1. At this time, the pre-event determination unit 22 takes in the image frame at time t ₀ from the video storage unit 8. In step S1004, the search time determination unit 18 determines a time t _i to search for before or after the occurrence of an event.

Next, the event longitudinal determination section 22 takes in the image frame at time t _i from the video storage unit 8, the time before or after an event occurs on the basis of the image frame set parameters and time t _i in step S1001 Is determined (steps S1005 and S1006). If you can not determine (NO in step S1005), the flow returns to step S1004, the search time determination unit 18, between the nearest searched-time in the past direction or future direction of time and the time t _i of searching just before Another time within the time interval (search interval) is reset as the time to be searched. When the time when determination is impossible is the time to be searched first in the search mode, the search section is not defined, and the search time determination unit 18 resets another time as the time to be searched.

If it is determined that the time t _i is before the occurrence of the event (YES in step S1006), the process proceeds to step S1007, and a search interval between the time t _i and the nearest searched time that is newer in time series than the time t _i Is determined as time t _{i + 1} to be searched next. Thereafter, the flow proceeds to step S1008. In step S1006, if the time t _i is determined after event occurrence, the flow proceeds to step S1009, in the search section between the chronologically older closest already searched time than the time t _i and the time t _i The time is determined as the time t _{i + 1} to be searched next. Thereafter, the flow proceeds to step S1008.

In step S1008, the search section narrowing determination unit 24 determines whether the search section [t _i , t _{i + 1} ] is equal to or smaller than a predetermined value, in other words, whether the search section is sufficiently narrowed. If not sufficiently narrowed down, index i is incremented in step S1010, and the flow returns to step S1005. Then, the determination by the event pre- and post-determination unit 22 is performed for the next time t _{i + 1} to be searched. If it has been narrowed down sufficiently, the process proceeds to step S1011 and the display control unit 20 receives a signal from the search section narrowing determination unit 24 and causes the display unit 10 to display a screen as shown in FIG. Here, if the user clicks “video playback”, the process proceeds to step S 1012, and the display control unit 20 causes the display unit 10 to display the video of the search section [t _i , t _{i + 1} ]. Instead of steps S1011 and S1012, the display control unit 20 may automatically start playing the video if the search interval is equal to or less than a predetermined value.

  According to the present embodiment, the event pre- and post-determination unit 22 automatically makes a determination before and after the occurrence of an event by image processing, so the burden of visual confirmation by the user can be reduced.

  In this embodiment, the image frame related to the event occurrence is used as the image frame to be compared with the image frame at the search time determined by the event pre- and post-determination unit 22, but instead, for example, an object related to the event such as a stolen object is the event. The pre-event image frame may be used (if it is in the pre-occurrence image frame).

  In addition, after the event pre- and post-determination unit 22 determines whether each search time is before or after the occurrence of the event, the determination result and an image at the time are displayed on the display unit 10, so that the user confirms the determination by image processing as needed. You may be able to do it.

  Furthermore, in the present embodiment, the search time determination unit 18 determines the time to be searched next, but instead, the search section including the event occurrence scene is displayed on the display unit 10 [see FIG. 3B] or voice or the like. It is made to present to a user using other means, and after determining whether the search time is before or after the occurrence of the event by the event pre- and post-determination unit 22, the next search is made to the user by referring to the presented search section. The time to be determined may be determined via the input unit 12.

Embodiment 3 FIG.
FIG. 11 shows Embodiment 3 of the video search apparatus according to the present invention. In addition to the configuration of the second embodiment, the control unit 14B of the video search device 6B detects an event occurrence that detects an image frame related to a scene in which an event has occurred using image processing when the search section is sufficiently narrowed down Part 26 is provided. For example, the event occurrence detection unit 26 performs image processing on a predetermined parameter (for example, a color component, a luminance component, etc.) based on a difference between two image frames arranged in time series. The event occurrence detection unit 26 receives a predetermined signal from the search section narrowing determination unit 24 and starts detection processing.

  When the event occurrence detection unit 26 detects an image frame related to the event occurrence scene, the display control unit 20 receives a predetermined signal from the event occurrence detection unit 26 and displays the image frame and “video reproduction” on the display unit 10. A screen similar to that shown in FIG. 3C including buttons is displayed.

FIG. 12 is a flowchart showing an example of video search of the video search device 6B. Steps S1201 to S1211 are the same as steps S1001 to S1011 in the flowchart of FIG. However, in step S1201, a parameter related to the image processing by the event occurrence detection unit 26 is specified as a parameter suitable for the event to be searched (for example, an image frame interval for inter-frame difference processing). If it is determined in step S1208 that the search section narrowing determination unit 24 determines that the search section [t _i , t _{i + 1} ] is equal to or smaller than a predetermined value (that is, the search section is sufficiently narrowed down), the process proceeds to step S1211 and the event occurrence detection unit 26 Receives a signal from the search section narrowing determination unit 24, performs image processing based on video data in the search section of the video storage unit 8, and detects an image frame related to the event occurrence scene. Thereafter, the display control unit 20 receives a signal from the event occurrence detection unit 26 and causes the display unit 10 to display the image frame and “video reproduction”. The user can observe the video including the event occurrence scene (the image frame related to the event occurrence scene and the image frames before and after the event occurrence scene) by clicking “video playback” displayed on the display screen (steps S1212 and S1213).

  According to the present embodiment, since the event occurrence detection unit 26 detects the event occurrence scene by performing image processing after the search section is narrowed down, the burden of visual confirmation by the user is reduced, and the device The amount of computation on the 6B side can be reduced and event occurrence scenes can be searched at high speed.

  In step S1212, the video is played when “video playback” is clicked. However, if there is an error in detection by the event occurrence detection unit 26, the image processing is further performed based on a user instruction. May be. For example, if the image processing is started from the time chronologically old that defines the search section, and an event frame does not appear in the image frame even if an image frame at a certain time is acquired, the section after the above time Image processing is performed on

  In the first embodiment, an event occurrence scene may be similarly detected by image processing even when the search interval is equal to or less than a predetermined value, and this configuration is also included in the scope of the present invention.

  Although specific embodiments according to the present invention have been described above, the present invention is not limited to these and can be variously modified. For example, in the above embodiment, the still image at the time is used to determine whether the time is before or after the occurrence of the event (automatic determination by user determination or image processing). For example, a moving image of several seconds) may be used to determine whether the time is before or after the occurrence of the event (user determination or automatic determination by image processing). Therefore, in the first embodiment, a moving image (video) including a time including the time is displayed on the display unit 10 instead of the still image at the time to be searched. In the second embodiment, the event pre- and post-determination unit 22 is configured to capture video data of a time including a time to be searched from the video storage unit 8 during image processing.

  In the first embodiment, the video data of the video camera 4 is transferred to the video storage unit 8 of the video search device 6 online via the network N. However, the scope of the present invention includes a video storage unit as a video storage unit. A configuration using a portable recording medium (for example, a CD-RW, a magneto-optical disk, etc.) on which video data is recorded is also included.

  Furthermore, the video data stored in the video storage unit 8 is associated with the time. For example, in the case of video content production / editing, this time does not need to be a real time, but may be a virtual time.

1 is an overall configuration diagram showing a monitoring system provided with a first embodiment of a video search apparatus according to the present invention. FIG. 2 is a block diagram of the video search device of FIG. (A) The figure which shows the display screen of monitoring mode. (B) The figure which shows the display screen of search mode. (C) The figure which shows the display screen after clicking "video reproduction" of Drawing 3 (b) in search mode. The flowchart which shows an example of the search process of the video search device of FIG. The figure for demonstrating the narrowing-down process of a search area. (A) is a figure which shows the display screen of monitoring mode, (b), (c) is a figure which shows the display screen of search mode in the middle of a search. (D), (e) is a figure which shows the display screen of the search mode in the middle of a search. (F) The figure which shows the display screen of the search mode in the middle of a search. (G) The figure which shows the display screen which caught the event occurrence scene, after switching to the display screen of FIG.3 (c) by clicking "video reproduction | regeneration" of FIG.8 (f). The block diagram which shows Embodiment 2 of the image | video search device which concerns on this invention. 10 is a flowchart illustrating an example of a search process of the video search device in FIG. 9. The block diagram which shows Embodiment 3 of the video search apparatus which concerns on this invention. 12 is a flowchart illustrating an example of a search process of the video search device in FIG. 11.

Explanation of symbols

6 Video search device

Claims (7)

In the video search device for searching for the event occurrence scene from the video data recording the event occurrence scene and its surroundings,
A video storage unit for storing video data;
A display unit for displaying an image related to a time to be searched based on video data stored in the video storage unit;
An input unit for causing the user to input determination information on whether the first time is before or after the occurrence of an event from an image related to the first time to be searched displayed on the display unit;
A search in which the next direction to be searched is determined to be the future direction or the past direction based on the determination information input via the input unit, and the second time to be searched next is determined as the first time A search time determination unit for determining within a search section between the searched third time closest to the first time with respect to the direction to be;
Video search device with After the display unit displays an image related to the second time on the display unit, when it is not possible to determine whether the event is before or after the occurrence of the event, the determination impossible information is input by the user. And
The search time determination unit determines a time different from the second time within the search section as a time to be searched next.
The video search apparatus according to claim 1, wherein: In the video search device for searching for the event occurrence scene from the video data recording the event occurrence scene and its surroundings,
A video storage unit for storing video data;
A display unit for displaying an image related to a time to be searched based on video data stored in the video storage unit;
An input unit for causing the user to input determination information on whether the first time is before or after the occurrence of an event from an image related to the first time to be searched displayed on the display unit;
Based on the determination information input via the input unit, the next direction to be searched is determined as the future direction or the past direction, and the first time and the determined direction to be searched are closest to the first time. A search interval determination unit that determines a search interval between the searched third time and
With
The display unit displays the search section, or the device further includes a presentation unit for presenting the search section,
The input unit causes the user to input the time within the search section displayed on the display unit or presented on the presentation unit as a time to be searched next by the user.
A video search apparatus characterized by that. In the video search device for searching for the event occurrence scene from the video data recording the event occurrence scene and its surroundings,
A video storage unit for storing video data;
An event pre- and post-event determination unit that determines whether the first time to be searched is before or after the occurrence of an event by image processing based on the video data stored in the video storage unit;
Based on the determination information by the event pre- and post-determination unit, the next direction to be searched is determined as the future direction or the past direction, and the second time to be searched next is determined as the first time as the direction to be searched A search time determination unit for determining within a search interval between the searched third time closest to the first time with respect to
Video search device with When the search time determination unit cannot determine whether the second time is before or after the occurrence of the event, the search time determination unit should next search for a time different from the second time within the search section. As determined,
The video search apparatus according to claim 4, wherein: In the video search device for searching for the event occurrence scene from the video data recording the event occurrence scene and its surroundings,
A video storage unit for storing video data;
An event pre- and post-event determination unit that determines whether the first time to be searched is before or after the occurrence of an event by image processing based on the video data stored in the video storage unit;
Based on the determination information by the event pre- and post-determination unit, the next direction to be searched is determined as the future direction or the past direction, and the first time and the determined direction to be searched for have been searched closest to the first time. A search interval determination unit that determines a interval between the third time and the search interval;
A presentation unit for presenting a search section;
An input unit for inputting a time within the search section presented in the presentation as a time to be searched next by the user;
Video search device with The event occurrence detection unit according to any one of claims 1 to 6, further comprising an event occurrence detection unit that detects an event occurrence scene by image processing based on video data in the search interval when the search interval is equal to or less than a predetermined value. Video search device.

Images