JP2006050404A - Recorded video editing method and display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video editing method and a video display method that enable the organization of videos such that a monitor (or operator) can temporally sequentially identify and display the image of a target object from videos taken by a plurality of image pick-up devices. <P>SOLUTION: Channel association information including an estimated position where a surveillance camera installed is registered. When an accident or an event happens, characteristic colors are inputted that are desired to be found in the recorded videos with the colors searched and pursued. A target object is selected from the search result, and a pursuit video is created. Thus, editing is performed such that colors can be searched based on the video channel association, and thereby the pursuit video can be identified and displayed temporally sequentially. The pursuit video can be handled in the same way as other ordinarily recorded videos and pursuit path information can be outputted. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method of recording video obtained by a plurality of imaging devices and editing and displaying the recorded video.

  A conventional video recording, editing, and display method will be described with reference to FIGS. FIG. 2 and FIG. 6 are diagrams for explaining a system that captures an image within a predetermined visual field range by a plurality of imaging devices and records and displays the obtained video.

  FIG. 2 is a plan view showing the relationship between the position of an imaging apparatus installed for imaging a predetermined visual field range and a tracking target moving body that moves within the predetermined visual field range. An imaging device installed to obtain an image of each predetermined partial field of view within the range, 207 is a moving body to be tracked, 207-0 is a position where the moving body 207 was present at time t0, 207- 1 is the position at which mobile 207 was at time t1, 207-2 was the position at mobile 207 at time t2, and 207-3 was the position at which mobile 207 was at time t3 207-4 is the position where the mobile unit 207 existed at time t4, 20-1 is an arrow indicating the path the mobile unit 207 has moved from time t0 to time t1, and 20-2 is the mobile unit 207. Is an arrow indicating the path traveled from time t1 to time t2, and 20-3 is an arrow indicating the path traveled by mobile unit 207 from time t2 to time t3. Mark, 20-4 is an arrow indicating the path traveled by mobile unit 207 from time t3 to time t4, and 21-1 to 21-9 are blocked by walls or walls and cannot move the mobile unit (non-movable area) , 22-1 to 22-6 and 23-1 to 23-6 are passages (movable areas).

The imaging devices 201 to 206 are, for example, television cameras. One camera number is associated with each imaging device. For example, camera number 1 (CH1) is added to the imaging device 201. Hereinafter, camera numbers 2 (CH2) are assigned to the imaging devices 202 to 206, respectively. ) To 6 (CH6) are added.
The camera number information is added to the captured video, for example.

With reference to FIG. 2, a system for tracking a moving body moving within a predetermined visual field range will be described.
The images captured by the imaging devices 201 to 206 are not shown in FIG. 2, but they are separated from the shooting location (field of view range), for example, a monitor by a transmission method comprising a wired method, a wireless method, or a combination thereof. And a monitoring room with a recording device. A monitor in the monitoring room can view images in the past or present field of view.

  FIG. 6 is a diagram schematically illustrating a conventional display state of recorded video. The video from each imaging device is recorded in the recording device with a time stamp and a camera number added, for example, as a video of each imaging device every 200 msec. The horizontal axis of FIG. 6 is an example in which time is displayed on the horizontal axis, and each row is an image in which images recorded for each imaging device (CH1, CH2, CH3,...) Are arranged in time order and displayed on the monitor. 101 is a video sequence acquired by the CH1 imaging device 201, 102 is a video sequence acquired by the CH2 imaging device 202, and 103 is a video sequence acquired by the CH3 imaging device 203. 101-1 to 101-12 are videos recorded in the time sequence of the video sequence 101, 102-1 to 102-12 are video sequences recorded in the time sequence of the video sequence 102, and 103-1 to 103-12 are video sequences of the video sequence 103. This is an image recorded in order of time.

  In FIG. 6, when the images 101-1 to 101-4 are viewed, it can be seen that the moving object 207 captured by the image capturing apparatus 201 is gradually approaching the image capturing apparatus 201. When the images 102-4 to 102-8 are viewed, it can be seen that the imaging device 202 is imaging the moving body 207 and is gradually approaching the imaging device 202 as well. Similarly, when the images 103-7 to 103-11 are viewed, it can be seen that the imaging device 203 is imaging the moving body 207, and this time gradually moves away from the imaging device 203.

This can be clearly seen by looking at FIG. 6 in comparison with FIG. That is, the moving body 207 moves from the left to the right in the path 22-1 captured by the imaging device 201, moves out of the field of view of the imaging device 201, and then the path that is the field of view of the imaging device 202. 22-2, moving from left to right again, then out of the field of view of the imaging device 202 and entering the path 23-5 of the field of view of the imaging device 203, and gradually passing the path 23-5 from top to bottom It turns out that it turns to the path | route 22-5, and remove | deviates from the visual field of the imaging device 203 by the image | video 103-12. At this next time point, it is not known whether the moving body 207 is bent into the passage 22-5 or the passage 22-6, but whether or not the moving body 207 is reflected in the video sequence captured by the imaging device 204. Can be judged. Note that the vertical and horizontal directions in FIG. 2 are directions that can be explained by the directions on the paper of FIG.
In addition, although the imaging device is installed only in the direction in which the moving body 207 travels, only the imaging device that is necessary for explanation is displayed in the figure, and is practically installed in another place.

As shown in FIG. 6, the video recorded in the recording device is displayed in chronological order for each imaging device, and can be confirmed by the system administrator or the user.
Conventionally, in order to display and confirm the recorded video for each imaging device, after displaying the video sequence 101, the object (moving body 207) as the target (tracking target) is confirmed, and then the video sequence 102 and the video sequence are displayed. In 103, the same time zone was displayed and confirmed.

Surveillance cameras for the purpose of crime prevention etc. are installed in places with heavy traffic and traffic, and are not constantly monitored. Instead, it is always recorded on a recording device such as a video tape recorder or digital recorder at regular intervals, and is verified when an accident or incident occurs.
Even if you know the date and time when an accident or incident occurred, it is not easy to find the target video from multiple surveillance camera videos, and each camera video is input to a separate channel of the recorder. As a result, it was impossible to continuously check the video.
That is, in the above-described conventional technology, when an object (moving body) to be tracked is selected from a recorded video sequence, a video of the target object is displayed and confirmed from a recorded video of another related imaging device. There was a need to do.
An object of the present invention is to eliminate the above-described drawbacks, and when a supervisor (or operator) selects a target object from an image captured by one imaging device, an image captured by another imaging device It is an object of the present invention to provide a recording video editing method and a display method that enable editing of video so that related video can be confirmed and displayed in time series.

In order to achieve the above object, a recorded video editing method and display method according to the present invention records videos taken by a plurality of imaging devices, and searches for specific objects from the recorded videos (for example, color search). Thus, a video (tracking video) in which videos tracking a specific object are arranged in time series is created and displayed.
For this reason, the present invention registers in advance time-series association information of video sequences for each imaging device assuming the installation position of the monitoring camera and the like. Then, the characteristic color of the target object is input into the recorded video, and color tracking search is performed. Select the target object from the search results. Search the characteristics of the selected object from time series related information, collect the video of the channel in which the target object is reflected in the tracking video channel, edit and display it so that it can be handled in the same way as normal recorded video, The tracking route information is output.

That is, the recorded video editing and display method of the present invention includes a plurality of imaging devices that capture images within a predetermined field of view, a recording device that records images captured by the imaging device, and a recording device. A monitoring system including a display device for displaying a video, searching for a video recorded in a recording device, detecting a video in which a reference video of a target object is present, Based on the registered association information, time-series association information is created.
In addition, the recorded video editing and displaying method of the present invention preferably displays a video in which a reference video exists based on related information.

By implementing the present invention, the system administrator or the user can reduce the work of searching for a video of a specific purpose from recorded video over a plurality of channels and confirming the content of the searched video. In addition, it is not necessary to copy the videos in order to collect the videos of the plurality of imaging devices.
For example, in the event of an accident or incident, you can quickly search for suspects and victims from the recorded video, which is useful for tracking suspects and victims, and outputs tracking data it can. Further, by accumulating the tracking data, route information can be accumulated.

The recorded video editing method and display method of the present invention are implemented, for example, in the following steps 1 to 3.
Step 1 “Registration of association information”: The system administrator or the user takes an image taking into consideration the installation position (distance between installation positions) of the imaging device, the height of the stairs, the width of the passage, the traffic volume, etc. Time-series association information (time-series association information) between devices (between video channels) is registered in advance in a recording device or the like.
Step 2 “Search”: Information on a reference video (one recorded video or information on an externally input video) is designated in advance or each time by a system administrator or a user. The specified video information is input to a recording device or the like in advance and selected by a system administrator or a user, for example. The recording device searches the recorded video and selects a video in which information of the selected reference video exists as a video in which a target object exists.
As a search method, for example, there is a method of inputting a feature color of an object for search and performing a color tracking search. Such a search method is described in, for example, Japanese Patent Application Laid-Open Nos. 6-223179 and 2000-253309.
Step 3 “Determination of tracking path”: From the time-series related information of the selected object, the video of the channel in which the target object is shown is gathered into the tracking video channel and associated so that it can be handled in the same way as a normal recorded video. It also outputs tracking route information.

An example of the operation of steps 1 to 3 will be further described with reference to FIGS.
FIG. 2 is a diagram for explaining a configuration example of a system that is also used in the description of the prior art and that captures images within a predetermined visual field range by a plurality of imaging devices and displays or records the obtained video. FIG. 3 shows an example of the simplest system configuration in the case where a video image taken by the imaging devices 201 to 206 shown in FIG. 2 is viewed by a supervisor in a monitoring room equipped with a monitor and a recording device, for example. FIG. 201 to 206 are imaging devices, 301 is a recording device for recording images of the imaging devices 201 to 206 of a plurality of channels, and 302 is a monitor for displaying images.
The imaging devices 201 to 206 and the recording device 301, and the recording device 301 and the monitor 302 are coupled by a transmission cable.

In FIG. 3, a recording apparatus 301 has functions such as an operation function, a frame switch function, a split display function, a recorded video search function, and a video output function.
In addition, the functions of the present invention may be realized by realizing each of these functions with different devices, and a collection of these different devices exchanging information with each other or accessing each other. Further, there may be a plurality of recording devices. Further, the recording device 301 can also output the video to the monitor at the same time while recording the video.
In addition, although the imaging devices 201 to 206 and the recording device 301 are coupled by a transmission cable, they may be coupled via a network or need not be wired.

  FIG. 4 is a diagram for explaining a method of creating information association for each channel (or each imaging device), that is, time-series related information (correlation information) in the video editing method and the video display method of the present invention. FIG. 401-1 is the video sequence captured by the imaging device 201 with the camera number (channel number) CH1, 401-2 is the video sequence captured by the imaging device 202 with the camera number CH2, and 401-3 is the imaging sequence 203 with the camera number CH3. The captured video sequence, 401-4 is the video sequence captured by the imaging device 204 with the camera number CH4, 401-5 is the video sequence captured by the imaging device 205 with the camera number CH5, and 401-6 is the imaging device 206 with the camera number CH6. Is a captured video sequence, and 402 is an arrow indicating that there is a correlation when there is a correlation between the respective video sequences 401-1 to 401-6.

  In FIG. 4, the direction of the arrow 402 indicates that the target object moves to the video sequence in the direction of the arrow in time series. For example, when the flow that normally moves is determined, if the arrow direction is set to one direction, it can be referred to as information that has moved in an abnormal direction.

  It is also possible to weight the correlation with time. For example, video sequence 401-2 and video sequence 401-3 are far away from each other (or moving distance), so it takes time to move, and video sequence 401-3 and video sequence 401-4 are close (or moved) Weighting (prioritization) is performed when it takes less time to move.

  For example, when it is searched that a moving object to be tracked is in the video sequence 401-3 at a time t and is in the video sequence 401-2 and the video sequence 401-4 at the next time t + 1, the video sequence 401- 4 is prioritized over video sequence 401-2. The closer distance is prioritized, and not only the distance but also the slope, stairs, escalator, elevator, pedestrian crossing (no traffic signal), pedestrian crossing (traffic signal) Priority is given by equating).

FIG. 5 is a diagram for explaining a frame selection situation when information is associated with each channel (or each imaging device) in the video editing method and the video display method of the present invention. Reference numeral 501 is a video sequence schematically showing the images captured by the imaging device 201 recorded in the recording device 302 in time order, and 502 is a schematic sequence of images captured by the imaging device 202 recorded in the recording device 302 in time order. 503 is a video sequence schematically shown by arranging the images captured by the imaging device 203 recorded in the recording device 302 in time order (the video sequences of the other imaging devices 204 to 206 are saved. ). Reference numeral 504 denotes a video sequence schematically showing videos in which search results are arranged.
The numbers enclosed in a frame indicate the camera number of the imaging device in which the number before the hyphen is imaged, and the continuation number assigned in order of the time at which the number after the hyphen is captured. For example, 1-1 indicates that the imaging device 201 with camera number 1 is the first image captured, 2-5 indicates that the imaging device 202 with camera number 2 is the fifth image, and 3 -11 indicates that this is the 11th image captured by the imaging device 203 with the camera number 3.

FIG. 5 shows an example of a color search performed in time series.
Now, the color search of the moving object to be tracked is performed in time series for the images 1-1 to 1-11, 2-1 to 2-11, and 3-1 to 3-11 in the image sequences 501 to 503. Execute. As a result, the video portion indicated by the double frame in the video 1-1 to 1-11, 2-1 to 2-11, 3-1 to 3-11 in the video sequence 501 to 503 (1-1, 1-2, 1-3, 1-4, 1-8, 1-10, 2-2, 2-4, 2-5, 2-6, 2-7, 2-8, 3-5, 3- 8, 3-9, 3-10, 3-11), video portions (frames) that can be estimated as moving objects to be tracked are considered as tracking candidates.

Next, when there are overlapping candidates at the same time from the correlation information (time-series related information) in FIG. 4, the video portion to be tracked is selected.
For example, for video part 2-1 and video part 2-2, video part 2-2 is excluded from the tracking candidates. Also, for example, in video part 1-4 and video part 2-4, video part 2-4 is excluded from the tracking candidates, and in video part 2-5 and video part 3-5, video part 3-5 is the tracking candidate. Excluded from. Similarly, for example, in video part 1-8, video part 2-8, and video part 3-8, video part 1-8 and video part 3-8 are excluded from the tracking candidates, and video part 1-10 and video part 3-8 are excluded. In part 3-10, video part 1-10 is excluded from the tracking candidates.

  In the above embodiment, priority is given to the same video sequence (camera number (CH)) as correlation information, and other channels are excluded from the tracking candidates. In addition, in the video part 1-10 and the video part 3-10, the video part 1-10 is separated from the video string 503 by the correlation information in FIG. Parts 1-10 are excluded from the tracking candidates.

As a result of this selection, a video frame is completed as shown in the video sequence 504 in FIG. 5, and the system administrator or the user can confirm it by displaying this video frame.
If there is a duplicate candidate at the same time, it remains as related information unless it is significantly deviated from the correlation. This duplication candidate can be arbitrarily deleted or replaced. The replacement is to replace the video part automatically selected by the search execution with the video part left as the related video part. For example, in FIG. 5, the video part 2-8 is replaced with the video part 3-8, and the video part 2-8 is replaced with the video part 3-8. Replace parts 2-7 with video part 3-8 and video part 3-8 with video part 3-9.

In addition, when the video part 3-9 and the video part 3-11 are separated in time, the video part 1-10 is left as a tracking candidate. For example, this is a case where no tracking candidate is found in the video sequence 502 (CH2).
Also, if there is no candidate in the video part at the same time as the search result, the video part of the video sequence with the candidate immediately before that is filled in the video frame, black display, blue display, etc. Make it clear that

FIG. 1 shows an example in which color search is executed in time series as in FIG. Video sequences 101 to 103 in FIG. 1 are the same as the video sequences 101 to 103 in FIG. Reference numeral 104 denotes a video sequence derived from the search result (time-series related information).
As shown in FIG. 1, by displaying the video sequence 104 in chronological order, it can be seen how the object to be tracked moves. This moved information can be output to another external device. The external device is, for example, an external recording device, a printer, a facsimile coupled with a telephone line, a personal computer, a video distribution device coupled with a network device, a server, or the like.
In the above embodiment, the number of video sequences is three. However, the number of image pickup devices installed may be any number as necessary.

In the color search for searching for tracking candidates, it is necessary to determine a color range in advance for each specified color. As the designated color, a plurality of colors can be designated and can be selected from the reference frame.
For example, if a red school bag elementary school student wearing a yellow school hat is a tracking candidate, yellow and red are designated colors, and the hat range is specified from the actual video frame, the width of the color component included in that range is automatically set. Recorded. The width of this color component (for example, the number of pixels) is automatically recognized as a candidate having a certain size and is set as a candidate. That is, even if only one pixel is a candidate color component, it does not react.
These functions may be incorporated in the recording device 301 of FIG. 3 or may be configured separately from the recording device 301 as a separate device (search device).

The figure which shows the video recording result of one Example of this invention. The top view which shows the relationship between the position of the imaging device installed in order to image the conventional predetermined visual field range, and the tracking target moving body which moves within the predetermined visual field range. The figure which shows the system configuration | structure of one Example of this invention. The figure for demonstrating the correlation information of one Example of this invention. The figure for demonstrating one Example of the frame selection condition at the time of the correlation of the information of this invention. The figure which showed typically the mode of the conventional display of the recorded image | video.

Explanation of symbols

20-1 to 20-4: arrows indicating the route traveled, 21-1 to 21-9: space, 22-1 to 22-6, 23-1 to 23-6: passage, 101 to 104: video sequence, 201 to 206: imaging device, 207: moving body, 207-0 to 207-4: position of moving body, 301: recording device, 302: monitor, 401-1 to 401-6: video sequence, 402: correlation Arrow indicating that there is, 501 to 504: video sequence.

Claims (2)

A monitoring system comprising a plurality of imaging devices that capture images within a predetermined visual field range, a recording device that records images captured by the imaging device, and a display device that displays the images recorded in the recording device Because
Search the video recorded in the recording device,
Detect the above video where the reference video of the target object exists,
A method for editing and displaying a recorded video, wherein time-series related information is created based on association information registered in advance for the detected video. The recorded video editing method and display method according to claim 1,
A method for editing and displaying a recorded video, wherein the video having the reference video is displayed based on the related information.