JP2006197485A - Moving picture data management method and device, and moving picture display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving picture data management method capable of processing additional information as editable additional information while guaranteeing the synchronization of image data with the additional information. <P>SOLUTION: In the moving picture data management method used in display processing by the moving picture display device through the moving picture data management device for managing image data corresponding to video photographed by an image pickup device and vector data obtained by performing moving body extraction processing of the video, in a connection step to be executed, the vector data representing a figure indicating the position and size of the moving object obtained corresponding to the image data is connected to the end of image data of each frame to be transmitted to the moving picture display device. In the moving picture display device, each time the image data of each frame is received from the moving picture data management device, display processing based on the image data of the frame is performed. Next, in a overwriting step, line drawing representing a figure generated on the basis of the vector data is overwritten on an image reproduced on the basis the image data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

The present invention relates to a moving image data management method, a moving image data management device, and a moving image display device in a monitoring system for monitoring facilities based on moving image data composed of image data taken by a monitoring camera (ITV / CCTV or the like). .
In surveillance systems that have a very wide surveillance area, such as airports, harbors, or large-scale plant surveillance systems, video data composed of image data taken by a surveillance camera that captures a wide area is intruders. It is used for monitoring.

FIG. 8 shows a configuration example of a conventional general monitoring system.
Image data photographed by the monitoring cameras 401 _{1 to} 401 _N shown in FIG. 8 are monitored in correspondence with each other via a video switching / distribution device (matrix video switcher, indicated by reference numeral VSW in FIG. 8). It is transferred to and displayed on display devices 402 _{1 to} 402 _N provided in a room, etc., and is used for monitoring work by a supervisor, and is recorded by video tape recorders 403 _{1 to} 403 _N and stored for a certain period of time. The

In such a conventional monitoring system using a moving image, a monitoring person monitors the moving image to perform operations such as finding an intruder or a suspicious object.
On the other hand, in the field of traffic monitoring on highways, moving object extraction technology is used to detect vehicles traveling on the road.
This technology extracts a moving object within the field of view of an image pickup means by comparing a plurality of images obtained by continuously shooting the same spot and extracting a changing portion. . In a general moving object extraction technique, an area of an image in which a moving object is detected is indicated by coordinate data indicating a position in the image and vector data indicating a range.

If such a moving body extraction technique is applied to the above-described monitoring system using a moving image, it becomes possible to automate the detection of an intruder or the like in the monitoring range.
Moreover, if the detection result is superimposed on the moving image using the technique disclosed in Patent Document 1 or Patent Document 2, the detection result represented by the vector data is monitored together with the moving image captured by the monitoring camera. It can be provided to the staff at the same time, and the work burden on the surveillance staff can be greatly reduced.

In Patent Document 1, a video captured by a surveillance camera is recorded, and at the same time, additional information such as a shooting date and time is recorded on a part of the video so that the video with the additional information superimposed can be reproduced. Technology is disclosed. In the technique disclosed in Patent Document 2, bitmap data representing additional information is created and encoded in the same manner as image data, and the encoded image data and the above-described bitmap are encoded. Both a request to prevent falsification of image data including additional information by combining the data and a request to remove and display additional information at the time of reproduction are satisfied.
Japanese Patent Laid-Open No. 9-46635 JP 2001-111959 A

  Incidentally, the additional information assumed in each of the above-mentioned patent documents is information such as the shooting date and time and the shooting location, and the content of this additional information is treated as part of the evidence as it is. For this reason, in the technical fields related to these patent documents, it is emphasized that the additional information is reliably combined with the video information, and that the contents are not changed after the video information and the additional information are once recorded. Yes.

However, the vector data itself obtained by applying the moving object extraction technique is a judgment material to be provided to the observer, and does not have an evidence capability in itself. Therefore, such vector data is not information that cannot be changed, but rather information that can be considered even when editing work should be actively performed to support the monitoring work.
For example, in a monitoring system that monitors an intruder into a large-scale facility such as a harbor, the size of an object to be detected may change significantly depending on the situation. In such a case, based on the vector data obtained by the moving object extraction process, the detected moving objects are classified by size, and editing is performed to display only the vector data corresponding to the moving objects of the required size. is required.

On the other hand, for example, after the video data and the vector data are synthesized using the technique disclosed in Patent Document 2, the editing described above is performed on the information represented by the vector data included in the synthesized data. It is impossible in principle to add.
Therefore, the technique disclosed in each of the above-described patent documents is not suitable for use in superimposing vector data premised on editing later on an image displayed as moving image data as additional information.

On the other hand, if the vector data is handled as being completely independent of the moving image data, it becomes difficult to guarantee the synchronization between the vector data and the corresponding frame of image data.
The present invention relates to a moving image data management method and moving image data capable of handling vector data as editable additional information while ensuring synchronization between the frame of image data constituting the moving image data and vector data to be superimposed on the frame An object is to provide a management device and a moving image display device.

The first moving image data management method according to the present invention includes a combining step executed in the moving image data management device, an image display step executed in the moving image display device, and an overwriting step.
The principle of the first moving image data management method according to the present invention is as follows.
Display by a moving image display device via a moving image data management device that manages image data corresponding to a video imaged by an imaging device and vector data obtained by performing a moving object extraction processing by a moving object detection processing device corresponding to this video image In the moving image data management method provided for the processing, the combining step executed in the moving image data management apparatus is the position and size of the moving object obtained by performing the moving object extraction process corresponding to this image data at the end of the image data of each frame. Are combined and transmitted to the moving image display device. The image display step executed in the moving image display device performs display processing based on the image data of each frame every time image data of each frame is received from the moving image data management device. Next, in the overwriting step, the line drawing representing the graphic generated based on the vector data is overwritten on the image reproduced based on the image data.

The operation of the first moving image data management method configured as described above is as follows.
In the combining step executed by the moving image data management device, the corresponding vector data is combined at the end of each frame of the image data corresponding to the video imaged by the imaging device, and used for display processing by the moving image display device. As a result, it is possible to guarantee synchronization between the image data and the vector data while maintaining independence between the image data and the vector data.

  Also, in the image display step executed in the moving image display device, after the image represented by the image data of each frame received from the moving image data management device is displayed, the overwriting step is executed and combined with the end of the image data A line drawing is generated based on the vector data, and this line drawing is overwritten on the above-described image. This process is repeated for each frame, and a natural image represented by image data is displayed by displaying an image in which a figure represented by vector data is superimposed on an image represented by image data of each frame constituting moving image data. Can be provided to the user by superimposing a moving image represented by a continuous sequence of images and a moving image represented by a sequence of figures represented by vector data.

The second moving image data management method according to the present invention comprises a video file forming step, an additional information file forming step, an accumulation step, a search step, a return step and a transfer step which are executed in the moving image data management apparatus.
The principle of the second moving image data management method according to the present invention is as follows.
Display by a moving image display device via a moving image data management device that manages image data corresponding to a video imaged by an imaging device and vector data obtained by performing a moving object extraction processing by a moving object detection processing device corresponding to this video image In the moving image data management method used for the processing, the video file forming step executed in the moving image data management device is photographed by the imaging device every predetermined time that is an integral multiple of an interval of receiving one frame of video from the imaging device. The image data corresponding to the recorded video is collected to form one video file. The additional information file forming step forms an additional information file in which the vector data respectively obtained from the motion detection treatment apparatus is collectively described corresponding to the image data of a plurality of frames collected as one video file in the video file forming step. To do. In the accumulation step, the additional information file and the video file are associated with each other and accumulated in the database. The search step searches the database for additional information files and video files corresponding to the time specified in the request in response to a request for moving image data from the moving image display device. The returning step forms a file of an appropriate format including a link describing the linkage with the video file corresponding to the additional information file together with the contents of the searched additional information file, and returns the file to the moving image display device. The transfer step transfers the corresponding video file to the moving image display device in response to the video file transfer request based on the link.

The operation of the second moving image data management method configured as described above is as follows.
The image data corresponding to the video imaged by the imaging device is collected into a video file in an appropriate format (for example, Motion JPEG (hereinafter abbreviated as M-JPEG) format) every predetermined time in the video file forming step. At the same time, in the additional information file forming step, vector data obtained from the moving object detection processing device corresponding to the video imaged by the imaging device at the predetermined time is also combined into one additional information file. For example, when video information from an imaging device that captures an image of 30 frames per second is filed every minute, one M-JPEG file that summarizes 1800 frames of image data and these frames are supported. One additional information file in which the vector data obtained in this way is collected is formed, and is associated with each other and accumulated in the database in the accumulation step.

  The video file and the additional information file stored in the database in this manner are searched in response to a request from the moving image display device in the search step. Based on the search result, in the return step, for example, an additional information file is stored. An XML format file describing the collected vector data and a link to the video file corresponding to the additional information file is formed. Then, by returning the XML format file to the requesting moving image display device, the contents of the additional information file are first passed to the moving image display device. Thereafter, in the transfer step, the corresponding video file is delivered to the moving image display device in response to the transfer request based on the above-described link.

A first moving image data management apparatus according to the present invention includes an insertion unit and a transmission unit.
The principle of the first moving image data management apparatus according to the present invention is as follows.
Image data corresponding to video captured by the imaging device and vector data obtained by performing moving object extraction processing by the motion detection processing device corresponding to this video, and moving image data for display processing by the moving image display device In the management device, the inserting means receives the motion detection processing result from the motion detection processing device corresponding to each image data between the end of the image data of each frame received from the imaging device and the start of the image data of the next frame. Vector data representing a figure indicating the position and size of the obtained moving object is inserted. The transmission means transmits stream data composed of image data and vector data obtained by the insertion means to the moving image display device.

The operation of the first moving image data management apparatus configured as described above is as follows.
Between each frame of image data corresponding to the video obtained by the imaging device, vector data obtained by the moving object detection processing device corresponding to each frame is inserted by the inserting means, and transmitted to the moving image display device by the transmitting means. Sent.
In this way, the moving image data composed of the image data obtained from the imaging device and the vector data obtained from the motion detection processing device are combined for each frame, and the stream data generated thereby is passed to the moving image display device. Thus, it is possible to provide display processing by the moving image display device in almost real time while ensuring synchronization between the moving image data and the vector data.

A second moving image data management apparatus according to the present invention comprises a video file forming means, an additional information file forming means, a database, a searching means, a linked file forming means, a returning means, and a transferring means. .
The principle of the second moving image data management apparatus according to the present invention is as follows.
Image data corresponding to video captured by the imaging device and vector data obtained by performing moving object extraction processing by the motion detection processing device corresponding to this video, and moving image data for display processing by the moving image display device In the management device, the video file forming unit collects the image data corresponding to the video shot by the imaging device at a predetermined time that is an integral multiple of the interval at which one frame of video is received from the imaging device. Form a file. The additional information file forming means forms an additional information file in which vector data respectively obtained from the motion detection apparatus is collectively described corresponding to the image data of a plurality of frames collected as one video file by the video file forming means. . The database, the additional information file, and the video file are stored in association with each other. In response to a request for moving image data from the moving image display device, the search means searches the database for an additional information file and a video file corresponding to the time specified in the request. The linkage file forming means forms a linkage file of an appropriate format including a link describing the linkage with the video file corresponding to the additional information file together with the contents of the searched additional information file. The return means returns the cooperation file to the moving image display device. The transfer means transfers the corresponding video file to the moving image display device in response to the video file transfer request based on the link.

The operation of the second moving image data management apparatus configured as described above is as follows.
Image data corresponding to the video imaged by the imaging device is collected into video files in an appropriate format (for example, Motion JPEG (hereinafter abbreviated as M-JPEG) format) every predetermined time by the video file forming means. At the same time, the additional information file forming means also collects vector data obtained from the moving object detection processing device corresponding to the video imaged by the imaging device at the predetermined time described above into one additional information file. For example, when video information from an imaging device that captures an image of 30 frames per second is filed every minute, a video file forming unit and an additional information file forming unit collect image data for 1800 frames. One M-JPEG file and one additional information file in which vector data obtained corresponding to these frames are collected are formed, associated with each other, and stored in a database.

  The video file and the additional information file stored in the database in this manner are searched by the search unit in response to a request from the moving image display device, and based on the search result, for example, the additional information file In this way, a linked file in the XML format is formed in which the vector data summarized in (1) and the link to the video file corresponding to the additional information file are described. Then, by returning the linked file in XML format to the requesting moving image display device by the return means, first, the contents of the additional information file are passed to the moving image display device. Thereafter, in response to the transfer request based on the above-described link, the corresponding video file is transferred to the moving image display device by the transfer means.

  In this way, a file in which the image data constituting the moving image data and the vector data to be superimposed on it are formed for each appropriate period (for example, 1 minute in the above example), and these are associated with each other in the database. By accumulating, it is possible to manage enormous amounts of moving image data and vector data as a manageable number of files while ensuring synchronization between moving image data and vector data. For example, moving images corresponding to a specified time It can respond quickly to data searches.

  Also, by returning a link file formed by adding a link to the video file corresponding to the content of the additional information file retrieved from the database to the requesting video display device, the correspondence between the additional information file and the video file Can be surely transferred to the moving image display device, and an appropriate video file can be transferred to the moving image display device in response to a transfer request based on this link, and can be used for moving image display processing.

The moving image display apparatus according to the present invention includes an image display means, a line drawing generation means, an overwrite means, a sorting means, a vector data holding means, a video file holding means, and a display control means.
The principle of the moving image display apparatus according to the present invention is as follows.
The image display means performs image display processing based on the image data of each frame in response to the input of the image data of each frame constituting the moving image data. The line drawing generation unit generates a line drawing representing a figure indicating the position and range indicated by the vector data in response to the input of vector data including the position and range in the image. Each time a line drawing is generated by the overwriting unit and the line drawing generation unit, the generated line drawing is overwritten on the image reproduced by the image display unit. The distribution unit receives stream data including image data of each frame constituting the moving image data and vector data inserted between the image data, and the image data and vector data of each frame included in the stream data are received. Each is extracted, and each time it is extracted, it is sorted and input to the image display means and the line drawing generation means. The vector data holding means includes a set of vector data indicating the position and range in the image corresponding to each of the images of a plurality of frames, and a link describing linkage to a video file made up of image data corresponding to the set of vector data. Receives a linked file containing and holds a set of vector data. The video file holding unit holds a video file obtained in response to a transfer request based on a link included in the cooperation file. Each time the display control means reads the image data of each frame from the video file holding means and sends it to the image display means, the display control means reads the corresponding vector data from the vector data holding means and sends it to the line drawing generation means.

The operation of the moving image display apparatus configured as described above is as follows.
For example, the stream data formed by the above-described moving image data management device is divided into image data of each frame and vector data corresponding to these frames by the distributing means provided in the moving image display device, and each of the image display means And the line drawing display means. Here, in the above-described stream data, vector data corresponding to this frame is inserted after the end of the image data of each frame. Therefore, when vector data is input to the distribution means, the corresponding image data is already present. Is provided for display processing by the image display means. Therefore, by using the line drawing generated by the line drawing generation unit based on the vector data received from the sorting unit for the processing of the overwriting unit, the natural image displayed based on the image data of the corresponding frame is converted into the vector data. It is possible to display a line drawing representing a figure based thereon in a superimposed manner. This process is repeated for each frame, and a natural image represented by image data is displayed by displaying an image in which a figure represented by vector data is superimposed on an image represented by image data of each frame constituting moving image data. Can be provided to the user by superimposing a moving image represented by a continuous sequence of images and a moving image represented by a sequence of figures represented by vector data.

  The vector data included in the cooperation file received from the video data management device is held in the vector data holding means, and the video file transferred in response to the request based on the link described in the cooperation file is the video file. It is held by the holding means. The vector data and the image data held in the vector data holding means and the video data holding means in this way are provided to processing by the image display means and the line drawing generation means via the display control means, respectively. In this way, based on the image data and vector data of each frame included in the video file whose association is indicated by the cooperation file, the moving image and vector data represented by a continuous natural image represented by the image data are used. It is provided to the user as a moving image superimposed with a moving image represented by a series of represented figures.

  According to the first moving image data management method or the first moving image data management device and the moving image display device described above, each frame of image data corresponding to the video obtained from the imaging device, vector data obtained from the moving object detection processing device, and Therefore, the synchronization for each frame can be ensured while maintaining the independence of the image data and the vector data. Therefore, it is possible to simultaneously provide the user with the video captured by the imaging device and the detection result obtained by the moving object detection processing device in real time, and to ensure editability regarding vector data.

  Further, according to the second moving image data management method or the first moving image data management device and the moving image display device, the moving image display device associates the association between the image data and the vector data stored in the database with the link described in the cooperation file. By passing to, synchronization of each frame can be ensured while maintaining independence between the image data and the vector data. Therefore, from the image data and vector data stored in the database, the one corresponding to the desired time range is cut out and passed to the moving image display device, and the natural image represented by the image data is continuously transmitted through the moving image display device. The moving image represented and the moving image represented by the continuation of the graphic represented by the vector data are provided to the user while being superimposed, and the editability of the vector data can be ensured.

  As a result, both the case where the image captured by the imaging device is displayed in real time and the case where the image stored in the database is displayed correspond to the vector data indicating the moving object detection result corresponding to the image data being displayed. Thus, it is possible to realize a monitoring system in which a line drawing generated in this manner or a line drawing selected as necessary is displayed in a superimposed manner and provided to the user.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 shows an embodiment of a moving image data management apparatus and a moving image display apparatus according to the present invention.
In the monitoring system shown in FIG. 1, the monitoring camera 201, the motion detection processing device 202, the server device 210, and at least one terminal device 220 (in FIG. 1, two terminal devices 220a and 220b are shown) Connected via a network with sufficient transmission bandwidth.

  In FIG. 1, a video signal obtained by photographing by the monitoring camera 201 is transferred to the moving object detection processing device 202, and is converted into image data in frame units in the course of image analysis processing by the moving object detection processing device 202. The image data is transferred to the server device 210 together with the vector data obtained by the motion detection processing performed by the motion detection processing device 202.

  In the server device 210 shown in FIG. 1, image data and vector data received from the moving object detection processing device 202 via the transmission path interface 211 are held in the image data holding unit 212 and the vector data holding unit 213, respectively. The accumulation processing unit 214 accumulates video files and additional information files described later in the database 215 and provides them to real-time transmission processing (described later) by the real-time transmission unit 216. Further, the video file and the additional information file stored in the database 215 are subjected to a clipping process (described later) by the clipping processor 217.

  The accumulation processing unit 214 shown in FIG. 1 uses the image data held in the image data holding unit 212 and the vector data held in the vector data holding unit 213 for a period of time when one frame of image data is obtained by the monitoring camera 201. A video file and an additional information file are collectively formed at a predetermined time interval corresponding to an integral multiple, and the formed video file and the additional information file are associated with each other in the database 215 at a file formation time indicating the time when these files are formed. Accumulate accordingly. For example, when image data of 1 frame per 2 seconds is obtained by the monitoring camera 201 (that is, the shooting cycle is 2 seconds), if the time interval for collecting the image data and vector data by the accumulation processing unit 214 is 1 minute, For example, 30 frames of image data are collected as one M-JPEG format video file, and vector data corresponding to these image data is formed as an additional information file corresponding to the above-described video file.

The format of the video file formed in the accumulation processing unit 214 may be any format that can handle natural images of a plurality of frames as digital data. For example, the MPEG format may be adopted.
Further, the real-time sending unit 216 shown in FIG. 1 stores the image data holding unit 212 in the vector data holding unit 213 as the moving object detection processing result corresponding to the image data of each frame each time the image data holding unit 212 holds the image data of each frame. By combining the held vector data with the image data of the frame described above as shown in FIG. 2A, stream data composed of the image data and vector data is formed, and this data is transmitted via the transmission path interface 211. Send stream data to the network.

  The stream data that has reached the transmission path interface 221 of the terminal device 220a or the terminal device 220b via the network is transferred to the superimposed display processing unit 223 via the real-time display control unit 222, and used for the superimposed display processing described later. In the terminal devices 220a and 220b illustrated in FIG. 1, the image data obtained by the superimposed display processing unit 223 is displayed by the display unit 224 and provided to the user. Further, the terminal devices 220a and 220b shown in FIG. 1 include a user interface unit 225. An instruction received from the user via the user interface unit 225 is received by the superimposed display processing unit 223 and the cut-out display control. Part 226 is used for processing.

In FIG. 1, the detailed configuration of the terminal device 220a is shown, and the detailed configuration of the terminal device 220b is not shown.
The cutout display control unit 226 shown in FIG. 1 cuts out part of the video information stored in the database 215 in the server device 210 in response to an instruction from the user received via the user interface unit 225. Is generated and sent to the server apparatus 210 via the transmission path interface 221.

  On the other hand, the cutout processing unit 217 provided in the server device 210 searches the database 215 for the video file and the additional information file corresponding to the specified time in response to the cutout request received via the transmission path interface 211. The searched video file and additional information file are returned to the requesting terminal device 220 (for example, the terminal device 220a) via the transmission path interface 211 in accordance with a procedure described later.

Next, an operation for displaying the image data obtained by the surveillance camera 201 and the vector data obtained by the moving object detection processing device in the monitoring system shown in FIG. 1 in real time will be described.
FIG. 3 is a sequence diagram for explaining transfer of stream data for real-time display.

  A video signal representing an image captured by the monitoring camera 201 shown in FIG. 1 (shown as a monitoring camera in FIG. 3) is firstly shown in FIG. 3 as shaded in FIG. 3) (shown as a moving object detection processing device). Then, in the course of the image analysis processing in the moving object detection processing device 220, for example, it is converted into image data in JPEG format, and further converted into a data format suitable for network transmission (for example, IP packet), and the server device 210 (shown as a server device in FIG. 3) and held in the image data holding unit 212 via the transmission path interface 211 shown in FIG. At this time, the image data of each frame held in the image data holding unit 212 as described above by the real-time sending unit 216 provided in the server device 210 is denoted by reference numeral (1) in FIG. As shown, it is sent to the terminal device 220a (shown as a terminal device in FIG. 3).

  Further, each time the moving body detection processing device 202 receives a video signal for one frame from the monitoring camera 201, the moving body detection processing device 202 performs a moving body detection process based on the image data for one frame corresponding to the video signal and the image data of the immediately preceding frame. In this moving object detection process, the moving object is detected corresponding to the image data of each frame, and vector data indicating the position and range of the detected moving object is generated.

The vector data obtained in this way is transferred to the server device 210 as indicated by a thick arrow in FIG. 3, held in the vector data holding unit 212 provided in the server device 210, and sent in real time. The data is sent to the terminal device 220a by the unit 216.
As shown in FIG. 3, when the transfer of the image data from the server device 210 to the terminal device 220a is completed, the vector data from the motion detection processing device 202 is held in the vector data holding unit 213 of the server device 210. Can be expected. Therefore, after the transfer processing related to the image data for one frame is completed by the real-time transmission unit 216, the operation of transferring the vector data read from the vector data holding unit 213 to the terminal device 220a is continuously performed by the imaging cycle of the monitoring camera 201. By repeating each time, it is possible to transfer the stream data having the configuration as shown in FIG. 2A to the terminal device 220a.

In the stream data having such a configuration, the image data of each frame and the vector data corresponding to each frame are maintained independently of each other, and the image data and the vector are reliably associated with each other. Synchronization with data can be guaranteed.
The format of the image data passed from the moving object detection processing device to the server device and the format of the image data handled by the server device may be any data format that can handle natural images as digital data, such as MPEG. It is also possible to adopt a format.

Next, a method for superimposing and displaying a natural image represented by image data and a line drawing represented by vector data based on such stream data will be described.
FIG. 4 shows a detailed configuration of the superimposed display processing unit.
In the superimposed display processing unit 223 illustrated in FIG. 4, the image restoration unit 231 receives the image data included in the above-described stream data via the real-time display control unit 222, and image data for one frame based on the image data. And is subjected to display processing by the display unit 224 via the frame memory 232. For example, when image data compressed by the JPEG method is obtained by the monitoring camera 201, the image restoration unit 231 performs corresponding restoration processing, reproduces image data representing the original natural image, and stores it in the frame memory 232. Store.

  On the other hand, vector data included in the stream data is passed to the drawing processing unit 233 via the real-time display control unit 222. The drawing processing unit 233 is a graphic indicating the position and size of the moving object based on the setting information related to the drawing processing stored in the setting information storage unit 234 and the vector data received via the real-time display control unit 222. Is generated, and the frame memory 232 is overwritten with the drawing data. In addition, the user inputs desired setting information via the user interface unit 225 shown in FIG. 1 and stores the setting information in the setting information storage unit 234. You can change the thickness and color of the strokes and reflect them in the drawing process.

As a result, as shown in FIG. 5A, an image in which a line drawing drawn based on vector data is superimposed on the natural image reproduced based on the image data as additional information is created on the frame memory 232, It can be provided to the user via the display unit 224 while maintaining the real-time property with the photographing by the monitoring camera 201.
Here, in the above-described stream data, the image data and the vector data are associated with each other but are independent, so that in the superimposed display processing in the terminal device 220, a desired editing process is added to the vector data. Thus, it can be used for display by the display unit 224.

  For example, by changing the setting information in the setting information storage unit 234 via the user interface unit 225, the shape or color of the graphic drawn in the frame memory 232 is changed, or only a graphic having a size greater than or equal to a predetermined value. Is rendered (see FIG. 5B), or an editing process is performed to display the locus of a figure displayed as vector data (see FIG. 5C), and the image displayed on the display unit 224 is displayed. Can be reflected.

Next, a method for cutting out image data and vector data stored in the database for a desired time range and using them for display processing in the terminal device will be described.
FIG. 6 shows detailed configurations of the cutout display control unit of the terminal device and the cutout processing unit of the server device. FIG. 7 shows a sequence diagram for explaining the cut-out display process.
In the cutout display control unit 226 shown in FIG. 6, the procedure control unit 235 generates a file included in the time range specified by this instruction based on the instruction from the user input via the user interface unit 225. A cut-out request is created for each time and sent to the server apparatus 210 via the transmission path interface 221 (see FIG. 7).

  In the cutout processing unit 217 illustrated in FIG. 6, the request reception unit 241 extracts the file formation time from the cutout request received via the transmission path interface unit 211, and designates this file formation time in the file search unit 242. Then, the corresponding video file and additional information file are searched. Based on the search result obtained by the file search unit 242, the cooperative file forming unit 243 shown in FIG. 6 reads the additional information file stored in the database 215 corresponding to the file formation time described above, and FIG. As shown in (b), an XML format linkage file including link information in which the content of the additional information file and the linkage to the video file associated with the additional information file are described in, for example, the HTTP format is formed. Further, the link file formed in this way is sent to the requesting terminal device 220 via the transmission path interface unit 211 (see FIG. 7).

  The link file that has arrived at the terminal device 220 in this way is transferred to the procedure control unit 235 of the cutout display control unit 226 via the transmission path interface 221 shown in FIG. 6, and vector data that is the content of the additional information file. And link information are separated. At this time, the procedure control unit 235 holds the vector data separated from the linkage file in the vector data holding unit 236 for use by the data transfer processing unit 238, and based on the link information, the HTTP format video file transfer request (See FIG. 7).

This video file transfer request is passed to the video file transfer unit 244 via the request receiving unit 241 of the cutout processing unit 217 shown in FIG. 6, and the specified video file is transferred from the database 215 by the video file transfer unit 244. A process of reading and sending to the terminal device 220 via the transmission path interface 211 is performed (see FIG. 7).
As described above, a linkage file in which the additional information file and the video file are associated with each other is formed, passed to the terminal device, and a transfer request for the video file is executed from the terminal device side. Processing can be simplified.

  The contents of the video file transferred in this way are accumulated in the reception buffer 237 via the transmission path interface 221 shown in FIG. 6, and are used for the processing of the data transfer processing unit 238. The data transfer processing unit 238 illustrated in FIG. 6 starts the data transfer operation for the superimposed display process when the image data for an appropriate number of frames is accumulated in the reception buffer 237. That is, each time the data transfer processing unit 238 reads the image data of each frame from the reception buffer 237 and transfers the image data to the superimposed display processing unit 223, the data transfer processing unit 238 receives vector data corresponding to the image data of the transferred frame. And transferred to the superimposed display processing unit 223.

  As a result, the data passed from the data transfer processing unit 238 to the superimposed display processing unit 223 has the configuration shown in FIG. 2A. Therefore, even when the information stored in the database 215 is cut out and displayed, the image is displayed. Data and vector data can be reliably synchronized, and the vector data synchronized with the image data of each frame can be superimposed and displayed and provided to the user.

Of course, in accordance with an instruction from the user interface unit 225, as described above, the vector data is edited in the superimposed display process, so that the user can selectively select an index that is effective for monitoring the facility. Can be displayed and provided to the user.
Also, if the setting information used for such vector data editing processing is stored together with stream data composed of vector data and image data, and transferred to another terminal device 220, the setting information used between the plurality of terminal devices It is also possible to share a video on which the same additional information is superimposed.

As described above, according to the present invention, the image data of each frame representing a natural image and the line drawing represented by the vector data indicating the moving object extraction processing result obtained corresponding to the image data are included in this line drawing. It is possible to superimpose and display in real time while performing a desired editing process and to be used for monitoring work by the user.
As a result, in a monitoring system that performs monitoring based on an image captured by a monitoring camera or the like, it is possible to automatically detect an intruder or the like by using a moving object extraction technique, and to support a monitoring operation by the user. .

In particular, surveillance systems for facilities with vast sites such as harbors and airports have a large field of view captured by a single surveillance camera, and many surveillance cameras are provided. The merit to introduce is thought to be very large.
Further, according to the present invention, video data composed of image data of each frame obtained by the monitoring camera and additional information composed of vector data corresponding to the moving object extraction processing result are cut out for a desired time range, and the terminal By superimposing the image data and the additional information on the apparatus, it can be used for monitoring work by the user.

  As a result, the information accumulated in the database can be used for analysis of intruder entry routes and optimization of setting information applied in vector data editing processing, etc., thus improving the effectiveness of the automatic monitoring system. And very useful.

It is a figure which shows embodiment of the moving image data management apparatus and moving image display apparatus concerning this invention. It is explanatory drawing of a data format. It is a figure explaining real-time display operation. It is a figure which shows the detailed structure of a superimposition display process part. It is a figure which shows the example of the image on which the additional information was superimposed. It is a figure which shows the detailed structure of the cut-out display control part of a terminal device, and the cut-out process part of a server apparatus. It is a sequence diagram explaining cut-out display processing. It is a figure which shows the structural example of the conventional general monitoring system.

Explanation of symbols

201, 401 Monitoring camera 202 Moving object detection processing device 210 Server device 220 Terminal device 211, 221 Transmission path interface 212 Image data holding unit 213 Vector data holding unit 214 Storage processing unit 215 Database 216 Real time sending unit 217 Cutout processing unit 222 Real time display control Unit 223 superimposed display processing unit 224 display unit 225 user interface unit 226 cutout display control unit 231 image restoration unit 232 frame memory 233 drawing processing unit 234 setting information storage unit 235 procedure control unit 236 vector data holding unit 237 reception buffer 238 data transfer processing Unit 241 request reception unit 242 file search unit 243 cooperative file formation unit 244 video file transfer unit 402 display device 403 video tape recorder

Claims (5)

Display by a moving image display device via a moving image data management device that manages image data corresponding to a video imaged by an imaging device and vector data obtained by performing a moving object extraction processing by a moving object detection processing device corresponding to this video image In the video data management method used for processing,
In the video data management device,
A combining step of combining vector data representing a figure indicating the position and size of the moving object obtained by performing the moving object extraction process corresponding to the image data at the end of the image data of each frame and transmitting the combined data to the moving image display device; Run,
In the video display device,
Each time image data of each frame is received from the moving image data management device, an image display step for performing display processing based on the image data of the frame;
Next, an overwriting step of overwriting a line drawing representing a graphic generated based on vector data on an image reproduced based on the image data is performed. Display by a moving image display device via a moving image data management device that manages image data corresponding to a video imaged by an imaging device and vector data obtained by performing a moving object extraction processing by a moving object detection processing device corresponding to this video image In the video data management method used for processing,
The moving image data management device includes:
A video file forming step of forming a single video file by grouping together image data corresponding to videos shot by the imaging device at a predetermined time that is an integral multiple of an interval of receiving one frame of video from the imaging device. When,
In the video file forming step, an additional information file forming for forming an additional information file in which vector data respectively obtained from the moving object detection treatment apparatus is collectively described corresponding to a plurality of frames of image data collected as one video file Steps,
Performing an accumulation step of associating the additional information file and the video file in a database,
In response to a request for moving image data from the moving image display device, a search step of searching the database for additional information files and video files corresponding to the time specified in the request;
A return step of forming a file of an appropriate format including a link describing the linkage with the video file corresponding to the additional information file together with the content of the searched additional information file and returning the file to the moving image display device And
And a transfer step of transferring the corresponding video file to the video display device in response to a video file transfer request based on the link. Image data corresponding to video captured by the imaging device and vector data obtained by performing moving object extraction processing by the motion detection processing device corresponding to this video, and moving image data for display processing by the moving image display device In the management device,
A moving object obtained as a result of the moving object detection processing from the moving object detection processing device corresponding to each image data between the end of the image data of each frame received from the imaging device and the start of the image data of the next frame. Insertion means for inserting vector data representing a figure indicating a position and size;
A moving image data management apparatus comprising: transmission means for transmitting stream data composed of image data and vector data obtained by the insertion means to the moving image display apparatus. Image data corresponding to video captured by the imaging device and vector data obtained by performing moving object extraction processing by the motion detection processing device corresponding to this video, and moving image data for display processing by the moving image display device In the management device,
Video file forming means for collectively forming image data corresponding to video captured by the imaging device at a predetermined time that is an integral multiple of an interval of receiving one frame of video from the imaging device. When,
Additional information file forming means for forming an additional information file that collectively describes vector data respectively obtained from the moving object detection treatment apparatus corresponding to a plurality of frames of image data collected as one video file by the video file forming means. When,
A database for storing the additional information file and the video file in association with each other;
In response to a request for moving image data from the moving image display device, search means for searching the database for additional information files and video files corresponding to the time specified in the request;
Linked file forming means for forming a link file of an appropriate format including a link describing the link with the video file corresponding to the additional information file together with the content of the searched additional information file;
Return means for returning the cooperation file to the video display device;
A moving picture data management apparatus comprising: a transfer means for transferring a corresponding video file to the moving picture display apparatus in response to a video file transfer request based on the link. Image display means for performing image display processing based on the image data of each frame in response to input of image data of each frame constituting the moving image data;
Line drawing generating means for generating a line drawing representing a figure indicating the position and range indicated by the vector data in response to the input of vector data including the position and range in the image;
Overwriting means for overwriting the generated line drawing on the image reproduced by the image display means each time a line drawing is generated by the line drawing generating means;
Receives stream data consisting of image data of each frame constituting video data and vector data inserted between the image data, and extracts image data and vector data of each frame included in the stream data, respectively. A sorting unit that sorts and inputs the image display unit and the line drawing generation unit, respectively, for each extraction;
A linkage file including a set of vector data respectively indicating a position and a range in the image corresponding to each image of a plurality of frames and a link describing linkage to a video file composed of image data corresponding to the set of vector data Vector data holding means for receiving and holding the set of vector data;
Video file holding means for holding a video file obtained in response to a transfer request based on a link included in the linkage file;
Each time image data of each frame is read from the video file holding means and sent to the image display means, the display control means reads out the corresponding vector data from the vector data holding means and sends it to the line drawing generation means. A moving image display device characterized by that.

Images