JP2009171472A - System and device for monitoring network video image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently confirm video image data picked up with a plurality of cameras. <P>SOLUTION: This video image network system includes a plurality of cameras for picking up video images, a retrieval device for retrieving video images picked up with the cameras and transmitting retrieving information created based on the video images picked up with the cameras to a monitor device, a video image storage device for storing video images picked up with the cameras, and the monitor device for displaying the picked-up video images. The monitor device displays a retrieving image contained in the retrieving information received from the retrieving device, determines whether or not a video image relevant to the retrieving information received from the retrieving device is required based on the retrieving information, and when it is judged that the video image relevant to the retrieving information is required, requires the video image storage device to provide the video image relevant to the retrieving information, and displays video image data acquired from the video image storage device. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a network video monitoring system including a plurality of cameras.

  With the widespread use of IP-capable imaging devices such as Web cameras, it has become possible to construct a wide-area and large-scale monitoring system using an IP network. In such a monitoring system, it is possible to acquire a large amount of monitoring images, and multi-faceted monitoring is possible.

However, if all of the acquired large number of monitoring images are to be displayed on the monitor device, a large number of monitor devices are required, and an efficient monitoring system cannot be configured. Thus, a method has been proposed in which a list of low-definition monitoring images (thumbnail images) is displayed and a high-definition monitoring image is displayed only for the selected monitoring image.
JP 2007-53568 A

  However, when a large number of monitoring images are displayed, it is difficult to select a monitoring image for displaying a high-definition monitoring image from thumbnail images with the above-described conventional technology. Therefore, a means for efficiently confirming the monitoring image included in the thumbnail image is required. For example, there are means for extracting important images from a large number of monitoring images and efficiently searching for related monitoring images.

  An object of the present invention is to present a network video monitoring system for efficiently confirming a large number of monitoring images.

  According to a representative aspect of the present invention, a video network system comprising a plurality of cameras that capture video and a monitor device that displays video captured by the camera, the video captured by the camera Comprising: an information generation device for generating search information; a search device for searching for video captured by the camera; and a video storage device for storing video captured by the camera. The video captured by the camera is acquired, search information is generated from the acquired video, the generated search information is transmitted to the search device, and the search device is received from the information generation device Search information is stored in a storage device, the stored search information is transmitted to the monitor device, and the monitor device receives the search information received from the search device. The search image included in the search information is displayed, the necessity of the video related to the search information is determined based on the search information received from the search device, and the video related to the search information is required The video storage device requests the video related to the search information, and the video storage device sends the requested video data to the request source monitor according to the video request from the monitor device. The monitor device displays the video data acquired from the video storage device.

  According to an embodiment of the present invention, a video can be efficiently confirmed by searching for an image (video) based on search information created from the video.

  Next, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating an example of a network video monitoring system according to an embodiment of this invention. FIG. 2 is a block diagram showing an outline of communication between components of the network video monitoring system according to the embodiment of the present invention.

  1, the network video monitoring system includes a camera group 2-0 and 2-1 including a plurality of cameras 220, a monitoring center 1 that controls the plurality of cameras 220 and displays an image of a desired camera 220, and a monitoring function. A network 100 that connects the center 1 to each camera group 2-0, 2-1 and a sensor network 8 that is connected to the sensor information conversion device 7 of the monitoring center 1 and transfers information from the sensor node are mainly configured. .

  In the network video monitoring system, a plurality of cameras 220 are installed in a building or the like, and an administrator or a supervisor of the monitoring center 1 monitors images of the plurality of cameras 220 and stores the images of each camera 220 in the storage device 4. It is something to do.

  The monitoring center 1 includes a display PC (monitor device) 5 that displays thumbnail images of a plurality of cameras 220 and displays desired camera 220 images as high-definition images. A thumbnail image or a high definition image displayed on the monitor 50 of the display PC 5 is monitored. Further, when receiving an event from a preset sensor node, the display PC 5 displays an image of the camera 220 corresponding to the position of the sensor node on the monitor 50. A sensor node is fixed at a predetermined position and is associated with a predetermined camera 220, and an image of the camera 220 is displayed on the display PC 5 from an event of the sensor node, thereby passing a person or an object. Can be monitored. Further, the display PC 5 can display the image of the camera 220 designated by the administrator's operation with high definition.

  In order to realize the above-described functions, the monitoring center 1 includes a display PC 5 that displays images, a storage device 4 that requests and stores images from the camera groups 2-0 and 2-1, and a camera group 2-0, A search PC (retrieval device) 6 that acquires image thumbnails and feature amounts from 2-1 and distributes them to the display PC 5, a request conversion device 3 that converts image requests from the storage device 4 as will be described later, and a sensor The sensor information conversion device 7 converts information from the network 8 as described later and associates the information with the identifier of the camera 220. Details of each part will be described below.

<Configuration of camera group>
The monitoring center 1 manages a plurality of camera groups 2-0 and 2-1, and each camera group is set for each predetermined area (for example, each floor) such as a building or for each camera 220. One camera group 2-0 and 2-1 includes a plurality of cameras 220 and one transmission control device 20. In the following description, since the configurations of the camera groups 2-0 and 2-1 are the same, the same reference numerals are assigned to the same components, and redundant descriptions are omitted.

  The camera 220 installed at a predetermined position captures and outputs an image (moving image) with a predetermined resolution at a predetermined frame rate. The camera 220 is connected to a camera PC (computer) 210.

  The camera PC 210 is a computer having a CPU, a memory, and a communication interface. As shown in FIG. 2, the storage device 4 of the monitoring center 1 frame-by-frame the moving image captured by the camera 220 in response to a request from the storage device 4. Send to. Therefore, the camera PC 210 transmits the latest one frame (image) to the storage device 4 only when there is an image request from the storage device 4, and the generation and characteristics of thumbnail images described later for the other frames. Discard when the amount extraction is complete.

  Further, as shown in FIG. 2, the camera PC 210 generates a thumbnail image from the image captured by the camera 220, extracts the feature amount of the image, and searches the thumbnail PC and the feature amount for the search PC 6 of the monitoring center 1. Send to. The thumbnail image is obtained by reducing the resolution of a moving image taken by the camera 220. For example, when the camera 220 outputs an image having a resolution of 640 × 480 dots (VGA), the camera PC 210 uses 128 × as a thumbnail image. A 96-dot image is generated. Further, the camera PC 210 extracts the feature amount of the image from the image captured by the camera 220 by a known method such as edge detection or contour detection. For example, using the pixel value information of the image, generate a multidimensional vector indicating the color information and edge pattern distribution in the image, and dimensionally compress the multidimensional vector using principal component analysis etc. A vector of about one hundred dimensions is generated and used as an image feature amount. Note that feature amount extraction based on color information performs composition division in a grid pattern on the entire image, and uses a multidimensional vector created from a color histogram of each divided area as the entire image feature amount. Here, as long as the feature amount represents the feature of the entire image, information such as a luminance value and a brightness value, or a combination thereof may be used instead of color information. In addition, the feature amount extraction based on the edge pattern sets a plurality of characteristic edge patterns in advance, divides the composition into a grid pattern, and counts the number of edge patterns included in each region. A histogram is generated from the number of edge patterns, and a multidimensional vector is generated. In the following description, an image having a higher definition than the resolution or thumbnail image of the original image of the camera 220 is referred to as a high definition image.

  Information transmitted from each camera PC 210 to the monitoring center 1 and information from the monitoring center 1 to each camera PC 210 or camera 220 are transferred via the transmission control device 20.

  Here, a predetermined identifier (for example, an IP address) is assigned to each camera PC 210 in advance, and the monitoring center 1 identifies the camera 220 based on the IP address.

  The storage device 4 designates the IP address of the camera PC 210 that acquires the image and transmits an image request to the request conversion device 3. The request conversion device 3 converts the destination of the image request from the storage device 4 into an address and a port number of the transmission control device 20 of each camera group, as will be described later, and transmits it to the network 100.

  When the transmission control apparatus 20 receives an image request from the request conversion apparatus 3 via the network 100, the transmission control apparatus 20 identifies the camera PC 210 from the port number of the image request and transfers the image request to the identified camera PC 210. For this reason, the camera PC 210 includes a transfer destination table 230 indicating the correspondence between the port number and the camera PC 210. The transfer destination table 230 is set in advance.

  The camera PC 210 returns one frame of the latest image (high definition image) in response to the image request from the transmission control device 20. The transmission control device 20 converts the image received from the camera PC 210 into a port number that identifies the camera PC 210 and then transmits the image to the request conversion device 3. The request conversion device 3 converts the port number of the received image into the IP address of the camera PC 210 and transmits it to the storage device 4. The storage device 4 specifies the camera PC 210 from the IP address of the transmission source of the received image and stores the image.

  The transmission control device 20 places a predetermined number of camera PCs 210 under it, transfers the image request transferred from the request conversion device 3 to the camera PC 210, and sends a response to the image request to the priority and sensor information as will be described later. Based on this, the frame rate is controlled. In order to perform priority control, the transmission control device 20 includes a priority history table 240.

  Further, the transmission control device 20 is placed on the monitoring target base side (that is, the camera group side), and the request conversion device 3 is placed on the monitoring center 1 side that performs aggregated monitoring, and is connected to the wide area network 100 in the meantime. As a result, the traffic flowing through the network 100 can be reduced.

  In the embodiment of the present invention, the configuration in which the camera PC 210 and the camera 220 are independent is shown, but the configuration in which the camera PC 210 and the camera 220 are integrated may be used.

<Configuration of monitoring center>
Next, components of the monitoring center 1 will be described below with reference to FIGS. 1 and 2.

  The storage device 4 is configured by a computer including a CPU, a memory, a communication interface, and a storage device. The storage device 4 places a large number of camera PCs 210 under management, and transmits an image request to each camera PC 210 to acquire a high-definition image. In addition, a high-definition image is returned in response to a stored image acquisition request from the display PC.

  The sensor information conversion device 7 includes a computer having a CPU, a memory, and a communication interface, and transmits information transmitted from the sensor network 8 including the infrared node 820, the name tag sensor 810, and the wireless base station 80, to the transmission control device 20 and The message is converted into a message format and a transmission procedure that can be interpreted by the search PC 6 and transmitted to each of them.

  The sensor information conversion device 7 includes a sensor-camera management table 70 that identifies the identifier of the camera PC 210 corresponding to the sensor node from the identifier of the sensor node. The sensor-camera management table 70 is a table in which a preset IP address of the camera PC 210 is set for an identifier of a fixed node among sensor nodes. For example, a person or an object passing near the fixed node can be photographed. The IP address of the camera PC 210 connected to the correct camera 220 is set.

  The search PC 6 is composed of a computer including a CPU, a memory, a communication interface, and a storage device. Further, the search PC 6 stores the feature amount and thumbnail image of the image from the camera PC 210 and the sensor information received from the sensor information conversion device 7, and the detection information (sensor information from the sensor network) and the thumbnail for the display PC 5. Send the image. Further, the search PC 6 sends information on the camera PC 210 to be prioritized to the transmission control apparatus 20.

  The request conversion device 3 includes a computer having a CPU, a memory, and a communication interface. The request conversion device 3 is arranged on a communication path between the storage device 4 and the camera PC 210, and relays an image request from the storage device 4 to the camera PC 210. In addition, the transmission source and transmission destination of the message are changed so as to pass through the transmission control device 20, and the transmission source and transmission destination of the message from the transmission control device 20 toward the storage device 4 are changed.

  The request conversion device 3 converts the destination (the IP address of the camera PC 210) included in the image request from the storage device 4 into the IP address and port number of the transmission control device 20 of each camera group, and the transmission source is the request conversion device 3. The message replaced with is transmitted to the transmission control device 20. Further, the request conversion device 3 converts the destination included in the reply (image) from the transmission control device 20 to the storage device 4, and sends the message converted to the IP address of the camera PC 210 included in the reply to the storage device 4. Send. Therefore, the request conversion device 3 includes an address conversion table 30 for converting the IP address and port number of the camera PC 210.

  The display PC 5 is composed of a computer having a CPU, a memory, and a communication interface, and displays thumbnail images of a plurality of cameras 220 and desired camera 220 images on the monitor 50 as high-definition images. The detailed configuration of the display PC 5 will be described later with reference to FIG.

  The display PC 5 notifies the transmission control device 20 of the identifier (camera ID) of the camera PC 210 designated by the administrator or the monitor as the camera PC 210 that prioritizes the frame rate. Based on this notification, the transmission control device 20 updates the priority history table 240.

<Sensor network>
Next, the sensor network 8 will be described. In FIG. 1, the sensor network 8 includes a radio base station 80 connected to the sensor information conversion apparatus 7 and a sensor node connected to the radio base station 80 via the radio network 800.

  As the sensor node according to the embodiment of the present invention, an example in which a fixed node fixed at a predetermined position and a mobile node attached to a person or an object is shown. In FIG. 1, the sensor network 8 includes an infrared node 820 that transmits a preset identifier from an infrared transmitter as a fixed node. The sensor network 8 includes a name tag sensor 810 that transmits the identifier received from the infrared node 820 as a mobile node to the radio base station 80.

  FIG. 3 is an explanatory diagram showing video distribution processing of the network video monitoring system according to the embodiment of this invention.

  The camera PC 210 executes an image registration program 410. In the image registration program 410, three threads of a video acquisition thread 110, a video distribution thread 120, and a search information registration thread 140 (see FIG. 4) are executed in parallel. Each thread is repeatedly executed.

  The storage device 4 executes a storage device program 430. The storage device program 430 is repeatedly executed.

  First, the processing of the video acquisition thread 110 will be described.

  The video acquisition thread 110 acquires the video data acquired by the camera 220 and stores the image data as a frame image in a buffer provided in the memory (111).

  Next, using the frame image stored in the buffer, a background image having the pixel value as the average value of the pixel values on the frame image series is calculated. Then, the background image is updated to the newly calculated background image (112).

  Next, a background difference image having a pixel value as a difference between the frame image and the background image is calculated. Then, the calculated background difference image and frame image are set as data transmitted by the inter-thread communication 130 (113).

  Thereafter, the background differential image is sent from the video acquisition thread 110 to the video distribution thread 120 by inter-thread communication (130).

  When receiving the request from the storage device 4 (121), the search information registration thread 140 acquires the latest frame image from the data transmitted by the inter-thread communication 130 (122), and uses the acquired frame image as the video storage server. (123).

  Next, the video storage server program 430 executed by the storage device 4 will be described.

  The storage device 4 transmits a high-definition image request to the camera PC 210 at a predetermined timing (for example, when an event is detected by another system at a predetermined time interval) (221). When the video data photographed by the camera 220 is acquired (222), the time interval of the acquired video data is adjusted, and the acquired video data is stored in the storage device (223). The adjustment of the time interval of the video data refers to the information attached to the video data acquired from the camera PC 210 (for example, the frame rate of the video data) and matches the frame rate of the format recorded in the storage device 4. The converted video data is recorded and the converted video data is recorded. In addition, when converting the frame rate of video data from a high rate to a low rate, it is only necessary to delete excess frames. On the other hand, when the frame rate of video data is converted from a low rate to a high rate, the preceding and following images may be used as missing frames, or the missing frames may be complemented.

  Since the video data whose time interval is adjusted in this way is stored in the storage device, image data with different frame rates can be stored in a unified format.

  FIG. 4 is an explanatory diagram showing an outline of search information registration processing of the network video monitoring system according to the embodiment of this invention.

  First, the processing of the video acquisition thread 110 is the same as that described above with reference to FIG.

  When the search information registration thread 140 acquires the data (frame image, background difference image) sent by the inter-thread communication 130 (141), the event detection process 142 detects the event. If no event is detected (result of 143 is “N”), the process returns to step 141 to acquire the next image. On the other hand, when an event is detected (result of 143 is “Y”), an image feature amount is extracted (144).

  Thereafter, a search information registration request including the extracted feature amount and the identifier of the original video data is sent to the search PC 6 (145). Note that the search information registration request may include the identifier of the camera 220 that captured the original video data, instead of the identifier of the original video data. When a registration confirmation notification of search information is received from the search server 21 (146), the process returns to step 141 to acquire the next image.

  Next, the search PC program 420 executed by the search PC 6 will be described.

  When receiving the search information registration request from the camera PC 210 (211), the search PC 6 stores the received search information (image feature amount and video data identifier) in the storage device (212). Thereafter, a registration confirmation notification of search information is sent to the camera PC 210 (213). The camera PC 210 transmits a registration event notification to the search PC 6 after storing the search information.

  FIG. 5 is a diagram showing an example of the configuration of the display PC 5 according to the embodiment of the present invention.

  The display PC 5 includes a CPU 301, a main memory 302, an input unit 303, a display unit 304, a communication unit 305, and a storage unit 310.

  The CPU 301 executes various arithmetic processes. The main memory 302 stores programs and data for the CPU 301 to execute various arithmetic processes. The input unit 303 receives input of information necessary for processing executed on the display PC. The input unit 303 is, for example, a keyboard. The display unit 304 outputs an image taken by the camera 220 and displays it on the monitor 50.

  The storage unit 310 stores programs and data for executing various arithmetic processes. The program stored in the storage unit 310 is loaded into the main memory 302 and then executed by the CPU 301.

  The programs stored in the storage unit 310 include an OS 311, an event acquisition program 316, a search device control program 317, a storage device control program 318, a data management program 319, a playback mode determination program 320, a playback section determination program 321, and display data generation A program 322 and a data transmission / reception program 323 are included.

  The OS 311 is an operating system executed on the display PC 5. The event acquisition program 316 acquires an event detected by the search PC 6.

  The search device control program 317 requests the specified image from the search PC 6 or requests an image similar to the specified image.

  The storage device control program 318 requests the designated image from the storage device 4 in which high-definition images (video) are stored. The data management program 319 manages the acquired image data.

  The reproduction mode determination program 320 determines a mode for reproducing an image acquired from the storage device 4. Details of the playback mode will be described later. The reproduction section determination program 321 determines a section for reproducing the image acquired from the storage device 4.

  The display data generation program 322 creates data for displaying the acquired image data.

  The data transmission / reception program 323 transmits information related to the acquisition target image to the search PC 6 or the storage device 4 or receives an image transmitted from the search PC 6 or the storage device 4.

  Data stored in the storage unit 310 includes camera information data 312, storage device information data 313, playback mode determination data 314, and display standby information data 315.

  The camera information data 312 stores information regarding the camera 220. For example, camera identification information (for example, IP address), arrangement information of the camera 220, and the like are stored.

  The storage device information data 313 is used to specify the storage device 4 in which the requested monitoring image is stored, such as when a plurality of storage devices 4 are included in the system. The reproduction mode determination data 314 stores information necessary for determining the reproduction mode.

  The display standby information data 315 includes information related to an event when the event detected by the search PC 6 is acquired by the display PC 5 and an image related to the event stored in the storage device 4 cannot be displayed. Stored. The display PC 5 sequentially displays images based on the information stored in the display standby information data 315. Details of the display standby information data 315 will be described later with reference to FIG.

  FIG. 6 is a flowchart showing a procedure for displaying a high-definition image (video) received from the storage device 4 according to the embodiment of the present invention. This process is executed by processing each program, and when an event detected by the search PC 6 is acquired, a high-definition image (video) related to the event is acquired from the storage device 4 and displayed. .

  The display PC 5 executes the event acquisition program 316 and acquires the event detected by the search PC 6 (S1101). The event is detected in the search PC 6 based on the sensor information received from the sensor information conversion device 7. In the search PC 6, an event may be detected from the frame image and the background difference image acquired from the camera PC. Specifically, the sum of squares of the pixel values of the background difference image is obtained, and the presence / absence of a motion component is determined by comparing the obtained square sum with a predetermined threshold value. As a result, it is determined that an event has been detected when the square sum of the obtained pixel values is large (there is a motion component).

  If no event is detected for a certain period of time, the images from the camera with the oldest playback time are sequentially played back. The last reproduction time is stored in the camera information data 312, for example.

  The display PC 5 determines whether or not a high-definition monitoring image related to the event acquired in the process of S1101 can be reproduced (S1102). If it is determined in step S1102 that the high-definition monitoring image related to the event is not reproducible (result of S1103 is “N”), data related to the event is added to the display standby information data 315 (S1104). ). The video corresponding to the data stored in the display standby information data 315 is reproduced at a predetermined timing. When the video is reproduced, the corresponding data is deleted from the display standby information data 315.

  On the other hand, if it is determined in step S1102 that the high-definition monitoring image related to the event can be reproduced (result of S1103 is “Y”), the display PC 5 executes the reproduction mode determination program 320. The playback mode is determined (S1105).

  The playback mode includes a live mode, a normal playback mode, and a reverse playback mode. In the live mode, the storage device 4 passes through the video acquired from the camera and transmits it to the display PC 5. It is characterized by a high frame rate and high definition. In the normal playback mode, the image data stored in the storage device 4 is played back. Since the image data stored in the storage device 4 is reproduced, it can be reproduced with a delay. In the reverse playback mode, the image data stored in the storage device 4 is reversely played back (reverse playback) from the image at the specified date and time. Video can be confirmed from new information.

  Further, the mode determination method will be described. When an image can be reproduced in real time based on the amount of traffic on the network, it is assumed to be live reproduction. If live playback cannot be executed and the time from the first notification time of the acquired event to the present time exceeds a predetermined maximum continuous playback time, reverse playback is performed from the last notification time. On the other hand, when the time from the first notification time of the acquired event to the present time is less than or equal to a predetermined maximum continuous playback time, normal playback is performed from the first notification time of the acquired event. The predetermined maximum continuous playback time is stored in the playback mode determination data 314.

  In addition, when the number of records registered in the display standby information data 315 exceeds a predetermined number, frame images are thinned out at a predesignated ratio in order to prevent an unreproduced image from being accumulated excessively. Play fast.

  The display PC 5 executes processing corresponding to the determined reproduction mode (S1106).

  When playing back video in the live mode, the display PC 5 acquires video for live playback from the storage device 4 (S1110). As described above, the image for live reproduction is an image transmitted through the video received by the storage device 4 from the camera PC to the display PC 5. The display PC 5 displays the acquired image on the monitor 50 (S1111).

  When playing back video in the normal playback mode, the display PC 5 first executes the playback section determination program 321 to determine the playback section (S1120). In the normal playback mode, since the video is sequentially played back from the first notification time, the first notification time is acquired and the latest report time or the current time is set as the playback section. The display PC 5 acquires the video of the determined playback section from the storage device 4 (S1121). Further, the acquired image is displayed on the monitor 50 (S1122).

  When playing back video in the reverse playback mode, the display PC 5 first executes the playback section determination program 321 to determine the playback section (S1130). In the reverse playback mode, since the video is played back from the latest notification time, the predetermined continuous playback maximum time PT is set as the playback section from the latest notification time. The display PC 5 acquires the video of the determined playback section from the storage device 4 (S1131). Further, the acquired image is displayed on the monitor 50 (S1132).

  When the display of the video is completed, the display PC 5 determines whether or not an input of display end has been received from the administrator (or the monitor) (S1107). When the display end input is not accepted (the result of S1107 is “N”), the process returns to S1101 to acquire an event. When the display end input is accepted (the result of S1107 is “Y”), this process is terminated.

  FIG. 7 is a diagram illustrating an example of the display standby information data 315 according to the embodiment of this invention.

  As described above, the display standby information data 315 stores information related to an event when a video cannot be reproduced after the event is acquired. Further, when an event is acquired, a record may always be added to the display standby information data 315, and the display standby information data 315 may always be referred to for video reproduction.

  The display standby information data 315 includes a camera ID 3151, an issue date / time (first) 3152, an issue date / time (latest) 3153, an issue time 3154, and an issue count 3155.

  The camera ID 3151 is an identifier of the camera that captured the image in which the event was detected. The issue date / time (first) 3152 is the date and time when the event was first detected. The issue date / time (latest) 3153 is the latest issue date / time for the same event. The issue time 3154 is the time from the first issue date to the latest issue date. The number of reports 3155 is the number of reports from the first report to the latest report.

  Information related to the above events is included in the detection information transmitted from the search PC 6. When an event corresponding to the record stored in the display standby information data 315 is acquired again, the issue date (latest) 3153, the issue time 3154, and the issue count 3155 may be updated. Even if the same event is presumed, if a predetermined time has passed since the issue date (latest), it is processed as another event.

  In addition, the order in which the video corresponding to the record of the display standby information data 315 is reproduced is the order of the number of reports per report time, that is, the order of the report time / number of reports. In addition, the video may be reproduced in the order of the number of times of reporting or in the order of long reporting time.

  FIG. 8 is a diagram showing an example of a screen that displays an image (video) according to the embodiment of the present invention.

  In the screen shown in FIG. 8, a high-definition video is displayed on the monitoring video at the top of the screen. In the alert image at the bottom of the screen, a list of low-definition images (thumbnail images) taken for each camera is displayed.

  In the monitoring video, an image from which an event has been acquired or a video corresponding to an image selected by an administrator (or a monitor) is displayed. As the monitoring video, an image acquired from the storage device 4 is displayed.

  The notification image is acquired from the search PC 6. Each image included in the notification image may be updated every time it is issued, and the image may be periodically updated when it is not issued. When images that have not been reported are periodically updated, the images that have been reported and images that have not been reported may be displayed separately. Further, detection information for identifying whether or not the displayed image is a notification image is also received from the search PC 6, and the CPU 301 can execute the display data generation program 322 to identify whether or not it is a notification image. Can be displayed.

  An administrator (or a supervisor) can check the notification image, select an image that may be abnormal, and search for a related image based on the selected image.

  FIG. 9 is a flowchart showing a processing procedure for searching for an image according to the embodiment of the present invention.

  The display PC 5 accepts selection of an image whose content (video) is to be confirmed from the thumbnail image (S1201). For example, an image is selected from the thumbnail images on the screen shown in FIG.

  The display PC 5 acquires key image candidates based on the image selected in the processing of S1201 (S1202). The key image is an image serving as a search source or a purpose of reproducing a video and is a key for confirming the video. For example, in the selected image, when the face of the person to be confirmed is not clearly displayed, a related image is displayed together with the selected image. The administrator (or monitor) selects a key image from the selected image and related images. A screen for displaying key image candidates and selecting a key image will be described later with reference to FIG.

Subsequently, the display PC 5 accepts selection of a key image (S1203). When the key image is selected, the image is searched in more detail based on the selected key image (S1204). In the search based on the selected key image, the search is performed based on the person in the key image, the shooting time, and the like. The person shown in the key image is specified (searched) based on the feature amount calculated from the face image of the person. In addition, when the person can be identified by the name tag sensor 810, sensor information can be used.
A screen for searching for an image based on the key image will be described later with reference to FIG.

  When the search of the image is completed, the display PC 5 displays the search result (S1205). Search results are displayed in a low-definition image list. A screen for displaying the search result will be described later with reference to FIG.

  An administrator (or a supervisor) can select a list-displayed image, reproduce a video based on the selected image, and perform a re-search.

  When an image is selected from the search result and a video playback instruction is accepted (result of S1206 is “Y”), the display PC 5 displays the video using the specified playback method (S1207). The playback method is, for example, normal playback or reverse playback.

  When the display PC 5 has not received a video playback instruction (the result of S1206 is “N”), or when video playback has been completed, the display PC 5 determines whether an end instruction has been received (S1208). If an end instruction has not been received (the result of S1208 is “N”), the screen shown in FIG. 8 is displayed, and the process returns to S1201. If an end instruction is accepted (the result of S1208 is “Y”), this process ends.

  FIG. 10 is a diagram showing an example of a screen for selecting a key image according to the embodiment of the present invention.

  The key image selection screen is displayed by selecting a thumbnail image of the screen shown in FIG.

  At the top of the key image selection screen, the selected image is displayed together with the previous and next frame images. For the second and subsequent images, images captured at the same time by a camera installed near the camera that captured the selected image are displayed.

  The key image selection screen shown in FIG. 10 shows a case where an image taken by camera # 2 is selected. Similarly, images of camera # 7, camera # 11, and camera # 1 installed near camera # 2 are acquired as key image candidates (S1202 in FIG. 9) and displayed.

  Further, when a key image is selected, a submenu 60 is displayed. Based on the selected key image, whether to search for an image, play a video (normal playback, reverse playback), or display detailed information. You can choose.

  The display PC 5 displays a search screen when executing an image search based on the selected key image. An example of the search screen will be described later with reference to FIG.

  Based on the selected key image, the display PC 5 acquires the corresponding video from the storage device 4 and reproduces it when executing normal reproduction. In this case, a video after the time point when the selected key image is captured may be displayed, or a video image from a certain time before the time point when the selected key image is captured may be displayed.

  When the reverse reproduction is executed based on the selected key image, the display PC 5 displays the video in reverse order starting from the time when the selected key image was taken.

  When displaying the detailed information of the selected key image, the display PC 5 displays, for example, the identifier of the camera in which the selected key image was shot, the installation location, the shooting time, the information of the person who was shot, and the like. .

  FIG. 11 is a diagram illustrating an example of a screen for searching for an image based on a selected key image according to the embodiment of this invention.

  In the search screen, the person included in the key image selected on the key image selection screen shown in FIG. 10 or the low-definition image searched based on the person included in the search key image is the initial search result. Is displayed. The search results are arranged in descending order of the similarity of the persons in the image. As an example, the degree of similarity focuses on a specific part such as an eye or mouth in a face image, extracts the feature amount of each focused part, and extracts each part and each part of the held face (See, for example, JP-A-2005-352727).

  Furthermore, the searched image can be narrowed down by inputting a narrowing condition. Although FIG. 11 shows an example in which the narrowing is performed based on the shooting time, the narrowing may be performed under other conditions. For example, you may narrow down by the installation place (photographing place) of a camera.

  When the search result image is selected, the submenu 60 is displayed as in the key image selection screen shown in FIG. 10, and the image is searched, the video is reproduced, or the detailed information is displayed based on the selected image. Can be displayed.

  According to the embodiment of the present invention, it is possible to efficiently check the presence or absence of an abnormality by extracting and presenting an important video from videos taken by a large number of cameras.

  Further, according to the embodiment of the present invention, it is possible to easily search for an image related to the confirmation target image, so that the stored image can be checked efficiently.

It is a block diagram which shows an example of the network image | video monitoring system of embodiment of this invention. It is a block diagram which shows the outline | summary of the communication between the components of the network video monitoring system of embodiment of this invention. It is explanatory drawing which shows the video delivery process of the network video monitoring system of embodiment of this invention. It is explanatory drawing which shows the outline | summary of the information registration process for search of the network image | video monitoring system of embodiment of this invention. It is a figure which shows an example of a structure of display PC of embodiment of this invention. It is a flowchart which shows the procedure which displays the high definition image received from the storage device of embodiment of this invention. It is a figure which shows an example of the display standby information data of embodiment of this invention. It is a figure which shows an example of the screen which displays the image (video | video) of embodiment of this invention. It is a flowchart which shows the procedure for searching the image of embodiment of this invention. It is a figure which shows an example of the screen for selecting the key image of embodiment of this invention. It is a figure which shows an example of the screen which searches an image based on the key image of embodiment of this invention.

Explanation of symbols

1 Monitoring Center 2 Camera Group 3 Request Conversion Device 4 Storage Device 5 Display PC
6 Search PC
7 Sensor Information Conversion Device 8 Sensor Network 20 Transmission Control Device 30 Address Conversion Table 50 Monitor 60 Submenu 70 Camera Management Table 80 Wireless Base Station 100 Network 210 Camera PC
220 Camera 301 CPU
302 Main memory 303 Input unit 304 Display unit 305 Communication unit 310 Storage unit 312 Camera information data 313 Storage device information data 314 Playback mode determination data 315 Display standby information data 316 Event acquisition program 317 Search device control program 318 Storage device control program 319 Data management program 320 Playback mode determination program 321 Playback section determination program 322 Display data generation program 323 Data transmission / reception program

Claims (10)

A video network system comprising a plurality of cameras that capture video and a monitor device that displays video captured by the camera,
An information generating device for generating search information for video shot by the camera;
A search device for searching video captured by the camera;
A video storage device for storing video captured by the camera,
The information generation device acquires a video shot by the camera, generates search information from the acquired video, transmits the generated search information to the search device,
The search device stores search information received from the information generation device in a storage device, and transmits the stored search information to the monitor device,
The monitor device is
Displaying a search image included in the search information received from the search device;
Based on the search information received from the search device, it is determined whether or not a video related to the search information is necessary, and if it is determined that a video related to the search information is necessary, the video storage device Request video related to the search information,
The video storage device transmits the requested video data to the requesting monitor device according to the video request from the monitor device,
The video network system, wherein the monitor device displays video data acquired from the video storage device.   The monitor device displays the search image for each camera, accepts selection of the search image, and requests a video from the video storage device based on the selected search image. The video network system according to claim 1.   The monitor device further displays an image related to the selected search image, and requests a video from the video storage device based on an image selected from the displayed image. Item 3. The video network system according to Item 2.   2. The monitor device according to claim 1, wherein when the video data requested from the video storage device can be immediately reproduced, the monitor device receives the video shot by the camera as it is via the video storage device. The described video network system. The monitor device is
If the video data requested from the video storage device cannot be played back immediately, it is determined whether the playback time of the video data requested from the video storage device is greater than a predetermined maximum playback time;
The video data acquired from the video storage device is reversely played back when the playback time of the video data requested from the video storage device is longer than the predetermined maximum playback time. The described video network system. In a video network system including a plurality of cameras for capturing video, a monitor device for displaying video captured by the cameras,
The video network system includes:
A search device for searching for video captured by the camera and transmitting search information generated based on the video captured by the camera to the monitor device;
A video storage device for storing video captured by the camera,
The monitor device is
Displaying a search image included in the search information received from the search device;
Based on the search information received from the search device, it is determined whether or not the video related to the search information is necessary, and when it is determined that the video related to the search information is necessary, the video storage device Request video related to the search information,
A monitor device that displays video data acquired from the video storage device.   The monitor device displays the search image for each camera, accepts selection of the search image, and requests a video from the video storage device based on the selected search image. The monitor device according to claim 6.   The monitor device further displays an image related to the selected search image, and requests a video from the video storage device based on an image selected from the displayed image. Item 8. The monitoring device according to Item 7.   7. The monitor device according to claim 6, wherein when the video data requested from the video storage device can be immediately reproduced, the video captured by the camera is directly received via the video storage device. The monitoring device described. The monitor device is
If the video data requested from the video storage device cannot be played back immediately, it is determined whether the playback time of the video data requested from the video storage device is greater than a predetermined maximum playback time;
7. The video data acquired from the video storage device is reversely played back when the playback time of the video data requested from the video storage device is longer than the predetermined maximum playback time. The monitoring device described.

Images