JP2010028594A - Moving object monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem that limit exists in trace performance by constraint of network transmission capacity, when centralized monitoring of the video image of many surveillance cameras is carried out at a surveillance center. <P>SOLUTION: The whole of device is constituted of at least one monitor site and a monitor center which are connected by a network. The monitor site includes an image acquiring unit which acquires a video image photographed by a camera; a moving object extracting unit which extracts a moving object in the acquired video image; a processing assignment table which determines assignment of calculation processing of characteristic amount to extracted moving object; a high-frequency feature calculating unit which calculates the high-frequency feature amount of the moving body, in accordance with the processing assignment table and an image selecting unit which determines the transmission rate of the image transmitted to the monitoring center in accordance with the processing assignment table. The monitoring center includes an image DB, which stores the video images transmitted by the transmitting image selecting unit; a characteristic amount DB which stores the characteristic amount calculated by the high-frequency characteristics calculating unit and a low-frequency characteristic calculating unit which calculates the characteristics which are not calculated by the characteristic amount DB. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an apparatus for monitoring a moving object such as a person or a car using a camera installed in a monitoring area.

  In order to deal with social unrest, such as the increase in crime rates, the number of cameras installed for the purpose of monitoring suspicious persons and suspicious vehicles is increasing. Monitoring using a large number of cameras requires monitoring support technology for efficiently monitoring the monitoring area with limited monitoring personnel resources.

  As such a monitoring support technique, there is a “monitoring system using a plurality of cameras” disclosed in JP-A-2006-146378 (Patent Document 1). Here, there is disclosed a technique for reducing the burden on a supervisor in a system configuration in which videos of a large number of surveillance cameras are centrally monitored at a surveillance center. A moving object is extracted from video captured by each camera using image recognition technology, and the moving object is collated between the cameras. Based on the movement trajectory of the moving body, the abnormal behavior is determined and an alarm is output. With this technology, the monitor can easily monitor a suspicious person without gazing at a large number of monitoring images.

JP 2006-146378 A

  In the above prior art, on the premise that the video shot at the monitoring site is stored in the center, the feature amount is extracted from the video database (DB) stored in the center and the person is tracked.

  However, the frame rate of the video stored in the center cannot be increased due to the limitation of the transmission capacity of the network used for transmitting the video taken at the monitoring site to the center. For this reason, high-frequency features such as motion features cannot be calculated on the center side and cannot be used for tracking. Therefore, the tracking performance has a limit.

  An object of the present invention is to provide a moving object monitoring apparatus that can perform tracking using high-frequency features such as motion features without increasing the frame rate of video stored in the center.

  In order to achieve the object, the entire apparatus is composed of at least one monitoring site and a monitoring center connected by a network. The monitoring site includes a video acquisition unit that acquires video captured by a camera, a moving object extraction unit that extracts a moving object in the acquired video, and a processing allocation table that determines the sharing of feature amount calculation processing for the extracted moving object A high-frequency feature calculating unit that calculates a high-frequency feature amount of the moving object according to the processing allocation table, and a video selection unit that determines a transmission rate of the video to be transmitted to the monitoring center according to the processing allocation table and transmits the video at the rate. . The monitoring center includes a video DB that stores the video transmitted by the transmission video selection unit, a feature DB that stores the feature calculated by the high-frequency feature calculator, and a low-frequency that calculates an uncalculated feature using the feature DB. A feature calculation unit is provided.

  According to the present invention, the high frequency characteristics can be used on the center side without increasing the video data transmitted through the network. This makes it possible to track a moving object with high accuracy.

(Embodiment 1)
Hereinafter, embodiments of the invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a functional configuration of the entire moving object monitoring apparatus according to an embodiment of the present invention.

  The entire apparatus is roughly divided into a monitoring site 100 and a monitoring center 102. Both are connected via the network 104. The monitoring site 100 is an area to be monitored and a monitoring device installed in the area. The monitoring center 102 is a place where the monitor monitors the video of the monitoring site 100 and a center monitoring apparatus installed at the place. Usually, the monitoring site 100 and the monitoring center 102 are physically separated from each other. The network 104 is a computer network such as a LAN (local area network) or a WAN (wide area network), and can exchange data with each other.

  Next, the components of the monitoring site 100 will be described.

  The camera 106 is a photographing device such as a network camera or a video camera. Multiple cameras are installed so that the entire surveillance area can be photographed. The camera installation density may be sparse or dense.

  The video acquisition unit 108 acquires video captured by the camera 106. When the camera 106 is a network camera, image data already digitized via the network is acquired at regular time intervals. In the case of a video camera, after obtaining an analog signal output from the camera, the analog signal is converted into a digital signal.

  The moving object extraction unit 110 extracts moving objects from the video data acquired by the video acquisition unit 108. For example, the background subtraction method may be used to extract the moving object. The background subtraction method saves video data with no moving object as the background, and calculates the difference between the video data showing the moving object and the luminance value for each pixel. This is a method for obtaining a region of a moving object. By using the background subtraction method, the presence / absence and the number of moving objects and the circumscribed rectangle for the area of the moving object are calculated. These are the output of the moving object extraction unit 110.

  The high frequency feature calculation unit 112 calculates a high frequency feature amount for the moving object extracted by the moving object extraction unit 110 according to the processing assignment table 114. The process sharing table 114 is a table that determines whether to perform processing on the monitoring site 100 side or the monitoring center 102 side when calculating a plurality of feature amounts of a moving object. Details of this table will be described later. The feature amount is a vector amount representing a feature unique to the moving object, and is used for tracking the moving object. Features include color and texture features, shape features, and motion features. For example, the mode luminance values of the lower and upper bodies can be used as the color characteristics of the moving object. In this case, the feature quantity is a two-dimensional vector.

  Here, the characteristics of human color and texture are static characteristics that do not change with time, and it is sufficient to have one image for calculation. That is, such a feature is a low frequency feature. On the other hand, the characteristics of human shape and movement are dynamic characteristics that change with time, and an image with a high frame rate is required for calculation. That is, such a feature is a high-frequency feature. Here, as the feature amount of motion, for example, a known three-dimensional higher-order local autocorrelation feature can be used. The three-dimensional higher-order local autocorrelation feature is a feature obtained by extending a higher-order local autocorrelation feature often used in face image recognition or the like in the time direction, and can obtain a feature amount of a moving image. This feature is a 251-dimensional vector quantity obtained by calculating 251 types of local autocorrelation features at each point in the voxel data in which images are arranged in time series and integrating over the entire voxel data.

  The high frequency feature calculation unit 112 calculates the high frequency features of the processing assignment table 114. The calculated high-frequency feature is transmitted to the monitoring center 102 via the network 104 and stored in the feature value DB 116 provided in the monitoring center 102. The feature amount DB 116 is a database (DB) that accumulates feature amounts of moving objects, and details will be described later. Note that all the calculated high-frequency features may be stored in the feature amount DB 116. Further, the transmission video selection unit 118 (to be described later) may store it in the feature amount DB 116 in synchronization with the video transmission timing.

  The transmission video selection unit 118 transmits the video data acquired by the video acquisition unit 108 to the monitoring center 102 side according to the processing allocation table 114. Here, an image having a frame rate necessary and sufficient to calculate low-frequency features excluding features calculated by the high-frequency feature calculating unit 112 is transmitted, and the result is stored in the image DB 120 on the monitoring center 102 side. Details of the video DB 120 will be described later.

  Next, components of the monitoring center 102 will be described.

  The video DB 120 is a database (DB) that stores the image data transmitted by the transmission video selection unit 118. As the format of the image data, a commonly used format such as JPEG (Joint Photographic Experts Group) or PNG (Portable Network Graphics) may be used. Further, moving image data may be stored instead of continuous still images. In this case, a format such as MPEG (Moving Picture Experts Group) may be used.

  The low frequency feature calculation unit 122 uses the feature amount data stored in the feature amount DB 116 and the low frame rate video included in the video DB 120 to calculate the low frequency feature for the moving object extracted by the moving object extraction unit 110. calculate. Here, the low-frequency feature is a feature that is not calculated by the high-frequency feature calculation unit 112 among features necessary for tracking a moving object. Examples of low frequency features include human color and texture features as described above. Note that the video for which the low-frequency feature is to be calculated may be searched from the video DB 120 based on the calculation time of the feature data stored in the feature DB 116.

  The tracking processing unit 124 uses the feature amount of the moving object stored in the feature amount DB 116 to track how the moving object has moved between the cameras. Here, the tracking target is not limited to a person, and any tracking object may be used. Next, using this tracking result, a camera image showing a moving object is selected from the image DB 120. Then, a moving locus of the moving object is generated using the tracking result. The movement locus is a line segment representing the past movement history of the moving object.

  The monitoring screen generation unit 126 uses the monitoring apparatus from the video data stored in the video DB 120, the video data showing the moving object selected by the tracking processing unit 124, and the trajectory data generated by the tracking processing unit 124. A monitoring screen to be provided to the user. A specific example of the monitoring screen will be described later.

  The input unit 128 is an input device such as a mouse or a keyboard, and accepts an operation of the monitoring device user such as “tracking start” and “tracking end”.

  The output unit 130 is an output device such as a display, and displays a monitoring screen generated by the monitoring screen generation unit 126 to the user.

  Next, a display example of the monitoring screen generated by the monitoring screen generation unit 126 and displayed on the output unit 130 will be described with reference to FIGS. 2 and 3. The monitoring screen is composed of a multiple video monitoring screen and a tracking screen. This will be described in order below.

  First, an example of a multiple video monitoring screen for browsing a plurality of videos will be described with reference to FIG.

  A map 200 on the left side of the screen is a map showing an outline of the entire monitoring area. This example shows that a three-story building is a monitoring target.

  The icon 202 indicates the geometric arrangement of the camera on the map 200. In this example, six cameras are arranged in the monitoring area.

  The video monitoring area 204 is an area for displaying video captured by the camera 106 (FIG. 1). In this example, images from four cameras such as the monitoring image 206 are displayed. When displaying images from five or more cameras, the display area can be changed with the scroll bar so that the number of divisions of the image monitoring area 204 is increased or a part of the image monitoring area 204 is displayed. do it. Conversely, when displaying images from three or less cameras, the number of divisions of the video monitoring area 204 may be reduced.

  With this multiple video monitoring screen, the user of the present apparatus can simultaneously browse a plurality of videos captured by the camera 106. When a suspicious person is confirmed on the screen, the tracking start can be instructed by clicking the monitoring video in the video monitoring area 204 with the mouse. By clicking the tracking end button 208 with the mouse, the currently running tracking can be stopped.

  For example, the icon 202 and the monitoring video 206 may be connected by a line segment as a link 210 so that the correspondence between the icon 202 and the video in the video monitoring area 204 becomes clear. Further, the name of the camera corresponding to the icon 202 may be displayed on the video in the video monitoring area 204.

  Next, an example of a tracking screen for monitoring a specific moving object will be described with reference to FIG. The configuration of the entire screen is basically the same as the multiple video monitoring screen of FIG. The only difference is that the locus 300 of the moving object is displayed on the map 200 and only the video of the camera tracking the tracking target is displayed in the monitoring area 204.

  The moving object trajectory 300 includes a thumbnail 302 and an arrow 306. The thumbnail 302 is image data obtained by reducing a video photographed by the camera. Image data captured by a camera whose tracking target corresponds to the icon 304 is used as a past history. On the other hand, an arrow 306 indicates that the thumbnail 302 has been moved to the thumbnail 308. The thumbnail 302 at the start of the trajectory indicates that tracking has started with the camera corresponding to this thumbnail, and the thumbnail 310 at the end of the trajectory indicates that the moving object is currently being tracked with the camera corresponding to this thumbnail. Yes. Here, the camera that is currently tracking the moving object is called a current camera.

  On the monitoring area 204, a monitoring video 206 is displayed as a video captured by the current camera. At this time, by highlighting the icon 202 corresponding to the current camera, the camera that captured the monitoring video 206 can be easily grasped. When the thumbnail 302 is clicked with a mouse, a monitoring video corresponding to the thumbnail may be displayed in the monitoring area 204.

  In this way, past representative images are displayed as thumbnails superimposed on the trajectory, so that the past situation of the moving object can be grasped only by looking at the moving trajectory. Although FIG. 3 shows an example of the tracking screen when the number of tracking is one, the number of tracking may be plural. When displaying a plurality of movement trajectories, the movement trajectories may be displayed with an offset in the right and downward directions to prevent the movement trajectories from overlapping each other and becoming difficult to see.

  Next, the basic principle of human object tracking will be described with reference to FIG. The left side is a monitoring video taken by the camera, and the right side is a graph of the feature amount stored in the feature amount DB 116 for the monitoring video. In addition, when graphing the feature values, not only the feature values for a specific monitoring video but also the feature values for several monitoring videos immediately before are displayed. Therefore, a plurality of feature amounts are plotted for one tracking target. This is to improve tracking accuracy in consideration of feature amount calculation errors. Note that the monitoring video acquisition times are t1, t2, and t3 (t1 <t2 <t3) from the top to the bottom. In FIG. 4, the feature amount graph represents a feature amount space including a plurality of types (here, two types) of feature amount 1 and feature amount 2.

  First, in the monitoring video 400 at time t1, a person 402 and a person 404 exist as moving objects, and correspond to the feature quantity 408 and the feature quantity 410 in the feature quantity space 406, respectively.

  Next, in the monitoring video 412 at time t2, a person 414 and a person 416 exist as moving objects, and correspond to the feature quantity 420 and the feature quantity 422 in the feature quantity space 418, respectively. Here, since the feature quantity 408 in the feature quantity space 406 and the feature quantity 420 in the feature quantity space 418 are located at the same place, the person 402 is likely to be the same as the person 414, and the monitoring video 400 changes to the monitoring video 412. It can be determined that it has moved.

  Here, for the equivalence determination between the feature amount space 408 and the feature amount 420, for example, the Euclidean distance between the respective centroids may be used. When this distance is smaller than a predetermined threshold value, it can be considered that both are sufficiently close and the same.

  In the monitoring video 424 at time t3, a person 426 exists as a moving object, and corresponds to the feature quantity 430 in the feature quantity space 428. Here, since the feature quantity 410 in the feature quantity space 406 and the feature quantity 430 in the feature quantity space 428 are located at the same place, the person 404 is likely to be the same as the person 426, and the monitoring video 400 changes to the monitoring video 424. It can be determined that it has moved.

  Thus, by searching for feature quantities having similar values in the feature quantity space, it becomes possible to track how a specific moving object has moved between cameras.

  Next, the flow of processing on the monitoring site 100 side of the moving object monitoring apparatus of the present embodiment will be described using the flowchart of FIG.

  In step 500, the processing from step 502 to step 508 is repeated at a predetermined frequency.

  In step 502, the video acquisition unit 108 acquires a monitoring video shot by the camera 106.

  In step 504, the moving object extraction unit 110 extracts the moving object in the video acquired in step 502.

  In step 506, the high frequency feature calculation unit 112 calculates the high frequency feature amount of the moving object extracted in step 504. At this time, the high frequency feature calculation unit 112 determines a high frequency feature to be calculated according to the processing assignment table 114.

  In step 508, the transmission video selection unit 118 selects a video to be transmitted from the videos acquired in step 502, and transmits the video to the video DB 120 for storage. At this time, the frame rate of the video to be transmitted to the monitoring center 102 side is determined from the processing allocation table 114, and the video to be transmitted is selected according to the frame rate.

  Next, the flow of processing on the monitoring center 102 side of the moving object monitoring apparatus of this embodiment will be described using a flowchart with reference to FIG.

  In step 600, the processing from step 602 to step 616 is repeated with a predetermined frequency.

  In step 602, a monitoring video to be processed is acquired from the video DB 120.

  In step 604, the low frequency feature calculation unit 122 uses the feature amount data stored in the feature amount DB 116 and the low frame rate video included in the video DB 120 to reduce the low frequency for the moving object stored in the feature amount DB 116. Calculate frequency characteristics. Here, the low-frequency feature is a feature that is not calculated by the high-frequency feature calculation unit 112 among features necessary for tracking a moving object.

  In step 606, the monitoring screen generation unit 126 processes the operation of the user of the monitoring apparatus. Here, the user's operation is input using the input means 128.

  In step 608, the processing from step 610 to step 616 is repeated for all traces currently being executed.

  In step 610, the time t to be processed is set to the current time. As a result, the tracking process in step 612 is performed based on the current time.

  In step 612, the tracking processing unit 124 performs tracking of the moving object with reference to the time t set in step 610. Details of the moving object tracking process will be described later.

  In step 614, it is determined whether or not a cross between the cameras has occurred as a result of the tracking of the moving object in step 612. If it has occurred, step 616 is executed. Here, the transition between the cameras means that the tracking target disappears from the camera originally shown and appears in another camera.

  In step 616, the tracking processing unit 124 generates a person's movement trajectory.

  In step 618, a monitoring screen is generated by the monitoring screen generation unit 126 and displayed on the output unit 130.

  Next, details of the input processing in step 606 in FIG. 6 will be described using the flowchart in FIG.

  In step 700, the operation of the user of the monitoring apparatus from the input unit 128 is accepted.

  In step 702, the input result in step 700 is determined. If “tracking start” is designated, steps 704 to 708 are executed. “Start tracking” is specified by, for example, clicking the monitoring video 206 of the multiple video monitoring screen in FIG. 2 with a mouse. On the other hand, when “end tracking” is designated, step 710 is executed. “Tracking end” is specified by, for example, clicking the tracking end button 208 of the multiple video monitoring screen in FIG. 2 with a mouse.

  In step 704, the feature quantity corresponding to the person shown in the camera designated in step 700 is stored as the trace target feature quantity.

  In step 706, the camera designated in step 700 is set as the current camera. As described above, the current camera is a camera that is currently tracking a moving object during tracking of the moving object.

  In step 708, a new trace is registered based on the information obtained in steps 704 and 706 to be added to the processing target in step 608 in FIG.

  In step 710, all currently running traces are stopped. As a result, there is no tracking to be processed in step 608 of FIG.

  Next, details of the moving object tracking process in step 612 of FIG. 6 will be described using the flowchart of FIG.

  In step 800, it is determined whether or not a tracking person exists in the video of the current camera. The tracked person is a person corresponding to the movement locus designated as the processing target in step 608 of FIG. In addition, although expressed as a person here, it shall include the whole moving objects, such as a vehicle. Whether or not a tracking person exists in the current camera is determined in the feature amount space as shown in FIG. If the tracking target feature quantity does not exist at the processing target time t in the feature quantity space corresponding to the current camera image, it is determined that there is no tracking person in the current camera, and steps 802 to 806 are executed.

  In step 802, it is searched whether there is a tracking person other than the current camera. Details of this search processing will be described later.

  In step 804, the result of the tracking person search process in step 802 is determined. If there is a tracking person in another camera, step 806 is executed.

  In step 806, based on the search result in step 802, the camera in which the tracking person exists is newly set as the current camera.

  In step 808, it is determined whether a predetermined time specified in advance has elapsed since the start of the tracking process. This is a process for aborting the tracking process in time. If necessary, it is possible to avoid the termination of the tracking process by specifying infinity at a predetermined time. If the result of determination is that a predetermined time has elapsed, step 810 is executed.

  In step 810, a stop process for stopping the tracking of the current processing target is executed.

  Next, details of the tracking person search processing in step 802 of FIG. 8 will be described using the flowchart of FIG.

  In step 900, steps 902 to 908 are repeatedly executed for all the cameras of the monitoring apparatus of the present embodiment.

  In step 902, it is determined whether or not the camera to be processed in step 900 is the same as the current camera. If they are the same, step 904 is executed.

  In step 904, the current loop processing in step 900 is skipped to proceed to the next loop processing in order to improve processing efficiency. This is because it is determined in step 800 of FIG. 8 that there is no person to be tracked in the current camera.

  In step 906, it is determined whether there is a person to be tracked in the camera that has been processed in step 900. Whether or not a tracking person exists in the camera to be processed is determined in the feature space as shown in FIG. As a result of the determination, if it exists, step 908 is executed.

  In step 908, the camera to be processed is returned as a return value, and the process is terminated. On the call side of this flow, it can be seen that camera crossing occurs and that there is a tracking target in the current processing target camera.

  In step 910, NULL meaning invalid is returned as a return value, and the process is terminated. It can be seen that there is no camera crossing on the call side of this flow.

  Next, an example of the data structure of the processing assignment table 114 in FIG. 1 will be described with reference to FIG. This table is composed of three columns of elements of feature quantity type 1000, required video rate 1002, and processing sharing 1004.

  The feature quantity type 1000 represents the type of feature quantity used in the moving object monitoring apparatus of the present invention. For example, the cell 1006 represents information related to the feature amount A.

  The necessary video rate 1002 represents the frame rate of the video used for calculating the feature amount. For example, the cell 1008 indicates that an image having a frame rate of 30 frames / second is used to calculate the feature amount A. In the present embodiment, when the processing allocation table 114 is created, the rows of this table are arranged in descending order from the highest required video rate 1002 value.

  The process sharing 1004 indicates whether the feature amount calculation process is executed at the monitoring site 100 or the monitoring center 102. A cell 1010 represents that the calculation process of the feature amount A is executed on the monitoring site 100 side.

  The high-frequency feature boundary value 1012 is a value indicating the range of the high-frequency feature, and sets the value of the processing share 1004 (monitoring site or monitoring center). In the example of FIG. 10, the value is 2, indicating that the first and second feature quantities, that is, feature quantities A and B, are high-frequency features, and the third and later, that is, feature quantities C and D are low-frequency features. Thereby, it can be determined that the calculation processing of the feature amounts A and B is executed at the monitoring site 100, and the calculation processing of the feature amounts C and D is executed at the monitoring center 102, and the value of the processing sharing 1004 is determined. In the present embodiment, the video frame rate of the high frequency feature is set higher than that of the low frequency feature.

  With this processing allocation table 114, the high frequency feature calculation unit 112 can set a feature amount to be calculated. In the example of FIG. 10, the feature amounts A and B are determined and calculated as calculation targets. The remaining feature amounts C and D are calculated by the low frequency feature calculation unit 122.

  In addition, the transmission video selection unit 118 can determine the frame rate of the video to be transmitted to the monitoring center 102 by using the processing allocation table 114. In the example of FIG. 10, the features to be calculated on the monitoring center 102 side are feature amounts C and D, and the frame rate required to calculate both is 5 frames / second. The transmission video selection unit 118 transmits video at this frame rate.

  Next, an example of the data structure of the feature amount stored in the feature amount DB 116 of FIG. 1 will be described using FIG. The feature amount data includes a table 1110 representing the entire feature amount data and a table 1118 representing the feature amount vector.

  A table 1100 is a table representing one feature amount data. This table includes data items 1102 to 1117.

  The data item 1102 is an ID that uniquely represents the camera. The feature amount is calculated from the video imaged by the camera corresponding to the ID.

  The data item 1104 is the acquisition time of the video for which the feature amount has been calculated.

  A data item 1106 is an ID of a moving object. This ID is determined in the process of tracking a person in one camera. To the last, it is uniquely determined within one camera, and is not uniquely determined for the same person among a plurality of cameras.

  The data item 1108 represents the position of the moving object on the image. This position information is calculated by the moving object extraction unit 110.

  The data item 1110 represents the size of the moving object on the image. As the size, for example, a circumscribed rectangle for a moving object may be considered. This size information is calculated by the moving object extraction unit 110.

  Data items 1112 to 1117 are pointers to areas for storing feature quantity vectors calculated for moving objects. The feature amount data corresponding to the feature amounts A, B, C, and D are stored. Here, the data item 1112 indicates the table 1118 as the data of the feature amount A. On the other hand, the data item 1114 indicates NULL having no value as the data of the feature amount B. That is, it indicates that the data of the feature amount B has not been calculated yet. Similarly, the data items 1116 and 1117 are NULL, indicating that the feature amounts C and D have not yet been calculated.

  On the other hand, the table 1118 representing the feature vector is composed of data items 1120 and variable-length data items 1122.

  Data item 1120 is the order of the vector. The order of the feature vector is not uniquely determined because various features such as color, texture, shape, and motion can be considered as the feature. In the present embodiment, the order of the feature quantity is held in the data item 1120 so that the feature quantity to be adopted can be freely changed. For example, when the mode luminance values of the lower body and the upper body are used as the color characteristics of the moving object, the order is 2.

  The data item 1122 is an element of the feature quantity vector, and there are as many items as the number of orders specified in the data item 1120.

  According to the embodiment described above, the center side can use high-frequency characteristics without increasing the video data transmitted through the network. This makes it possible to track a moving object with high accuracy.

(Embodiment 2)
In the first embodiment, the configuration of the apparatus when there is one monitoring site is shown, but there may be a plurality of monitoring sites. An embodiment in this case is shown below. Since the configuration is basically the same as that of the first embodiment, only the changes will be described.

  FIG. 12 is a block diagram illustrating a functional configuration of the entire moving object monitoring apparatus including a plurality of monitoring sites. A monitoring site 1200 is newly added to the moving object monitoring apparatus shown in FIG.

  The monitoring site 1200 is a monitoring target different from the monitoring site 100. The functional configuration of the monitoring site 1200 is the same as the functional configuration of the monitoring site 100. The high-frequency features of the video calculated by the high-frequency feature calculators 112 of the monitoring site 100 and the monitoring site 1200 are stored in the feature amount DB 116 of the monitoring center 102. Similarly, the images transmitted by the transmission image selection units 118 of the monitoring site 100 and the monitoring site 1200 are stored in the image DB 120 of the monitoring center 102. On the monitoring center 102 side, processing is performed using the feature amount DB 116 and the image DB 120 in which information on a plurality of monitoring sites is stored.

  According to the embodiment described above, information on a plurality of monitoring sites can be collected in the monitoring center. Therefore, a plurality of monitoring sites can be monitored at the monitoring center.

  In the embodiment described above, the same processing allocation table 114 is used at a plurality of monitoring sites, but the contents of the processing allocation table 114 may be changed for each monitoring site. In this way, flexible customization according to the monitoring site becomes possible.

(Embodiment 3)
In the first and second embodiments, the sharing of processing is statically determined using the processing sharing table 114, but may be dynamically determined according to the operating status of the apparatus. An embodiment in this case is shown below.

  FIG. 13 is a block diagram illustrating a functional configuration of the entire moving object monitoring apparatus that dynamically changes the processing sharing. A network load measuring unit 1300, a processing load measuring unit 1302, and an on-site video DB 1304 are newly added to the moving object monitoring apparatus shown in FIG.

  The network load measuring unit 1300 measures the load factor of the network 104 and reflects the result in the processing assignment table 114 as described later. This load factor may be obtained, for example, by dividing the measured value of the amount of data actually transmitted through the network 104 by the transmission capacity of the network 104.

  The processing load measuring unit 1302 measures the processing load rate of the apparatus on the monitoring site 100 side, and reflects the result in the processing sharing table 114 as described later. In the figure, an example is shown in which the processing load of the moving object extraction unit 110 and the high frequency feature calculation unit 112 is measured. This load factor is represented, for example, by a ratio of time during which each device is not in a standby state, that is, some processing time, in a certain unit time.

  The on-site video DB 1304 is a database that temporarily saves video when processing cannot be continued under the current network load and processing load of each device. Details will be described later.

  FIG. 14 is an example of a data structure of a process sharing table used for dynamic change of process sharing. This is obtained by adding a lower limit value 1400 and an upper limit value 1402 to the data structure of the processing assignment table shown in FIG.

  The lower limit value 1400 indicates the lower limit value of the range that the high frequency feature boundary value 1012 can take. As will be described later, the lower limit value 1400 varies according to the load factor measured by the network load measuring unit 1300.

  The upper limit value 1402 indicates an upper limit value of a range that the high frequency feature boundary value 1012 can take. As will be described later, the upper limit value 1402 varies according to the load factor measured by the processing load measurement unit 1302.

  From the above, the range that the high frequency characteristic boundary value 1012 can take is determined to be the lower limit value 1400 or more and the upper limit value 1402 or less.

  FIG. 15 is a flowchart showing a flow of processing for dynamically changing processing sharing.

  In step 1500, the network load measuring unit 1300 measures the load factor NL of the network 104.

  In step 1502, the load factor NL measured in step 1500 is compared with the threshold value TNL1, and if the load factor NL exceeds the threshold value TNL1, step 1504 is executed. Here, the threshold value TNL1 is a value close to 100%, for example, 90%. When the load factor NL exceeds the threshold value TNL1, it means that the load factor of the network is high and close to the limit of the transmission capacity.

  In step 1504, 1 is added to the lower limit value 1400 of FIG. Here, max () is a function for obtaining the maximum value among the arguments. This function is used so that the value is larger than the number of feature quantity types and an invalid value is not taken. By adding 1 to the lower limit value 1400, the effective range of the high frequency feature boundary value 1012 is narrowed in a direction in which the feature amount calculated on the monitoring site 100 side increases and the load factor of the network 104 decreases.

  In step 1506, the load factor NL measured in step 1500 is compared with the threshold value TNL2. If the load factor NL is lower than the threshold value TNL2, step 1508 is executed. Here, the threshold value TNL2 is sufficiently smaller than 100%, for example, 50%. When the load factor NL is lower than the threshold value TNL2, it means that the load factor of the network is low and there is a margin in transmission capacity.

  In step 1508, 1 is subtracted from the lower limit value 1400 in FIG. Here, min () is a function for obtaining the minimum value among the arguments. This function is used so that the lower limit value 1400 does not take an invalid value. By subtracting 1 to the lower limit value 1400, the effective range of the high-frequency feature boundary value 1012 decreases in the amount of feature calculated on the monitoring site 100 side, and widens the load factor of the network 104.

  In step 1510, the processing load measurement unit 1302 measures the processing load factor PL of the monitoring site 100 side device, for example, the moving object extraction unit 110 and the high frequency feature calculation unit 112.

  In step 1512, the load factor PL measured in step 1510 is compared with the threshold value TPL1, and if the load factor PL exceeds the threshold value TPL1, step 1514 is executed. Here, the threshold value TPL1 is a value close to 100%, for example, 90%. When the load factor PL exceeds the threshold value TPL1, it means that the load factor of processing of each device is high and close to the limit.

  In step 1514, 1 is subtracted from the upper limit value 1402 in FIG. As in step 1508, min () is used so as not to take an invalid upper limit value. By subtracting 1 from the upper limit value 1402, the effective range of the high frequency feature boundary value 1012 is reduced in the direction in which the feature amount calculated on the monitoring site 100 side is reduced and the processing load factor is reduced.

  In step 1516, the load factor PL measured in step 1510 is compared with the threshold value TPL2. If the load factor PL is lower than the threshold value TPL2, step 1518 is executed. Here, the threshold value TPL2 is sufficiently smaller than 100%, for example, 50%. When the load factor PL is lower than the threshold value TPL2, it means that the processing load factor of each device is low and there is a margin in processing.

  In step 1518, 1 is added to the upper limit value 1402 in FIG. As in step 1504, max () is used so as not to take an invalid value larger than the number of feature quantity types. By adding 1 to the upper limit value 1402, the effective range of the high-frequency feature boundary value 1012 increases in the direction in which the feature amount calculated on the monitoring site 100 side increases and the load factor of the apparatus increases.

  In step 1520, the lower limit value and the upper limit value calculated in the processing so far are compared. When the lower limit value is equal to or lower than the upper limit value, it means that there is an appropriate high-frequency feature boundary value 1012 corresponding to the current network load factor and the load factor of processing of each device. In this case, step 1522 is executed. When the lower limit value is larger than the upper limit value, it means that there is no appropriate high-frequency feature boundary value 1012 according to the current network load factor and the load factor of processing of each device. In this case, step 1524 is executed to save the video as an exception process.

  In step 1522, an appropriate high-frequency feature boundary value 1012 corresponding to the current network load factor and the load factor of processing of each device is determined. The high frequency characteristic boundary value 1012 is in the range of the lower limit value 1400 or more and the upper limit value 1402 or less. An appropriate value within this range may be adopted as the high frequency feature boundary value 1012. For example, a lower limit value 1400 or an upper limit value 1402 may be adopted. Moreover, you may employ | adopt the intermediate value of both.

  In step 1524, it is considered that the processing cannot be continued under the current network load and the processing load of each device, and the processing is interrupted and the video is saved. Specifically, the feature amount calculation processing of the high-frequency feature calculation 112 and the video transmission processing of the transmission video selection unit 118 are interrupted, and the video is saved in the on-site video DB 1304 instead. The saved video may be processed after the network load and the processing load of each device are reduced, for example.

  According to the embodiment described above, it is possible to dynamically determine an appropriate sharing of processing according to the load status of the network and the load status of the processing of each device. As a result, it is possible to flexibly deal with load fluctuations due to external factors.

  Although FIG. 13 shows an example in which the network load measuring unit 1300 and the processing load measuring unit 1302 are installed on the monitoring site 100, these devices may be installed on the monitoring center 102 side. FIG. 16 shows a configuration when the processing load measuring unit 1302 is installed on the monitoring center 102 side. Instead of the processing load measuring unit 1302, a center processing load measuring unit 1600 is added to the monitoring center 102 side.

  The center processing load measurement unit 1600 measures the processing load rate of the device on the monitoring center 102 side, and reflects the result on the processing sharing table 114, as with the processing load measurement unit 1302 described above. In the figure, an example of measuring the processing load of the low frequency feature calculation unit 122 and the tracking processing unit 124 is shown. In addition, when there are a plurality of monitoring sites, finer control is possible. For example, instead of uniformly reflecting them in the processing assignment table 114 of all the monitoring sites, the monitoring sites may be sequentially changed with a time difference.

  According to the embodiment described above, it is possible to dynamically determine an appropriate sharing of processing according to the processing load state of the monitoring center as well as the monitoring site. As a result, it becomes possible to flexibly cope with load fluctuations caused by external factors.

(Embodiment 4)
In the first to third embodiments, an example has been shown in which a moving object is tracked using a feature amount that is shared and calculated between the monitoring site side and the monitoring center side, but this feature amount may be used for video search. good. An embodiment in this case is shown below. Basically, since most of the configuration is the same as that of the first embodiment, a portion where there is a change will be described.

  FIG. 17 is a block diagram showing a functional configuration of the entire monitoring apparatus having a video search function. Except for the video search unit 1700 and the search screen generation unit 1702, it is the same as that shown in FIG.

  The video search unit 1700 searches for a video similar to the video specified by the user based on the feature amount data stored in the feature amount DB 116. Here, for example, a feature vector may be used for the search. The feature amount vector is obtained by connecting various feature amounts of the feature amount A to the feature amount D shown in FIG. Then, the length of the difference vector between the feature quantity vector of the search target video and the feature quantity vector stored in the feature quantity DB 116 is calculated as the similarity. When the similarity is small, it can be considered that the images are similar.

  The search image generation unit 1702 generates a monitoring screen to be provided to the user of this monitoring apparatus based on the search result of the video search unit 1700.

  FIG. 18 is an example of a search screen generated by the search image generation unit 1702. The search screen includes a query condition setting area 1800, a search query designation area 1802, a search execution button 1806, and a search result display area 1808.

  The query condition setting area 1800 is an area in which conditions are set in order to narrow down videos that are search query candidates. For example, the conditions are specified by camera ID and date / time. Here, the camera ID is an ID for identifying a camera set at the monitoring site.

  The search query designation area 1802 displays a list of videos close to the conditions designated in the query condition setting area 1800.

  A search query 1804 is a video selected by the user of this apparatus from the videos displayed in the search query designation area 1802. This video is used for inquiry of similar images to the search processing system. In the figure, the outer frame of the search query 1804 is displayed thick, indicating that it is in a selected state.

  The search execution button 1806 is a button for instructing start of search execution. When the user of this apparatus presses this button, video search using the search query 1804 is started.

  A search result display area 1808 is an area in which search results are displayed. As a result of the search, videos similar to the search query 1804 are displayed in the similar order.

  According to the embodiment described above, the center side can use high-frequency characteristics without increasing the video data transmitted through the network. Thereby, a video search with high accuracy becomes possible.

(Embodiment 5)
In the first to third embodiments, an example has been shown in which a moving object is tracked using a feature amount that is shared and calculated between the monitoring site side and the monitoring center side, but this feature amount is detected as an abnormal action in the video. You may use for. An embodiment in this case is shown below. Basically, since most of the configuration is the same as that of the first embodiment, a portion where there is a change will be described.

  FIG. 19 is a block diagram illustrating a functional configuration of the entire monitoring apparatus having an abnormal behavior detection function. Except for the abnormal behavior detection unit 1900 and the abnormal display screen generation unit 1902, the configuration is the same as that shown in FIG. 1.

  The abnormal behavior detection unit 1900 detects abnormal behavior using the feature amount data stored in the feature amount DB 116 and sends the result to the abnormal display screen generation unit 1902. Here, as an abnormal behavior detection method, for example, a known abnormal behavior determination method is used in which a feature vector of a normal behavior video is learned and a video deviating from the normal behavior characteristics is determined as abnormal. be able to.

  The abnormal display screen generation unit 1902 generates a monitoring screen to be provided to the user of this monitoring apparatus based on the detection result of the abnormal behavior detection unit 1900.

  FIG. 20 is an example of a monitoring screen generated by the abnormality display screen generation unit 1902. This screen is almost the same as the screen shown in FIG. 2, but an abnormality occurrence image 2000 is added.

  The abnormality occurrence image 2000 is an image determined to be abnormal by the abnormal behavior detection unit 1900, and the outer frame of the image is highlighted to make it easy for the user of this apparatus to visually recognize. In addition to highlighting the outer frame of the video, the video may be displayed in another new window, or a warning may be issued by voice to alert the user of the apparatus.

  According to the embodiment described above, the center side can use high-frequency characteristics without increasing the video data transmitted through the network. Thereby, it is possible to detect abnormal behavior with high accuracy.

It is a block diagram showing the function structure of the moving object monitoring apparatus by this invention. It is a figure explaining the example of a multiple image | video monitoring screen. It is a figure explaining the example of a tracking monitoring screen. It is a figure explaining the basic principle of moving object tracking. It is a flowchart showing the processing flow on the monitoring site side. It is a flowchart showing the processing flow by the side of a monitoring center. It is a flowchart showing the process flow of an input process. It is a flowchart showing the processing flow of tracking of a moving object. It is a flowchart showing the flow of the search process of a tracking person. It is a figure explaining the data structure of a process sharing table. It is a figure explaining the structure of feature-value data. It is a block diagram showing the function structure of the whole moving object monitoring apparatus provided with the some monitoring field. It is a block diagram which shows the function structure of the whole moving object monitoring apparatus which changes a process share dynamically. It is a figure explaining the data structure of the processing allocation table used for the dynamic change of processing allocation. It is a flowchart showing the flow of a process of the dynamic change of a process sharing. It is a block diagram showing the functional structure of the whole moving object monitoring apparatus provided with the processing load measurement part in the center side. It is a block diagram which shows the function structure of the whole monitoring apparatus provided with an image | video search function. It is a figure explaining the example of a search screen. It is a block diagram which shows the function structure of the whole monitoring apparatus provided with an abnormal action detection function. It is a figure explaining the example of a search screen.

Explanation of symbols

100 Monitoring Site 102 Monitoring Center 104 Network 106 Camera 108 Video Acquisition Unit 110 Moving Object Extraction Unit 112 High Frequency Feature Calculation Unit 114 Processing Sharing Table 116 Feature DB
118 Transmission Video Selection Unit 120 Video DB
122 Low-frequency feature calculator 124 Tracking processor 126 Monitor screen generator 128 Input means 130 Output means

Claims (11)

In a moving object monitoring apparatus that monitors a moving object using a plurality of cameras,
The moving object monitoring device comprises at least one monitoring site and a monitoring center connected by a network,
The monitoring site is
A video acquisition unit that acquires video captured by the camera;
A moving object extraction unit for extracting a moving object in the acquired video;
A process assignment table for determining the share of the feature value calculation process for the extracted moving object;
A high-frequency feature calculation unit that calculates a high-frequency feature amount of the moving object according to the processing assignment table;
Determining a transmission rate of video to be transmitted to the monitoring center according to the processing allocation table, and a video selection unit for transmitting the video at the rate,
The monitoring center is
A video database for storing videos transmitted by the transmission video selection unit;
A feature quantity database for storing the feature quantities calculated by the high-frequency feature calculation section;
A moving object monitoring apparatus comprising: a low-frequency feature calculation unit that calculates an uncalculated feature in the feature amount database. The moving object monitoring apparatus according to claim 1,
The process sharing table includes a feature sharing type representing a feature quantity type, a required video rate representing a frame rate of a video necessary for calculating the feature quantity, and a process sharing representing a place where the feature quantity calculation process is executed. Having at least one combination and a high frequency feature boundary value indicating a range of the high frequency feature,
The moving object monitoring apparatus, wherein the processing share value is determined based on the high-frequency feature boundary value. The moving object monitoring apparatus according to claim 2,
The processing allocation table is sorted according to descending or ascending order based on the required video rate. In the moving object monitoring device according to any one of claims 1 to 3,
The processing allocation table includes an upper limit value and a lower limit value for the high-frequency feature boundary value,
The moving object monitoring apparatus, wherein the high-frequency characteristic boundary value can be changed within a range between the upper limit value and the lower limit value. In the moving object monitoring device according to any one of claims 1 to 4,
It has a network load measurement unit that measures the load factor of the network,
The network load measuring unit updates the high-frequency feature boundary value of the processing assignment table based on the measured load factor. The moving object monitoring apparatus according to claim 5,
The moving object monitoring apparatus, wherein the network load measuring unit is installed in at least one of the monitoring site and the monitoring center. In the moving object monitoring device according to any one of claims 1 to 4,
A processing load measuring unit for measuring the processing load factor of the apparatus is provided.
The moving load monitoring device, wherein the processing load measuring unit updates the high frequency characteristic boundary value of the processing sharing table based on the measured load factor. The moving object monitoring apparatus according to claim 7, wherein
The moving object monitoring apparatus, wherein the processing load measuring unit is installed in at least one of the monitoring site and the monitoring center. In the moving object monitoring device according to any one of claims 1 to 8,
A moving object monitoring apparatus comprising: a moving object tracking unit that tracks the moving object based on data stored in the feature amount database. In the moving object monitoring device according to any one of claims 1 to 8,
A moving object monitoring apparatus comprising: a video search unit that searches for video based on data stored in the feature database. In the moving object monitoring device according to any one of claims 1 to 8,
A moving object monitoring apparatus comprising: an abnormal action detection unit that detects an abnormal action of the moving object based on data stored in the feature amount database.

Images