JP2013092955A - Video analysis device and system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video analysis device for quickly analyzing a scene in which various dynamic objects appear.SOLUTION: The video analysis device includes: a dynamic object area detection part 103 for detecting an area in which a dynamic object exists from a frame image 102 of an input video; and an object category determination part 104 for determining the category of the object detected by the dynamic object area detection part. The video analysis device generates an "existence probability map" showing probability that an object appears in each coordinate in a video from the object area and the category determined by the object category determination part, and stores the "existence probability map" as time series data to generate the existence probability map of each category in a fixed time span.

Description

  The present invention relates to a video analysis apparatus and system for analyzing a dynamic object in a video.

  In a video surveillance system, a function for detecting a dynamic object in a video and analyzing / searching the detected object has been put into practical use. For example, there is a video surveillance system that has the function of notifying a user when a suspicious person or an important person is captured by detecting the face area of a person in the video and comparing it with face images accumulated in the past To do.

  Currently, the detection target object is most commonly a human face, but in the future, for example, it is desired to include various dynamic objects such as vehicles and various articles as detection / analysis targets. Here, in order to reduce the processing, it is necessary to improve the analysis processing efficiency.

  Regarding improvement in analysis processing efficiency, for example, Patent Document 1 discloses a means for limiting an area in which image processing for detecting an object area is performed using the existence probability of an object. The method of Patent Document 1 is to determine a region for image processing using static information of an imaging system such as a focal length and resolution, and a photographing environment and photographing equipment are limited like an in-vehicle camera. It is effective in other environments.

  On the other hand, as a scene analysis method other than image matching, in Patent Document 2, a scene in which a person corresponding to condition data representing a specific person action appears is searched by storing a person's flow line in a database. Describes how to do. The search based on the flow line is effective for performing a scene search focusing on the motion of one object.

JP 2010-003254 A JP2009-284167

  However, the method of Patent Document 1 cannot be applied in an environment (uncontrolled environment) where the shooting situation and the position of the subject in the video cannot be predicted in advance, like a surveillance camera.

  Patent Document 2 is effective for detecting a specific action such as an illegal action, but does not analyze a scene in which various dynamic objects appear at high speed.

  The video analysis device according to the present invention is detected by a dynamic object region detection unit that detects a region where a dynamic object exists from a frame image of an input video, and the detected dynamic object region detection unit. An object category discriminating unit for discriminating the category of the detected object, and for each category discriminated by the object category discriminating unit, a probability that an object appears in the detected dynamic object region (coordinate or field) An “existence probability map” is generated and saved as time series data, thereby generating an existence probability map of each category in a certain time span.

  With the above configuration, an object existence probability map in the video space can be obtained for each object category. As a result, it is possible to speed up the object detection by using the existence probability map to limit the execution area of the image recognition process in the object detection process.

1 is a functional block diagram of a video analysis system 100 according to the present invention. It is a figure for demonstrating the production | generation of the existence probability map in the video analysis system 100 by this invention. It is a flowchart showing the production | generation process procedure of the existence probability map in the video analysis system 100 by this invention. It is a figure for demonstrating an example of the object detection of multiple categories. It is a flowchart showing the process sequence of an example of the object detection of multiple categories. It is a flowchart showing the process sequence which calculates the reliability of an object detection. It is a figure for demonstrating speeding-up of the object detection using a presence probability map. It is a flowchart showing the process sequence of the acceleration of the object detection using an existence probability map. It is a figure for demonstrating the abnormal scene detection using an existence probability map. It is a flowchart showing the process sequence of the abnormal scene detection using an existence probability map. It is a figure for demonstrating the similar scene search using an existence probability map. It is a figure which shows the structure and data example of the video database for a similar scene search. The figure which shows the structure of a similar scene search system. The figure which shows the sequence of a similar scene search. The figure explaining the Example of the application to marketing. The figure explaining the Example of the wide area existence probability map production | generation using a PTZ camera. The figure explaining the Example of the PTZ control of the camera using the wide area presence probability map for every time.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is an example of an overall configuration diagram of the video analysis system 100. The video analysis system 100 includes a video input device 101, an operation information input device 107, a display device 110, and a computer 111. The computer 111 includes a video input unit 102, a dynamic object region detection unit 103, an object category determination unit 104, an existence probability calculation unit 105, an existence probability accumulation unit 106, an operation information input unit 108, and an existence probability output unit 109. It can be implemented on a general-purpose computer. The video input device 101 is an input interface for capturing external video into the system, such as a video playback device or a camera.

  The video input unit 102 receives video data from the video input device 101, converts it into a frame image (still image), and outputs it to the dynamic object region detection unit 103.

  The dynamic object region detection unit 103 receives the frame image from the video input unit 102, and specifies the coordinates of the region where the dynamic object is reflected in the frame image by image recognition processing. The dynamic object means an object that can move by itself, and examples thereof include animals, cars, and bicycles. In this embodiment, the dynamic object region detection unit 103 can detect a plurality of types (categories) of objects. The dynamic object region detection unit 103 outputs the coordinate information of the detected object region and the reliability of the detection result. For example, the object area is derived as rectangular data, and the coordinate information is in the form of [horizontal coordinates of the upper left corner of the rectangle, vertical coordinates of the upper left corner of the rectangle, horizontal coordinates of the lower right corner of the rectangle, vertical coordinates of the lower right corner of the rectangle] Is output. The reliability of the detection result is a value representing the “object likeness” of the rectangular image.

  The object category discriminating unit 104 specifies a semantic category of an object shown in the area detected by the dynamic object detecting unit 103. The category is, for example, “person” or “car”. The category granularity can be freely changed according to the purpose of use and discrimination accuracy. For example, the “person” category may be further classified, and lower categories such as “male” and “female” may be output. .

  The existence probability calculation unit 105 generates an existence probability map for each category. First, the existence probability calculation unit 105 derives a single frame existence probability map using the rectangular data and reliability detected by the dynamic object detection unit 103 and the category obtained by the object category determination unit 104. . The existence probability map is a table-like data structure corresponding to the XY coordinates of the camera image, and the existence probability at each coordinate is calculated based on, for example, the reliability of the detection area. The existence probability map of a single frame is stored in the existence probability accumulation unit 106 as time series data. Next, an existence probability map of a certain time span is read from the existence probability accumulation unit 106 and aggregated, thereby generating an existence probability map in that time span and outputting it to the existence probability output unit 109.

  The existence probability accumulation unit 106 stores the existence probability map for each category calculated by the existence probability calculation unit 105 as time series data.

  The operation information input device 107 is an input interface for transmitting user operations to the system, such as a mouse, a keyboard, and a touch device.

  The operation information input unit 108 transmits the user operation information input from the operation information input device 107 to the existence probability calculation unit 105. For example, an instruction for initializing the existence probability held in the existence probability accumulation unit 106 or designating a time span for accumulating the existence probability is transmitted to the system.

  The existence probability output unit 109 outputs the existence probability map data calculated by the existence probability calculation unit 105 to the display device 110 as an analysis result.

  The display device 110 is an output interface such as a liquid crystal display or a CRT, and displays the analysis result received from the existence probability output unit 109 on the screen.

  FIG. 2 is a diagram illustrating a state in which the existence probability calculation unit 105 generates an existence probability map, and represents time-series processing from left to right. Reference numerals 201, 202, and 203 are frame images extracted by the video input unit 102. A thick rectangle in the image represents an object region detected by the dynamic object region detection unit 103. In the frame image 201, two types of objects of “car” and “person” are detected. Based on this result, existence probability maps 204 and 207 are calculated, respectively. The value of each coordinate in the existence probability map is a value proportional to the reliability of the corresponding object region. An existence probability map is similarly obtained for the next frame image 202 and is added together with the existence probability map generated by the frame image 201. As a totaling method, for example, there is a method of using the maximum value or the average value of the reliability of each coordinate in the time series data. As a result, an existence probability map such as 205 or 208 in a time span of two frames is generated. By repeating the above processing up to the frame image 203, an existence probability map such as 206 or 209 is finally generated. In this way, the data itself is managed by coordinates that are “points”, and is managed as “places” in units of areas at the time of update or use.

  FIG. 3 is a flowchart for explaining processing in which the video analysis system 100 outputs an existence probability map. Hereinafter, each step of FIG. 3 will be described.

  In step S301, the video input unit 102 acquires a frame image (still image) from the input image. Frame images are repeatedly acquired at regular time intervals. Here, the input video does not necessarily need to be a video from a fixed camera, and a free viewpoint camera may be used as long as the video coordinates correspond to the coordinates of the existence probability map. Hereinafter, for the sake of brevity, a case where a fixed camera is used will be described.

  In step S302, the dynamic object region detection unit 103 performs image recognition processing on the frame image acquired in step S301 to obtain coordinate data of the dynamic object region. In the detection process, it is assumed that a reliability indicating “object-likeness” is given to each dynamic object region. There are various methods for performing object detection in a plurality of categories. As an example, a method based on a matching method using a plurality of templates will be described later with reference to FIG.

  In step S303, the existence probability map relating to the processing target frame image is initialized. Here, the same number of existence probability maps as the number of categories discriminable by the object category discrimination unit 104 are created.

  Steps S <b> 304 to S <b> 307 are repetitive processing for all regions detected by the dynamic object region detection unit 103.

  In step S <b> 305, the category of the detected object is determined by the object category determination unit 104. As the determination method, for example, when a collation method using a plurality of templates is used in step S302, a template category that matches the target object may be output.

  In step S306, the existence probability map for the category determined in step S305 is updated. In this step, a value proportional to the reliability of the region is set as the coordinate value corresponding to the object region. If it has already been set, use the larger value.

  In step S <b> 308, the existence probability map in a single frame of each category is stored in the existence probability accumulation unit 106.

  In step S309, an existence probability map in a certain time span is read from the existence probability accumulation unit 106 and aggregated for each category. As the counting method, as described in the explanation of FIG. 2, the maximum value or the average value is used.

  In step S310, the existence probability map in the fixed time span obtained in step S309 is output to the existence probability output unit 109.

  In step S311, if all the frame images have been cut out in the video input unit 102, the process flow ends. If a frame image that has not been processed remains, the process returns to step S301 to process the next frame image.

  The existence probability map generation processing in the video analysis system of the present invention has been described above. Next, an example of a method for detecting a plurality of categories of dynamic object regions in step S302 in FIG. 3 will be described with reference to FIG. The method described below is an object detection method based on a matching method for a plurality of templates.

  In the method shown in FIG. 4, an image feature amount of a typical image (template) of an object to be detected is extracted in advance and stored in a template database. The image feature amount is obtained by quantifying features of the image itself such as color features and shape features, and is given as, for example, fixed-length vector data.

  When the input image 401 is given, first, the position and size of the scanning window 402 are mechanically changed to extract an object candidate region. Next, the nearest neighbor template in the feature vector space is searched from a plurality of templates prepared in advance for all candidate regions. If the distance between vectors with the nearest neighbor template is less than or equal to the threshold, it is determined that the object is an object, and the candidate area is added to the detection result. At this time, the distance from the nearest template can be used as the reliability of the detection result.

  A processing procedure of an example of object detection of a plurality of categories will be described using the flowchart of FIG.

  In step S501, candidate areas are extracted from the input image. Candidate areas are mechanically extracted by moving and resizing the scanning window at fixed steps. The processing from step S502 to step S506 is performed for all candidate regions.

  In step S503, the reliability of the candidate area is calculated. As a reliability calculation method, as described in FIG. 4, there is a method of using feature amount-based collation with a plurality of templates. The processing procedure will be described in detail with reference to the flowchart of FIG. As another method, a method using a machine learning-based classifier is also known. In this case, it is necessary to prepare as many classifiers as the number of categories handled by the system.

  In step S504, if the reliability of the candidate area calculated | required by step S503 is below a threshold value, it will move to step S505 and will skip step S505 otherwise.

  In step S505, the candidate area being processed is added to the detection result list.

  When the processes in steps S502 to S506 have been completed for all candidate areas, step S507 outputs a detection result list, and the process flow ends. The detection result is a combination of area coordinate information (for example, [horizontal coordinates of the upper left corner of the rectangle, vertical coordinates of the upper left corner of the rectangle, horizontal coordinates of the lower right corner of the rectangle, vertical coordinates of the lower right corner of the rectangle]) and reliability. Is output as

  FIG. 6 is a flowchart showing a processing procedure for calculating the reliability of the candidate area in step S503 of FIG. FIG. 6 shows a reliability calculation method based on the matching method with a plurality of templates described in FIG.

  In step S601, the state is initialized with the nearest neighbor template T = null and the distance d = 0 from the nearest neighbor template.

  In step S602, the image feature amount of the candidate area is extracted. The image feature amount is extracted by the same method as the template database described in FIG.

  Next, the processes in steps S603 to S607 are performed on all templates in the template database.

  In step S604, a feature vector distance d 'between the input image and the processing target template T' is obtained.

  In step S605, if the distance obtained in step S605 is smaller than d, the process moves to step S606, and if not, step S606 is skipped.

  In step S606, the nearest template T is replaced with the template T 'being processed, and the distance d from the nearest template is replaced with d' obtained in step S604.

  If all templates in the database have been processed, the process moves to step S601.

  In step S601, the reliability of the candidate area is output. The reliability can be defined as, for example, the reciprocal 1 / d (d ≠ 0) of the distance d from the nearest template.

  The existence probability map generation method by the video analysis system 100 has been described above. In the video analysis system 100, the video input unit 102 cuts out a frame image from the input video, and the dynamic object region detection unit 103 and the object category determination unit 104 determine the position, category, and detection reliability of the object in the frame image. calculate. From these pieces of information, the existence probability calculation unit 105 generates an existence probability map in the frame image being processed for each category. Further, the existence probability map in a certain time span is obtained by totaling the time series data of the existence probability map stored in the existence probability accumulation unit. Thereby, the possibility that a specific category object appears in a specific place in the video can be obtained.

  In the following, speeding up of object detection using an existence probability map will be described.

  As described with reference to FIGS. 4 to 6, when detecting an object of a plurality of categories from an image, it is necessary to perform matching processing with all templates for all candidate regions, and the processing load is very large. . The same applies to the case where discrimination is performed using a discriminator for each category by machine learning.

  Therefore, in the present invention, the object detection speed is increased by using the existence probability map and omitting the recognition process for the region where the target category is difficult to appear. Hereinafter, a processing procedure will be described with reference to FIGS. 7 and 8.

  FIG. 7 is a diagram for explaining speeding up of object detection using the existence probability map.

  In the example of FIG. 7, it is assumed that the existence analysis map 702 for the “car” category and the existence probability map 704 for the “people” category have been generated using the video analysis apparatus 100 in advance.

  When the frame image 701 is input, the existence probability maps of “car” and “person” are checked, and only an area with a high existence probability (inside the thick frames of 703 and 705) is extracted and detected. It becomes a processing target. Further, for an area where the existence probability of the “car” category is high, the reliability is obtained using only the template 707 of the “car” category. Similarly, for an area where the existence probability of the “person” category is high, the reliability is obtained using only the template 708 of the “person” category.

  In the case of using classifiers by machine learning, the number of classifiers used for discrimination can be reduced, so that the processing can be improved in the same manner.

  As described above, the entire detection process is speeded up by the two-stage efficiency of reduction of candidate areas and simplification of reliability calculation. Even when there is only one type of category to be detected, high-speed detection is possible by reducing the previous candidate area.

  FIG. 8 is a flowchart showing a processing procedure for speeding up object detection using an existence probability map. FIG. 8 speeds up the object detection processing of the plurality of categories in FIG.

  S801 to S808 are processes for the existence probability map for each category.

  S802 is a process of extracting candidate areas, similar to step S501 in FIG. 5, but areas with low existence probabilities are ignored. Here, the region having a high existence probability or the region having a low probability is based on a predetermined threshold value or a value set by the user.

  S803 to S807 are processes for all candidate regions extracted in S802. The change from FIG. 5 is processing related to the reliability calculation in S804.

  In S804, the reliability of the candidate area for the target category is calculated. S503 in FIG. 5 obtains “object-likeness”, whereas in S804, “object-likeness of a specific category” is obtained. As described with reference to FIG. 7, by limiting the categories, for example, the number of templates to be compared can be reduced, so that the discrimination process can be made more efficient.

  Steps S805, S806, and S809 are the same as those in FIG.

  The speeding up of object detection using the existence probability has been described above. By using the existence probability map accumulated in the past, it is possible to reduce the number of image processes for newly input frame images, and to efficiently detect objects in a plurality of categories.

  As another method of using the existence probability map, abnormal scene detection will be described below with reference to FIGS. 9 and 10. In the second embodiment, an area having a high existence probability is used. However, an area having a low existence probability is used here.

  FIG. 9 is a diagram for explaining the abnormal scene detection using the existence probability map.

  In the example of FIG. 9, it is assumed that the “person” existence probability map 905 has been generated in advance.

  When the frame image 901 is input, the objects 903 and 904 in the “person” category are detected by the dynamic object region detection unit 103 as in the object detection result 902. Here, a region where the existence probability is less than or equal to the threshold value is extracted from the existence probability map 905 by the extraction means. Next, areas 906 and 907 on the detection result area existence probability map 905 are obtained. In this example, the region 906 is detected in a region having a high existence probability, but the region 907 is a detection result in a region having a low existence probability. Here, the region having a high existence probability or the region having a low probability is based on a predetermined threshold value or a value set by the user. Such a detection result is an “event that rarely occurs” and can be regarded as an abnormal scene. In the third embodiment, when an abnormal scene is detected, for example, a warning message is displayed on the display device 110.

  FIG. 10 is a flowchart for explaining an abnormal scene detection processing procedure using an existence probability map.

  In step S <b> 1001, the video input unit 102 acquires a frame image from the input video.

  In step S1002, the dynamic object region detection unit 103 identifies an object region from the frame image acquired in step S1001.

  Steps S1003 to S1008 are processing for all object regions detected in step S1002.

  In step S1004, the object category determination unit 104 determines the category of the detected object category.

  In step S1005, the existence probability map of the category determined in step S1004 is read from the existence probability accumulation unit 106, and the existence probability of the detected area is obtained.

  In step S1006, if the existence probability of the object in the detected area calculated in step S1005 is equal to or less than the threshold value, the process moves to step S1007, and if not, step S1007 is skipped. As described above, the determination processing in step S1006 may simply use only the existence probability of the region, or may use the reliability of the detection region obtained in step S1002. For example, if the existence probability of the region is q and the existence probability obtained from the reliability is p, a measure of the probability difference between p and q is K = p × Log (p / q) + (1−p) × Log (( 1-p) / (1-q)) may be used. Since K becomes larger as p and q differ, it can be understood that the larger the value of K, the more abnormal the scene.

  In step S1007, a warning message is displayed on the display device 110 to inform the user that an abnormal scene has been detected.

  In step S1009, if there is a next frame in the video, the process returns to step S1001 to continue the process. Otherwise, the process ends.

  The abnormal scene detection using the existence probability map has been described above. By using the existence probability map, the system can automatically find an abnormal scene from the input video and warn the user.

  The second and third embodiments are methods using the existence probability map in real time. Hereinafter, a method of realizing a similar scene search by associating and storing information obtained from the existence probability map with respect to past accumulated video will be described.

  FIG. 11 is a diagram for explaining similar scene search using an existence probability map.

  In order to perform a similar scene search, it is necessary to calculate numerical data (video feature amount) representing video features and store it in the video database 1109.

  In the example of FIG. 11, first, existence probability maps 1103 and 1104 are derived for the accumulated video 1101 using the video analysis system 100 of the first embodiment.

  Next, the existence probability map is converted into a feature quantity usable for search. The feature amount can be obtained, for example, by dividing the existence probability maps 1103 and 1104 into a grid pattern such as 1106 and 1107, obtaining an average value of existence probabilities in each grid, and connecting them to vectorize them.

  In the example of FIG. 11, the feature amount is calculated using the existence probability map of two types of categories “car” and “person”, but each category includes one category or two or more categories. A feature vector can be obtained by applying the same processing to the above.

  The obtained feature quantity is registered in the video database 1109 in association with the video data. The structure of the database will be described again with reference to FIG.

  When the search is performed, the existence probability map is similarly derived and the existence probability map is characterized for the query video 1110.

  As a result, the obtained feature quantity is compared with the feature quantity of the video stored in the database, and output in ascending order of the vector distance.

  FIG. 12 is a diagram illustrating a configuration of the video database 1109 and a data example. Here, a configuration example of the table format is shown, but the data format may be arbitrary.

  The database includes a video ID field 1201, a video data field 1202, and a video feature amount field 1203 as basic items. If necessary, other bibliographic information (video shooting location, date and time) may be added.

  The video ID field 1201 holds the identification number of each video. The video data field 1202 holds moving image data that is reproduced when the user confirms the search result. The video feature value field 1203 holds the video feature value calculated from the existence probability map. For example, as shown in FIG. 11, the image feature amount is obtained by dividing the existence probability map of each category into a grid pattern, and quantifying the average value of the existence probabilities of each region, and is represented by fixed-length vector data. The

  FIG. 13 is an overall configuration diagram of the video analysis apparatus 100 that enables retrieval of similar scenes. FIG. 13 shows a system configuration in which a video database 1109 is connected to the configuration of FIG. 1, and a feature amount calculation unit 1301 and a search result output unit 1302 are added to the computer 111.

  The feature amount calculation unit 1301 calculates a video feature amount from the presence probability map calculated by the presence probability calculation unit 105. The video feature amount is output as fixed-length vector data as described with reference to FIG.

  At the time of video registration, the video database 1109 stores the video feature amount calculated by the feature amount calculation unit 1301 and the video data input by the video input unit 102 in association with each other. Further, when searching for similar scenes, the search is performed for data that is close to the feature amount of the query video extracted by the feature amount calculation unit 1301 and the vector distance, and is output in ascending order of distance.

  The search result output unit 1302 outputs the search result obtained from the video database 1109 to the display device 110.

  FIG. 14 is a sequence diagram of similar scene search. 1401 in FIG. 14 is a data flow at the time of video registration, and 1402 is a data flow at the time of similar scene search.

  At the time of video registration 1401, first, the video 1403 input from the video input device 101 is sent to the computer 111.

  The computer 111 generates an existence probability map 1404 in accordance with the processing flow of FIG. Subsequently, the feature amount calculation unit 1301 performs feature amount conversion 1405 of the existence probability map. Finally, the feature quantity / video data set 1406 is sent to the video database 1109.

  The video database 1109 registers the feature amount sent from the computer 111 and the video data set 1406 in association with each other (1407).

  At the time of similar scene search 1402, the user issues a search request 1409 to the computer 111 from the operation information input unit 107. At this time, a query video 1408 is sent from the video input device 101 to the computer 111.

  In the computer 111, as in the registration, the existence probability map is generated 1410 and the feature value 1411 is transmitted to the database 1109 as the query feature value.

  The database 1109 searches for data having a close query feature and vector distance (1413), and returns a similar video list 1414 as a search result to the computer 111.

  The computer 111 shapes the similar video list to form display data, and transmits the display data to the display device 110.

  The display device 110 presents the display data 1415 to the user as a search result (1416).

  The similar scene search using the existence probability map has been described above.

  By using video feature quantities that focus on the existence probability of objects in the “field” of the video, it is possible to obtain search results that cannot be obtained simply by using the apparent feature quantities, and various applications can be realized.

  For example, in a road camera video, by using video data of a traffic accident occurrence site as a query, it is possible to look for places and time zones where accidents are likely to occur, focusing on the number of people and cars.

  In addition, as an application to marketing, it is possible to find places and layouts that are similar in human flow by dividing the detection target of surveillance video in a store into detailed categories according to gender, age, and the like.

  For example, FIG. 15 is a diagram showing a display screen example when the similar scene search of the present invention is applied to marketing. The display screen includes a query video 1501, a search condition input form 1502, a search button 1503, and a search result display area 1504, and is displayed on the display device 110.

  In this example, in the video database, in addition to video data and feature quantities, shooting date and time, implementation status of sales promotion activities, changes in the number of sales, and the like are stored. The sales promotion activities include, for example, announcements regarding products, display of product information and sales floor information on digital signage, and the like.

  The user uses the input device 101 to specify a query video 1501 and additional search conditions (such as a date / time range) and clicks a search button 1503. The video analysis device 100 generates a presence probability map from the query video, and searches for a similar scene from the database based on the feature amount calculated based on the map. At this time, the search result is narrowed down according to the additional condition. Information on similar stores is displayed in the search result display area 1504 by sending the obtained search result to the display device 110 together with information associated with the video. Using this result, the user may perform an effective sales promotion activity, or may automatically execute the same sales promotion activity as that of a store where the number of sales is favorable. For example, advertisement display on digital signage is considered to be relatively easy to automate.

  In the above description, the processing flow has been described on the premise of a fixed camera. As mentioned in the description of FIG. 3, as long as the correspondence between the coordinates of the video and the coordinates of the existence probability map can be taken, A map can be generated. The following describes a method for generating a wide-area existence probability map and an example of its use.

  FIG. 16 is a diagram for explaining a method of generating a wide area existence probability map from a wide area video imaged by a camera capable of panning, tilting, and zooming (PTZ).

  The camera of the video input device 101 can shoot video in the range of 1601 by controlling the PTZ. However, only the range of 1602 can be shot at a time even when zoomed out to the maximum.

  Therefore, in the video analysis apparatus of the present invention, the camera PTZ is controlled to divide a wide area video into a plurality of partial area videos (1602, 1603, 1604, etc.) and to take a video for a certain period of time for each partial area video. Generate an existence probability map. By combining the existence probability maps of the partial regions obtained as a result, a wide-area existence probability map 1608 is obtained.

  FIG. 17 is a diagram illustrating automatic control of the PTZ of the camera as an example of utilizing the wide area existence probability map.

  The above-mentioned wide-area existence probability map is totaled for each time zone to obtain an existence probability limited to that time. Reference numerals 1701, 1702, and 1703 are existence probability maps obtained from videos taken at 6 o'clock, 12 o'clock, and 18 o'clock respectively. When shooting a wide area, the area where the dynamic object is highly likely to change is changed for each time zone. Therefore, PTZ control suitable for the time zone can be used to record useful images. it can. In the example in the figure, there are many dynamic objects in the A, B, C, and D areas at 6 o'clock, so PTZ control scheduling is performed so that A, B, C, and D are captured as shown in the transition diagram 1704. . Similarly, B, C, D, F, G, and H are preferentially photographed at 12 o'clock as in transition diagram 1705, and F, G, and H are preferentially photographed at 18:00 as in transition diagram 1706.

100: Video analysis device, 101: Video input device, 102: Video input unit, 103: Dynamic object region detection unit, 104: Object category determination unit, 105: Existence probability calculation unit, 106: Existence probability accumulation unit, 107: Operation information input device 108: Operation information input unit 109: Existence probability output unit 110: Display device 111: Computer

Claims (7)

A dynamic object region detection unit for detecting a region where a dynamic object exists from a frame image of the input video;
An object category discriminating unit for discriminating a category of an object detected by the dynamic object region detecting unit;
For each category determined by the object category determination unit, an existence probability calculation unit that generates an existence probability map representing a probability that an object appears in the detected dynamic object region;
An existence probability accumulation unit for storing the existence probability map as time series data;
The existence probability calculation unit generates an existence probability map for each category in a certain time span from the existence probability map of the time-series data stored in the existence probability storage unit,
And a presence probability output unit that outputs a presence probability map for each category in the fixed time span. Furthermore, from the existence probability map for each category in the fixed time span, there is an extraction means for extracting an area having an existence probability equal to or higher than a threshold value,
The video analysis apparatus according to claim 1, wherein the dynamic object detection unit detects the dynamic object in a region where the existence probability is a threshold value or more. Furthermore, from the existence probability map for each category in the fixed time span, having an extraction means for extracting an area having an existence probability equal to or less than a threshold value,
The video analysis apparatus according to claim 1, wherein the dynamic object detection unit detects an abnormal scene when the dynamic object is detected in an area where the existence probability is equal to or less than a threshold value.   The video analysis apparatus according to claim 1, wherein the input video is a wide-area video shot by a camera capable of panning, tilting, and zooming. The existence probability map is generated for each time zone,
5. The video analysis apparatus according to claim 4, further comprising control means for controlling pan, tilt, and zoom of the camera so that an area having a high existence probability is photographed for each time period. A dynamic object region detection unit for detecting a region where a dynamic object exists from a frame image of the input video;
An object category discriminating unit for discriminating a category of an object detected by the dynamic object region detecting unit;
For each category determined by the object category determination unit, an existence probability calculation unit that generates an existence probability map representing a probability that an object appears in the detected dynamic object region;
A feature amount calculation unit for calculating a video feature amount from the existence probability map;
A video database that stores the video feature quantity and the input video in association with each other;
When a search request for a scene similar to a query video is received, a video having a feature quantity similar to the video feature quantity of the query video calculated by the feature quantity calculation unit is searched from the video database and output as a similar video A video analysis apparatus characterized by: A video input device;
A video input unit for extracting a frame image from the video input from the video input device;
A dynamic object region detection unit for detecting a region where a dynamic object exists from the frame image;
An object category discriminating unit for discriminating a category of an object detected by the dynamic object region detecting unit;
For each category determined by the object category determination unit, an existence probability calculation unit that generates an existence probability map representing a probability that an object appears in the detected dynamic object region;
An existence probability storage unit for storing the existence probability map as time series data;
The existence probability calculation unit generates an existence probability map for each category in a certain time span from the existence probability map of the time series data stored in the existence probability accumulation unit,
An existence probability output unit that outputs an existence probability map for each category in the fixed time span;
A video analysis system comprising: a display device that displays the output existence probability map.