JP2009027393A - Image searching system and personal searching method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a highly accurate personal searching system for extracting different characteristic amounts of the same person from images having various characteristics because the cameral setting conditions are different. <P>SOLUTION: An image monitoring system includes an image extracting unit for extracting signals from imaging devices such as one or more cameras and an image data unit for holding an input image obtained by the image extracting unit. The image monitoring system further includes a condition designating unit for designating personal characteristic, time, and camera from the input image, an image searching unit for searching, by image recognition, an image matched in the conditions designated by the condition designating unit from the input image groups stored in the image data unit and a result displaying unit for displaying the results of the image searching unit, in addition to the means for selecting a plurality of personal characteristics designated by the condition designating unit with the user designation or personal tracking method and for adding such personal characteristics to the searching conditions of the condition designating unit. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a so-called video recording apparatus having a function of recording and recording image data from video acquired from an imaging apparatus such as a camera, and a video monitoring system that realizes functions such as intruder detection by image recognition.

  A video monitoring system having a function of detecting a moving object such as a person or a vehicle that appears in a monitoring area by image recognition from a video acquired from a camera has a function of recording only a video in which a moving object appears by using a detection result, Since the display device has a function of presenting a warning icon and can be alerted by sounding a buzzer or the like, it is useful for reducing the burden of monitoring work that conventionally required constant confirmation work. In addition, in the above-described video monitoring system, when an illegal act such as theft occurs, the recorded video can be used for subsequent crime verification.

  In recent years, crime diversification and an increase in the number of criminal offenses have led to an increased awareness of crime prevention at mass retailers, financial institutions, buildings, offices, etc., and video surveillance systems have been introduced. The capacity of video recording apparatuses is also increasing, and the number of cameras is increasing as cameras are installed in various places due to the spread of IP cameras and the like. As described above, it is very difficult to identify a criminal from a recorded image by visual observation by a monitor, and there is an increasing demand for a function that supports monitoring work.

  For example, Patent Document 1 discloses a means for identifying a specific person such as a criminal from image data recorded by an image recognition technique for the purpose of monitoring work support. In Patent Document 1, an image for setting a search condition is designated, and an object that is present in the image or a keyword similar thereto is used as a search key (mainly a face area in Patent Document 1), and an image having high similarity to the search key A function for acquiring and presenting image data from image data is shown.

JP 2002-183205 A

  A video acquired from a camera installed in a monitoring area may have various situations such as an angle of view, lighting conditions, a person's orientation and posture. In that case, even if the shape information of the object is the same person due to changes in the spatial characteristics of the image data, the color characteristics of the image, and the posture, the information changes variously. For this reason, since the person feature acquired from the designated image as in Patent Document 1 is limited to the person feature in the case of a certain specific condition, when trying to identify a specific person from the image using this as a search key, Although it is the same person, it is possible that missed information that would be missed occurs.

  In order to deal with these problems, it is necessary to search using robust human features that are less affected by lighting conditions and orientations. However, even with a robust feature amount, it is considered extremely difficult to reduce false alarms due to the non-rigidity of people and the variety of appearances.

  An object of the present invention is to solve the above problems, and to improve the accuracy of person search in a person search system.

  In order to solve the above-described problem, a video search system according to the present invention includes a condition specifying unit that specifies a search condition including the characteristics of a search target object, and the condition specifying unit from the input image group held in the image data unit. The user searches for an image of the search target object from the image displayed on the result display unit, an image search unit that searches for an image that matches the condition specified in Step 1, a result display unit that displays the search result of the image search unit, and Image selecting means that can be selected, and means for updating the characteristics of the search target object of the condition designating unit using the image selected by the image selecting means.

  The present invention also provides a condition designating unit for designating a human characteristic to be searched for, a search time or an imaging device, and an image that matches a condition specified by the condition designating unit from the input image group held in the image data unit. And a result display unit for displaying the search result of the image search unit. The user can select an image of a person to be searched and displayed on the result display unit and selected by the user. The above-described problem is solved by a person search method for updating the person feature of the search target specified by the condition specifying unit from the search target person image.

  According to the present invention, it is possible to perform a person search even for images that are the same object but have different appearances. Therefore, it is possible to improve the search accuracy of the video search system.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a configuration in the case where the person search function is applied to a video surveillance system. The hardware is configured by an electronic computer system including a CPU and executes each function. The video is acquired from a video acquisition unit 101 that acquires a signal obtained from the imaging device 100 such as one or a plurality of TV cameras installed in the monitoring target area as a video, and is recorded on a recording medium such as a hard disk drive from the video acquisition unit 101. The image data section 102 recorded as a data group, selecting an image from the image data section 102 and acquiring a person feature from a designated image, or a condition designating section for designating a search time and a camera to be searched as a search condition 103, an image search unit 104 that calculates the similarity between the image of the image data unit 102 and the specified image based on the condition specified by the condition specifying unit 103, and a result display unit 105 that outputs the result of the image search unit 104 to a monitor or the like Consists of.

  The video acquisition unit 101 may acquire the image data as a one-dimensional array or a two-dimensional array from a video signal input from a video recording device or the like in which the image data is recorded. In order to reduce the influence of noise, flicker, etc., this image data may be subjected to processing such as a smoothing filter, contour enhancement filter, and density conversion as preprocessing. Further, a data format such as RGB color or monochrome may be selected according to the application. Furthermore, the image data may be reduced at a predetermined size in order to reduce processing costs. The condition designation unit 103 and the result display unit 105 allow the user to designate conditions or confirm results through a graphical user interface or the like.

  Next, details of the condition specifying unit 103 will be described with reference to FIG. In the condition designating unit 103, a person designating unit 200 that designates a person region from an image and acquires a person feature, a person tracking unit 201 that tracks the designated person within the same camera, and a feature extraction from the image data of the tracked person. A feature extraction unit 202, a person feature holding unit 203 that holds the person feature acquired by the above processing, a search condition specifying unit 204 for specifying a search time, a camera to be searched, and the like, and presenting a person image and determining the correct answer to the user It is comprised by the candidate presentation means 205 to be made.

  The person designation means 200 uses a menu screen as shown in FIG. Of the images recorded in the image data section 102, for example, an image showing a person who seems to be a criminal is displayed by adjusting the camera designation 300 and the time designation 302, and the person area is displayed from the image output 301 with a mouse or the like. Specify more. Here, it is also possible to directly specify a human image acquired in advance. Further, the search condition designating unit 204 sets the time to be searched in the search time designation 303 and the camera to be searched in the target camera designation 304 on the menu screen as shown in FIG. To do. It is desirable for the user that the target camera designation 304 has specifications linked to the camera installation drawing.

  For the image specified by the person specifying means 200, the person tracking means 201 tracks the specified person in image data acquired before and after the acquisition time of the specified image. This is to cope with the fact that a feature amount suitable for person search cannot be acquired when feature extraction described later is performed from one image. When the feature extraction of a person to be searched depends on a single image of a certain camera, the appearance of this person on other cameras is different, so that the search accuracy may be lowered. Even if the images are acquired by the same camera, if the image acquisition times are different, for example, the appearance changes from the front of the person to the side of the person, and the feature amount changes accordingly. Therefore, person tracking means 201 is introduced to acquire a plurality of feature quantities of the person. The person tracking by the person tracking unit 201 can be obtained by image collation between frames before and after the specified image acquired by the person specifying unit 200, tracking the center of gravity, or the like.

The person tracking means 201 can designate a person area in each frame. The feature extraction unit 202 executes feature extraction that integrates these image features. As shown in FIG. 4, for example, a case where a person feature is a histogram of RGB values will be described as an example. If the tracking result is a N number is N pieces generated when performing feature extraction (a) from the feature histogram H ₁ to (b), wherein the histogram H _N for each of the tracking result. Next, the similarity of each histogram is calculated by, for example, histogram intersection.

Ｋはヒストグラムの量子化数であり、ｉはヒストグラムの１つの要素である。

K is the quantization number of the histogram, and i is one element of the histogram.

  When a histogram having a similarity lower than a predetermined threshold, that is, a human feature having a different appearance, is acquired, an image recognition error may occur. Therefore, as shown in FIG. 5, it is possible to avoid acquiring the wrong feature amount by presenting the candidate image to the user with the candidate presenting means 205 and causing the user to determine whether or not they are the same person. . The user selects a correct answer from the candidate images 500 and presses an add button 501 to add an image feature. It is possible to compensate for a portion that cannot be determined by image recognition through user identification.

  By integrating histogram pairs having high similarity, it is possible to avoid redundant search processing when the same feature histogram exists. This makes it possible to acquire a plurality of different features of the same person in the same camera, and to suppress a person search misreport. As another feature amount, as shown in FIG. 4C, if the size and face of a person can be acquired, face data and the like can be combined. In addition, various feature amounts such as human shape data, luminance information, and gait state can be selected. The designated person feature data 400 acquired by the above processing is held by the person feature holding means 203.

  Next, details of the image search unit 104 will be described with reference to FIG. The image search unit 104 is a part that identifies a specific person from the image of the image data unit 102 based on the search condition specified by the condition specifying unit 103. The image search unit 104 includes a person region extraction unit 600 that extracts a person region from an image recorded in the image data unit 102, a feature extraction unit 601 that extracts a person feature from a person image acquired from the person region extraction unit 600, and a feature extraction unit. The similarity calculation means 602 that compares the person characteristics acquired in advance with the condition specifying unit 103 and the condition specifying unit 103 causes the user to determine whether the image with the high similarity displayed on the result display unit 105 is a specific person. A person feature adding unit 603 updates the person feature of the image determined to be correct to the person feature held in the person feature holding unit 203 by addition or integration. Here, the update by integrating the features of the person may be performed by the method described in the feature extracting unit 202, for example.

  The operation of the image search unit 104 will be described in detail below.

  First, the search target is limited based on the target camera specified by the search condition specifying unit 204 and the search target time, and an image is acquired from the image data unit 102. Next, a person area extraction unit 600 extracts a person area. The person area is extracted by using the background subtraction method, the frame subtraction method, and a person classifier created in advance by learning human data to identify the human area and extract the human area. Get as data.

  In the processing by the person area extracting unit 600, the image data specified by the search condition specifying unit 204 in the image data unit 102 is searched in order from image data temporally close to the image specified by the condition specifying unit 103. This is because, before performing a search for the entire specified time, the search result is presented in the middle of the search, and the user is allowed to select the correct answer, thereby expanding the person feature in the same manner as the condition specifying unit 103 described above. This is because the search accuracy can be improved. Details of this will be described later.

  Next, the person feature amount is extracted by the feature extraction unit 601 from the image data of the person acquired by the person region extraction unit 600. This process is the same as the process performed by the feature extraction unit 202 in the condition specifying unit 103. That is, a feature histogram such as an RGB value and a face image are acquired from the image. Next, a comparison process between the feature amount and the person feature of the specific person held by the person feature holding unit 203 is performed, and a similarity indicating whether the extracted person is the same as the designated person is obtained by comparing the person feature data. The similarity calculation unit 602 executes this. The feature amount comparison includes a color histogram, a gradient vector histogram, face matching, and image matching, but any one or a plurality of comparison methods may be used.

  Next, the processing of the result display unit 105 will be described with reference to FIG. An example of a method for displaying an image in which a specific person exists based on the calculated similarity is a screen configuration in the form shown below. FIG. 7A shows a method of displaying the camera and time of the video in the thumbnail format in the order of similarity. In the case of images with similar frame numbers, there are many similar images, and the same image may be arranged at the top. Therefore, when the image having the highest similarity within a certain time is displayed as the image information 701 as the representative image 700, good.

  In addition, when the user is the same person as specified by the user, the person feature can be expanded by the user's judgment. The time specified by the condition specifying unit 103 is divided into several periods or cameras, and the result of similarity calculation is presented to allow the user to specify the correct answer. When this process is repeated and all the image data satisfying the search condition is searched, the search process is terminated. By performing such user interactive processing, the ambiguity of image recognition can be left to the person's judgment, and it becomes possible to realize a person search system with reduced false alarms.

  In addition, the user causes the user to select an image from which the search target person has been correctly extracted in the search result display unit, extracts the person feature of the selected image, and extracts the person feature extracted by the person feature adding means 603 as the person feature. By updating the holding means 203 by addition or integration, the characteristics of the person to be searched can be extracted under any shooting conditions, so that the search accuracy can be improved.

  FIG. 7B shows an example of a display method that can express the camera, time, and movement between cameras. Similarly to the example of FIG. 7A, an image having the highest similarity among the similar image groups is displayed as the representative image 702 within a predetermined time interval. As a result, the user can grasp the relationship between the movement path of the person and the time. Next, in FIG. 7C, the relationship between the camera installation position and the video is grasped. When the designated person exists, the camera installation information 704 and the pop-up display 703 of the image of the designated person are displayed on the same screen. To display. Further, a method is conceivable in which the image can be confirmed by forward or backward feeding with the image scroll 705. These result display methods can be selected according to the usage method of the user.

  According to the present invention, when the user selects an image that is incorrect by the result display unit 105, the person shown in the image is different from the specific person, and the similarity is obtained when the person exists in the search result. A function that suppresses false alarms can be realized by adjusting it to a low level.

  In addition, if the person specified by the condition specifying unit 103 exists in a certain camera, the person existing in the other camera is identified using the obvious condition that it does not exist in the other camera within the same time. By performing a search by adding a condition that it is different from a person, a function that can suppress false alarms as described above can be realized.

  Furthermore, by adding camera installation information, for example, it is impossible for a person who has moved to the camera on the first floor to instantaneously exist on a camera on another floor. It is possible to suppress misreporting by performing a low adjustment or the like. In addition, when the user corrects an incorrect person in the result display unit 105 and specifies an image of a location having a low existence probability from the above camera installation information, the user is notified of a determination error. It is possible to realize functions that can be performed.

  Although the embodiment of the present invention refers to the video surveillance system, the present invention is not limited to this, and the present invention can also be used for searching for various videos such as TV videos and video videos.

  Furthermore, in the embodiments of the present invention, the person search system has been described in detail. However, the search system of the present application is not limited to the person search. Any object can be applied.

  According to the present invention, it is possible to hold a plurality of person features by means of a person tracking by image recognition or an interactive interface by user designation, and even for an image that can be reported as a person due to a difference in appearance while being the same person, It becomes possible to perform a person search with high accuracy. Therefore, it is possible to support the work of a supervisor and to realize an efficient video search system.

It is the figure which showed the whole structure of one Example in this invention. It is a figure explaining the detail of the condition designation | designated part in this invention. It is the figure which showed the example of the user interface of the condition designation | designated part in this invention. It is the figure which showed the feature example of the person feature in this invention. It is the figure which showed the user interface which implement | achieves the similar person selection in this invention. It is a figure explaining the detail of the image search part in this invention. It is the figure explaining the Example of the result display part in this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging device 101 Image | video acquisition part 102 Image data part 103 Condition specification part 104 Image search part 105 Result display part 200 Person specification means 201 Person tracking means 202,601 Feature extraction means 203 Person feature holding means 204 Search condition specification means 205 Suggestion presentation Means 300 Camera designation 301 Image output 302 Time designation 303 Search time designation 304 Target camera designation 400 Designated person feature data 500 Candidate image 501 Add button 600 Person region extraction means 602 Similarity calculation means 603 Person feature addition means 700 and 702 Representative image 701 Image information 703 Pop-up display 704 Camera installation information 705 Image scroll

Claims (10)

A video acquisition unit that acquires signals from one or more imaging devices;
In a video search system having an image data unit that holds an input image acquired from the video acquisition unit,
A condition designating unit for designating a search condition including the characteristics of the search target object;
An image search unit for searching for an image that matches the condition specified by the condition specifying unit from the input image group held in the image data unit;
A result display unit for displaying a search result of the image search unit;
An image selection means that allows a user to select an image of the search target object from the images displayed on the result display section;
Means for updating the characteristics of the search target object of the condition specifying unit using the image selected by the image selection means;
A video search system characterized by comprising: The condition designating unit includes a search target designating unit that allows a user to designate a search target object;
The video search system according to claim 1, further comprising: object feature holding means for holding features of a plurality of the search target objects extracted from images before and after the specified search target object.   The image search unit includes an object extraction unit that extracts an object, a feature extraction unit that extracts a feature of the object extracted by the object extraction unit, a feature of the object extracted by the feature extraction unit, and the condition designation unit The video search system according to claim 1, further comprising: similarity calculation means for calculating the similarity of the characteristics of the search target object.   The video search system according to any one of claims 1 to 3, wherein the image selection unit includes a unit that acquires the person feature from an image that is not determined to be a search target person and corrects a search result. . The search condition of the condition specifying unit includes camera installation position information,
The said image selection means has a means to judge a user's existence probability of a search target object from the said camera installation position information, and to alert a user when a user selects the wrong image. The video search system described in Crab. A video acquisition unit that acquires signals from one or more imaging devices;
In a person search method having an image data unit that holds an input image acquired from the video acquisition unit,
A condition designating unit for designating a human characteristic to be searched or a search time or an imaging device;
An image search unit for searching for an image that matches the condition specified by the condition specifying unit from the input image group held in the image data unit;
And a result display unit for displaying a search result of the image search unit. The image of the search target person selected by the user and displayed on the result display unit by a method by which the user can select the search target person image. The person search method characterized in that the person feature of the search target specified by the condition specifying unit is updated.   The person according to claim 6, wherein the condition designating unit extracts the person feature from images before and after the person image to be searched designated by a user, and holds a plurality of the person features. retrieval method.   The image search unit extracts a person, extracts a feature of the person, and calculates a similarity between the feature of the person and the person feature to be searched specified by the condition specifying unit. The person search method according to claim 6 or 7.   9. The method according to claim 6, further comprising means for acquiring the person feature from an image that is not determined to be a person to be searched when the search condition is updated, and correcting the search result. Person search method.   The update apparatus further comprises means for determining the existence probability of a person to be searched from camera installation position information when updating the search condition and alerting the user when the user selects an incorrect image. The person search method according to any one of claims 6 to 9.

Images