Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
  • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/40—Scenes; Scene-specific elements in video content
  • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
  • G06V20/47—Detecting features for summarising video content

Definitions

• the invention relates to the search for information in a video archive and relates more specifically to a method and a device for identifying an object in a video archive comprising a plurality of images acquired on a network of cameras. .
• the invention also relates to a computer program stored on a recording medium and adapted, when executed on a computer, to implement the method according to the invention.
• the main drawback of this representation comes from the fact that the access point to the information is unique and is constituted by the root of the hierarchical tree, hence problems in the search for information.
• the data can also be organized according to a network model in the form of a graph where the archived entities are connected to each other by means of logical pointers.
• Object-oriented databases are also known which are capable of storing a multitude of information in objects such as, for example, an individual form, a machine, a resource ... to which values and attributes are associated.
• a fundamental problem is that it is particularly difficult to quickly identify an object in a video archive of a database containing a large number of images, particularly when the there is very little information on the object sought. Such a situation arises for example when searching, from a simple report, an unidentified individual in a video surveillance archive containing thousands of hours of recording. In this context, it is currently necessary to manually view all recorded video archives.
• Another object of the invention is to enable a human operator to have access to structured visual summaries of the objects present in a heterogeneous video database.
• Another object of the invention is to provide the human operator with optimized tools for navigating the database through an interactive search strategy.
• a method of identifying an object in a video archive comprising a plurality of images acquired on a camera network, including a characterization phase of the object to be identified and a search phase.
• said object in said archive said phase characterizing method for defining for said object at least one extractable semantic feature of said video archive, even on low resolution images, and directly interpretable by an operator, said search phase consisting in filtering the images of said video archive according to the semantic feature defined above, to automatically extract from said archive the images containing an object having said semantic feature, to define a group of objects comprising all the objects present in the video archive having said semantic feature, and to measure the similarity of the object to be identified with any other object of the previously defined group as a function of visual characteristics and space-time constraints on the path of the object to be identified in the space covered by the camera network.
• the step of measuring similarity comprises the following steps:
• the method according to the invention further comprises a step of assigning to each similarity measure a likelihood coefficient.
• the method according to the invention comprises a step of merging the results of the steps of the similarity measurement so as to define a single unified measure of similarity, for defining a distance in the space of the objects to be identified.
• the method according to the invention comprises a dynamic structuring of this space of the objects of interest, by means of the distance defined above, so as to be able to browse interactively in the video archive according to a hierarchical tree.
• the invention applies in the search for a human person in which said object to be identified is a human person for whom there is only a summary report.
• the semantic feature of said human person consists of a visible physical characteristic and / or a visible accessory.
• the method according to the invention is implemented by a device for identifying an object in a video archive comprising a plurality of images acquired on a network of cameras, characterized in that it comprises a characterization module of the object to be identified and a search module of said object in said archive, said characterization module comprising means for defining for said object at least one extractable semantic feature of said video archive, even on low resolution images, and directly interpretable by an operator, said search module comprising means for filtering the images of said video archive according to the semantic feature defined above, means for automatically extracting from said archive the images containing an object having said semantic feature, means for defining a group of objects comprising all the objects present in the video archive having said semantic feature, and means for measuring the similarity of the object to be identified with any other object of the gr oupe defined above according to visual characteristics and spatio ⁇ time constraints on the path of the object to be identified in the space covered by the camera network.
• said similarity measuring means comprise: a first calculation module configured to estimate the compatibility of the semantic feature of the object to be identified with the semantic feature extracted from the images of the other objects of the group defined above, and / or, a second calculation module configured to estimate the spatio-temporal compatibility of the path of the object to be identified with the path of another object of the previously defined group having a semantic characteristic similar to that of the object to be identified.
• the method according to the invention is implemented in said device by a computer program stored on a recording medium and adapted, when it is executed on a computer, to identify an object in a video archive comprising a plurality of images acquired on a network of cameras, said computer program comprising instructions for performing a characterization phase of the object to be identified and instructions for carrying out a search phase of said object in said archive, said characterization phase consisting in defining for said object has at least one extractable semantic feature of said video archive, even on low resolution images, and directly interpretable by an operator, said search step of filtering the images of said video archive according to the semantic feature defined above, to extract automatically from said archive the images cont by having an object having said semantic feature, defining a group of objects having all the objects present in the video archive having said semantic feature, and measuring the similarity of the object to be identified with any other object of the group defined previously in function of visual characteristics and constraints on the spatio- time of the object to be identified in the space covered by the camera network.
• the goal is to quickly find images of the incident, if they exist, and find the complete route of the suspect in the area covered by the network of cameras to determine the path spatio-temporal and identify it.
• the conventional approach is to view the images taken by the cameras near the indicated location of the incident and at times similar to that indicated by the witnesses to identify the incident in the video archive filmed.
• the approach proposed by the present invention consists in exploiting the reports given by the witnesses to systematize the research of the suspect and to filter the data before optimizing the search for images in the CCTV archive.
• the description of the suspect provided by the witnesses is used to define semantic information about the suspect.
• the latter can for example be tall, very thin, wear a long black coat and sunglasses, with a beard and long hair.
• these characteristics some are exploitable by the method according to the invention and programmed directly into the system.
• This pretreatment comprises the following steps:
• the detection of the movements is carried out by modeling the scene by gaussian mixtures (the background being fixed), and the tracking is performed by means of a Kalman filter, and then completed. by local analysis using local descriptors of the SIFT or SURF type, for example, or even simpler and more punctual modeling, in order to solve ambiguities due to occlusions.
• the detection of the persons is obtained for example by detecting the faces by using the techniques based on cascades of classifiers such as Adaboost ® and Haar filters, then possibly going back to the complete body envelope by shape analyzes with, possibly, postulates on physiognomic ratios or detectors of individuals based on learning techniques.
• the specialized algorithms used to characterize each of the persons are, for example, classifiers capable of indicating whether an individual has long or short hair, beard or not, has a very rounded or rather elongated face, is overweight. or has a slender figure, etc.
• a measure of reliability of the response is provided for each of the extracted information.
• These characterizations (or descriptors extracted from the images) are directly interpretable by a human operator and can be directly related to the semantic information collected during a testimony. In addition, they are calculated even on low resolution images. It is indeed not necessary to have hundreds of pixels wide on a face to determine if a person is wearing glasses.
• the classifiers are obtained according to the method described below:
• - Descriptors images are extracted locally on the thumbnails extracted (for example, to determine if a person wears a beard, one is interested in the lower half of the mask of detection of the face); these descriptors can be, for example, histograms of colors, gradients, spatial distribution properties characterizing the textures, responses to filters (Gabor for example), etc .; - classifiers are then constructed by machine learning to indicate which faces have the characteristic "beard"; an alternative approach is to learn distance measurements specific to these characteristics and then use these specific distances to determine the proximity or difference between two faces on certain semantic aspects.
• the reliability measure can be provided directly by the classifier. It can also be modeled a posteriori by translating for example the previous outputs into probabilities.
• Spatio Temporal ⁇ compatibility constraints may be binary (a person can not be in two places at once) or fuzzy (floating confidence value, ie more or less likely). Thanks to these constraints, observations between several cameras can be matched, more or less complex and reliable relationships are established between all the entities of the database.
• the video surveillance archive is represented by a semantic database associated with each individual seen in at least one of the videos.
• the structuring of the semantic database consists of the following steps:
• This report transmitted by the witnesses is exploited.
• This report includes semantic characterizations, possibly with associated confidence measures based on the witnesses' memories and the consistency of the statements;
• the database is filtered from said semantic features while retaining only the individuals having these characteristic features by removing all the individuals that do not have these features;
• the structuring of the database can be dynamic. For this purpose, it suffices to add, delete or adapt semantic criteria for the hierarchical structuring can be updated to reflect the expectations of the operator. Thus, it is possible to qualify the reliability of body information or add new information on the shape of the face and the wearing of a cap. It is also possible to automatically propose new structuring to the user.
• the user can navigate the database efficiently according to the individuals and their characteristics and no longer according to the cameras and the scrolling of the time.
• the corresponding video sequence can be viewed; this designation makes it possible to specify more precisely the visual appearance, which makes it possible to complete the similarity measurements.
• it provides spatiotemporal information on the location of the individual.
• the already filtered database is again filtered to remove all individuals whose positions and acquisition dates do not correspond with the spatio-temporal constraints of the normal displacement of the designated individual; The remaining individuals are ordered according to a combination of semantic factors, appearance characteristics, and the probability that this is the designated individual, due to spatio-temporal constraints on movement (a distance that can be estimated, a probable speed that can be calculated and a defined maximum speed).
• the user can then browse this ordered list and very effectively and very quickly perform the tracking and back-tracking of the designated individual, by browsing the archive via the spatio constraints. ⁇ temporal, semantic properties and appearance criteria, without having to worry about the selection of cameras or the timestamp of data.
• images of a scene are recorded (step 2) by a camera network 4 comprising several cameras distributed geographically over a monitored area.
• step 6 a time range is selected during which the recorded images will be analyzed.
• phase T2 describes the operation by an operator 20 of the database constituted during the phases Tl to T3.
• step 22 the operator designates the time range of the filmed event.
• step 24 the operator provides, via a user interface, attributes of the desired individual.
• the system displays (step 26) the filtered images from the structured database generated in the previous steps.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Data Mining & Analysis (AREA)
• Databases & Information Systems (AREA)
• General Engineering & Computer Science (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=40467086

Family Applications (1)

Country Status (6)

Families Citing this family (17)

Citations (1)

Family Cites Families (20)

• 2008
  • 2008-08-27 FR FR0855737A patent/FR2935498B1/fr active Active
• 2009
  • 2009-08-26 CN CN200980133643.0A patent/CN102187336B/zh not_active Expired - Fee Related
  • 2009-08-26 EP EP09809332A patent/EP2316082A1/fr not_active Ceased
  • 2009-08-26 WO PCT/EP2009/060960 patent/WO2010023213A1/fr active Application Filing
  • 2009-08-26 US US13/059,962 patent/US8594373B2/en active Active
• 2011
  • 2011-02-08 IL IL211129A patent/IL211129A0/en not_active IP Right Cessation

Patent Citations (1)

Also Published As

Similar Documents

Legal Events