WO2009146756A1

WO2009146756A1 - Image processing apparatus with calibration module, method for calibration and computer program

Info

Publication number: WO2009146756A1
Application number: PCT/EP2008/065529
Authority: WO
Inventors: Marcel Merkel; Hartmut Loos; Jan Karl Warzelhan
Original assignee: Robert Bosch Gmbh
Priority date: 2008-06-06
Filing date: 2008-11-14
Publication date: 2009-12-10
Also published as: EP2300961A1; DE102008002275A1

Abstract

With the aim of relieving the load on monitoring personnel and improving the monitoring quality, image processing methods have been proposed which automatically evaluate the recorded video data using digital image processing. Customary methods involve moving monitored objects being isolated from the essentially static scene background, tracked over time and an alarm being triggered in the event of relevant movements. However, the video data first of all show only a two-dimensional representation of the monitoring areas. Tracking the monitored object over time - also called trajectory - without further evaluation thus provides no inference of the actual trajectory of the monitored object in the monitoring area. An image processing apparatus 13 for a video monitoring system for monitoring a monitoring area 1, which may have at least one monitored object 2 and at least one concealment geometry 3, with a detection module 15 which is designed to detect one of the monitored objects 2 is proposed, wherein the monitored object 2 is detected on the basis of a set of subarea detectors 7a-e which detect different subareas of the monitored object 2, and wherein detection of the monitored object 2 prompts the set of subarea detectors 7a-e to be divided into a positive set 8 of subarea detectors which have identified the associated subarea of the monitored object 2 and into a negative set 9 of subarea detectors which have not identified the associated subarea of the monitored object 2, with a calibration module 16, 17 which is designed to calibrate a depth map and/or a concealment geometry for the monitoring area 1 on the basis of a detected monitoring object 2 with a negative quantity 8.

Description

description

title

Image processing device with calibration module, method for calibration and computer program

State of the art

The invention relates to an image processing apparatus for a video surveillance system for monitoring a surveillance area, which may have at least one surveillance object and at least one masking geometry, with a detection module, which is designed to detect one of the monitored objects, wherein the detection of the monitored object is based on a set of partial area detectors , which detect different portions of the surveillance object, and wherein upon detection of the surveillance object, the amount of the detail area detectors into a positive amount of

Subarea detectors, which have detected the associated portion of the surveillance object, and in a negative amount of subarea detectors, which have not recognized the associated portion of the surveillance object is divided. The invention also relates to a method for calibration and to a corresponding computer program.

Video surveillance systems are often used to monitor public places or buildings, such as train stations, intersections, libraries, hospitals, etc., but also for private use, such as factory surveillance. Such video surveillance systems typically include one or more surveillance cameras directed to relevant surveillance areas. The video data generated by the surveillance cameras are combined in many embodiments in a monitoring center and there evaluated either manually by monitoring personnel or automated. With the aim of relieving monitoring personnel and improving the quality of surveillance, image processing methods have been proposed which evaluate the recorded video data automatically via digital image processing. In conventional methods, moving monitoring objects are separated from the essentially static scene background, tracked over time and triggered an alarm for relevant movements. However, the video data initially only show a two-dimensional representation of the surveillance areas or 2D images of the surveillance areas in the image plane of the surveillance cameras. A tracking of the surveillance object over time - also called trajectory - gives without further

Evaluation thus no conclusion on the actual trajectory of the monitored object in the surveillance area or on the trajectory in the real 3D scene. By means of a so-called depth map, which shows a connection between the monitored area - also called surveillance scene - and the actual spatial conditions, it is possible to use a

Establish the relationship between the two-dimensional movement of the surveillance object on the video image and the actual trajectory of the surveillance object in the surveillance area or in the real 3D scene.

Another problem in this context results from the existence of further objects, in particular stationary and / or quasi-stationary objects in the surveillance area, which obscure the surveillance object in sections and / or temporarily. Such obscuration obstructs the detection and tracking of the surveillance objects.

In this context, DE 10 2006 027120 A1 proposes an image processing method for detecting and processing visual obstacles in a surveillance scene, wherein a plurality of state data records of a surveillance object are detected, each having an object position and a size measured at the object position, ie height, of the surveillance object Include an image sequence in an image. By comparing the measured size of the monitored object with a modeled, perspective size of the monitored object at the same object position, one of the visual obstacles is concluded. The image processing method is based on the fact that the monitoring object is assigned to a class, for example persons, of whom a middle one belongs or usual size is known and based on this size, the modeled, perspective size is determined.

In the scientific article by M.A. Sotelo, I. Renno, J. Orwell and G. Jones: "Pedestrian Detection Using SVM and Multi-Feature Combination", IEEE

Intelligent Transportation Systems Conference, September 17-20 2006 is proposed for the detection of passers-by, in a classical approach of a person using a so-called bounding box, which encloses the entire passers-by, to use for detection. Alternatively, a component approach is described wherein the body of the person is divided into several sections and each one

Subarea is independently learned by a specific detector or classifier. The individual local parts of the body are assembled in a further step to the person in the totality. This scientific article is probably the closest state of the art.

Disclosure of the invention

In the context of the invention, an image processing apparatus with the features of claim 1, a method for calibration with the features of claim 8 and a computer program with the features of claim 11 is proposed.

Preferred or advantageous embodiments of the invention will become apparent from the subclaims, the following description and the accompanying figures.

The invention relates to an image processing device which is suitable and / or designed as an additional module or as an integral part of a video surveillance system. The video surveillance system preferably includes a plurality of surveillance cameras directed to relevant surveillance areas in a real 3D scene. Depending on the actual event, at least one monitoring object, but also several monitoring objects, and at least one masking geometry or several, can be present in the surveillance area (s)

Masking geometries appear. The masking geometry may be formed as a temporary, quasi-stationary or stationary geometry. Concealment geometry is preferably understood to mean any object which leads or can lead to an optical concealment of the surveillance object. It may be formed, for example, as a wall, a bench, a cabinet, a shelf, a staircase, etc. The image processing device comprises a detection module, which is designed in terms of programming technology and / or circuitry to detect one of the monitored objects. The step of detecting is preferably also the segmentation and / or recognition and / or classification of the

Monitored object.

The detection takes place on the basis of a set of partial area detectors, which respectively detect different partial areas of the monitored object. The partial area detectors can be used, for example, in analogy to the partial area detectors in the scientific article by Sotelo et al. be educated. In one possible implementation of the invention, the subareas detectors are designed as classification devices which examine the image content of a search area, often designed as a rectangular bounding box, for the existence of the subarea.

The detection module is designed in terms of programming and / or circuitry such that the quantity of the subarea detectors after the evaluation are subdivided into a positive quantity and a negative quantity and optionally into a neutral quantity. The positive quantity are assigned to partial area detectors which have positively recognized the respective corresponding partial area of the monitored object. The negative quantity relates to the subarea detectors for which the associated subarea of the surveillance object was not recognized. Partial range detectors are classified in the optional neutral quantity, which have not provided any information for any other reason.

The invention now proposes that the image processing device has a calibration module, which is formed by circuitry and / or programming technology, based on a detected monitoring object with a negative quantity, a depth map and / or a masking geometry for the

Calibrate monitoring area, in particular to generate and / or to update.

It is a consideration of the invention that by additional knowledge or a priori knowledge, hereinafter also referred to as expert knowledge, on the surveillance object based on Negative quantity, that is based on unrecognized subregions of the monitoring object, is closed to properties of the monitoring area.

In particular, it is possible to deduce occlusion geometries which obscure the subareas of the surveillance object of the subarea detectors of the negative amount and / or properties of a depth map of the surveillance sig.

The depth map is preferably formed as a relation between the spatial and / or 3D properties of the surveillance area and the image of the surveillance area by the video camera, ie the 2D image of the surveillance scene in the image plane of the video camera. For example, each point of the monitored area, ie the surveillance scene, is assigned a real, three-dimensional object point in the surveillance area.

With regard to the calibration of masking geometries, the advantage of the image processing device can be seen in the high significance, since after a positive recognition of a partial area of the monitored object and a non-recognition of further partial areas of the monitored object, the last-mentioned partial areas must be covered. With regard to the depth map, there is an advantage in that the

Depth card can also be calibrated in monitoring areas where almost no floor surface of the monitoring area can be seen.

In a preferred embodiment of the invention, the amount of subregion detectors is specific and / or selective for a surveillance object type or genus. A corresponding surveillance object type are, for example, persons, in particular adult persons or children, cars, motorcycles, shopping carts, etc. Because the quantity of subarea detectors is specifically or selectively designed for a surveillance object type, expert knowledge of the interpretation of the detection results

Monitoring object type are introduced. For example, an average adult size of 1.80 m is assumed.

In a particularly preferred embodiment of the invention, the calibration module is formed, so that when a negative amount Masking geometry is identified or suspected in the region of the subset detectors of the negative set. Generally in the invention, and in particular in this embodiment, it is preferred if the subarea detectors of a common set assign a defined spatial relation to one another, which is given by the surveillance object type. For example, divide one person into several

Partial area detectors, which comprise a head detector, a body detector and a bed detector as subareas detectors, can, in the assignment of the head detector and the body detector to the positive quantities and the Beindetektors to the negative amount to a concealment geometry in the area of Beindetektors, ie vertically below the head. and / or the Beindetektors be closed.

The proposed image processing device thus preferably identifies or suspects a concealment line or edge as an edge line of a masking geometry between adjacent and / or adjacent and / or overlapping partial area detectors of a positive quantity and a negative quantity.

In a possible alternative or further development of the invention, in the case of a monitoring object with a negative quantity, the surveillance object is extrapolated using the positive quantity and a depth line or edge and / or a

Base point of this monitoring object generated. By way of example, when a person is detected by the detection of a head by a head detector and a body by a body detector, the person is extrapolated in length by taking averages of persons, for example, a height of 1.80 m. In this way, the probable position of the foot area of the person is closed. The conclusion of the presumed foot area is interpreted as the depth edge, ie the vertical edge of the person. After the monitoring object is perspectively larger or smaller depending on the position in the surveillance area, the extrapolation for the surveillance object is corrected by supplementary evaluation of the subarea detectors from the positive quantity by a perspective correction factor. For example, the perspective correction factor is determined by a specific head size on or via a relationship between the head and the body. In this case, it is generally particularly preferred if the monitoring object is extrapolated in a vertical, ground-facing direction of the surveillance area in order to generate the depth edge. With this procedure, the position information of the depth edge or the foot point, which is concealed in the current situation of the monitoring area by a Verdeckungsgeometrie, can be used to calibrate the depth map.

Another object of the invention relates to a method for calibration, in particular generation and / or updating, a depth map and / or concealment geometry in a surveillance area, preferably using the image processing apparatus according to one of the preceding claims or as just described with the features of the claim 8th.

In the method, in a first step, a set of partial area detectors is applied to a surveillance object, wherein the partial area detectors detect different partial areas of the surveillance object, in particular independently of one another. In one possible implementation, as a preparation for this step, first moving objects are segmented from the background, in particular the scene background, which takes place, for example, by subtracting a current image from a scene reference image. One or the set of partial area detectors is applied to each found object. In a scenario in which only persons can exist as a surveillance object type, a single set of subarea detectors suffices. For monitoring areas in which several monitoring object types can occur, several quantities can optionally be used.

In the detection and / or verification of the monitored object, the amount of

Partial area detectors divided into at least a positive amount and a negative amount. If the negative quantity is not empty, that is, if not all partial area detectors recognize "their partial area" of the monitored object, this indicates a masking at the position of the partial area detector. With regard to the said example with the person as monitoring object type are typically the

Legs and / or parts of the upper body hidden. In a first possible alternative, the hidden portions are suspected or identified as hidden by a masking geometry. In an alternative or a continuation of the method becomes the height, depth or vertical extent of the monitored object estimated by using expert knowledge of the dimensions or dimensions of the hidden subsections.

In a practical implementation of the invention, the set of partial area detectors is designed to detect a person. It is particularly preferred that the quantity of subarea detectors has at least one cardinality of two, wherein preferably at least four subareate detectors are arranged vertically one above the other with respect to the surveillance area. The subarea detectors may be adjacent, adjacent and / or overlapping each other.

A last subject of the invention relates to a computer program with program code means having the features of claim 11.

Brief description of the drawings

Further features, advantages and effects of the invention will become apparent from the description of preferred embodiments of the invention and the accompanying figures. Showing:

Figure l a - c schematic video images of a surveillance area with a

Monitoring object to illustrate the inventive

procedure;

FIG. 2 shows a schematic representation of a set of partial area detectors on

Example of a person;

Figure 3 is a schematic application example of the amount of

Subrange detectors in Figure 2;

Figure 4 is a block diagram illustrating an apparatus according to the invention.

Embodiment (s) of the invention Figures 1 a and 1 b show a schematic video image of a monitoring area 1, which is formed in the example shown as a store scene. In the shop scene, a moving surveillance object in the form of a person 2 is shown, which is arranged half hidden behind a shelf 3 and according to the figure 1 b uncovered next to the shelf 3 according to the figure 1 a.

In particular, for calibration of a depth map and / or masking geometries, it is interesting to determine a base point 4 of the moving monitoring object 2. The foot point 4 represents the free end of the vertical extent of the monitored object 2 in the direction of the ground. A typical approach to

Determination of the foot 4 is implemented by the moving monitoring object 2 is circumscribed by a so-called bounding box 5 and the center of the foot edge 6 of the bounding box 5 is interpreted as foot 4. Other monitoring objects 2 can also be detected by this procedure. The bounding box 5 is determined by known image processing algorithms, wherein, for example, in a first step, the moved monitoring object is separated from the substantially static background and then segmented. The segmented regions are subsequently circumscribed with search rectangles or bounding boxes 5, and the content of the search rectangles is verified as searched surveillance objects 2 via classification devices.

As can be seen from a comparison of FIGS. 1 a and 1 b, a correct foot point 4, which represents the vertical extent of the monitoring object 2 in the direction of the floor, could be determined in FIG. In FIG. 1a, however, the foot point 4 is not the actual foot point but a point which is arranged in the center of the body of the monitored object 2. Such inaccurate footsteps 4 as in FIG. 1a may lead to misinterpretation of the events in the surveillance area 1.

FIG. 2 shows a modified approach for detecting the surveillance object 2 as an exemplary embodiment of the invention, wherein subareas detectors 7 a to 7 e are used instead of a detector covering the entire surveillance object 2, each covering only a specific subarea of the surveillance object 2. Thus, the head detector 7 a for the detection of the head, the shoulder detector 7 b for detecting the shoulder area, the upper body detector 7 c for detection of the upper body, the center body detector 7 d for detection of the hip area and the foot detector 7e designed for the detection of the feet / legs. The subareas detectors 7 a to e are arranged overlapping in sections (compare subarea detectors 7 a to 7 c), partially adjoining (see subarea detectors 7 c to e) and partially spaced (not shown) and have a defined and / or content-based relative arrangement to each other. The partial area detectors 7a-e are arranged, for example, such that the head detector 7a can not lie between the central body detector 7e and the foot detector 7e.

FIG. 3 shows, in a schematic representation, the quantity of the subarea detectors 7 a to 7 e which was applied to one or the surveillance object 2 in the surveillance area 1 in FIG. Due to the masking by the shelf 3, only the partial area detectors 7 a to c detect the associated partial areas positively, the

However, partial area detectors 7 d and e can not recognize the assigned partial area. The partial area detectors 7 a to c are assigned to a positive quantity 8, the partial area detectors 7 d and e to a negative quantity 9.

From the distribution of the partial area detectors 7 a to 7 e into the positive quantity 8 or negative quantity 9, conclusions can be drawn about the geometry of the surveillance area 1 or of the surveillance object 2.

As a first conclusion, between the partial area detector 7c and the partial area detector 7d, that is, between a partial area detector of the positive quantity 8 and a partial area detector of the negative quantity 9 which are directly adjacent to each other, a concealment edge 10 can be identified or at least suspected. The concealment edge 10 describes an edge line of a concealment geometry, such as the shelf 3.

As an alternative or supplementary conclusion, the monitoring object 2 can be extrapolated or estimated in the direction of the foot 4 by deducing from the properties of the subareas detectors 7 a to c of the positive quantity 8 the properties of the hidden subareas detectors 7 d and e of the negative quantity 9. Possible information-rich properties are: the relative Arrangement of the partial area detectors 7 a to c of the positive quantity 8, length and / or height of the individual partial area detectors 7 a to c of the positive quantity 8, detailed information of the subareas of the monitored object 2 enclosed by the partial area detectors 7 a to 7 c of the positive quantity 8, an estimate of the depth position of, some or all of the partial area detectors 7 a to c of the positive quantity, etc.

By extrapolating or estimating the monitored object 2, it is possible, as shown in FIG. 1 c, to proceed from a partially concealed surveillance object

2 about an estimate on the Verdeckungskante 10 and 4 to close the foot. The knowledge of the concealment edge 10 can be used to supplement a collection about the concealment geometries in the surveillance area 1, the base point 4 can be used to calibrate a depth map, that is a relation between the course of a horizontal plane in the surveillance area 1 with the two-dimensional image of the surveillance area 1, be used.

FIG. 4 shows a schematic block diagram of a video surveillance system 11, which is connected to a plurality of surveillance cameras 12, which are designed and / or arranged to monitor one or more surveillance areas 1. The image data recorded by the surveillance cameras 12 are forwarded to an image processing device 13, which is designed to calibrate masking geometries or a depth map, respectively.

In a first step, in a segmentation module 14, moving monitoring objects 2 are segmented from the background of the surveillance area or zones 1. The segmentation is carried out, for example, by subtracting the current surveillance image from a scene reference image.

The monitoring objects 2 segmented by this step are respectively verified in a detection module 15 by a set of partial area detectors 7 a to 7 e. If only persons can or should be detected in the monitoring area 1 as monitoring objects 2, a single quantity of subarea detectors 7 a to e is sufficient, which are specific to the In the case of more complex monitoring areas 1, different sets of subareas detectors may also be used, each set being specifically or selectively tuned to a type of surveillance object, for example a first set for persons, a second set for bicycles, a third lot on cars, a fourth

Amount to be matched to children, etc.

At least one set of partial area detectors 7 a to e is executed on one, some or each segmented monitoring object 2, the partial area detectors 7 a to e depending on the result, for example a classification device of the partial area detector 7 a to e, into a positive quantity 8 or a negative quantity 9 are divided.

Do not detect all sub-area detectors 7 a to e a set the associated monitoring object 2, so in this example, the persons 2, this indicates a

Masking at the position of the monitoring object 2 in the monitoring area 1 out. Typically, the legs and / or parts of the upper body of the person 2 are covered. In this case, the data of the detected and partially concealed monitoring object 2 are transferred from the detection module 15 into a first obscuring geometry estimation module 16 and into a second footfall estimation module 17.

In the first estimation module 16 for the masking geometry, a cover edge 10 is used on the basis of the transitions or peripheral positions of the subarea detectors of the positive quantity 8 and the negative quantity 9. This derived

Masking edge 10 is transferred to a database 18 which collects information about masking geometries, thereby allowing modeling of the monitoring area 1 with respect to the masking geometries.

The second estimation module 17 extrapolates as described

Monitoring object 2 and passes the estimated foot point 4 together with the further information of the monitoring object 2 to a database 19, which includes information about a depth map, that is, about the depth extent of the monitored area monitored 1. Both databases 18 and 19 can so in the monitoring mode optionally calibrated for the first time, supplemented or constantly updated.

In summary, the proposed device or the proposed method allows the analysis of partial occlusions in the

To improve monitoring area 1 and at the same time to learn a depth graduation of the scene in the monitoring area 1. The thereby improved modeled scene geometry can be used in object tracking to make the automatic video surveillance system 11 more efficient.

Claims

claims

1. An image processing device (13) for a video surveillance system for monitoring a surveillance area (1), which may have at least one surveillance object (2) and at least one masking geometry (3),

with a detection module (15) which is designed to detect one of the monitoring objects (2),

wherein the detection of the surveillance object (2) is performed on the basis of a set of partial area detectors (7 a - e), the different partial areas of the

Monitoring object (2) detect, and wherein in the detection of the monitoring object (2) the amount of the subareas detectors (7 a - e) in a positive set (8) of subrange detectors, which have detected the associated portion of the surveillance object (2), and in a Negative quantity (9) of partial area detectors, which determine the assigned partial area of the monitored object

(2) have not been recognized, divided,

marked by

a calibration module (16,17), which is formed on the basis of a detected

Monitoring object (2) with a negative amount (9) to calibrate a depth map and / or concealment geometry for the surveillance area (1).

2. Image processing device (13) according to claim 1, characterized in that the amount of sub-area detectors (7 a-e) specifically and / or selectively for a

Monitoring object type or genus is formed.

3. image processing device (13) according to claim 1 or 2, characterized in that at a negative amount (9) a Verdeckungsgeometrie (3) in the Area of the partial area detectors of the negative amount (9) is identified and / or suspected.

4. image processing device (13) according to any one of the preceding claims, characterized in that between adjacent subareas detectors a

Positive amount (8) and a negative amount (9) a masking line (10) of a masking geometry is identified and / or suspected.

5. Image processing device (13) according to any one of the preceding claims, characterized in that in a monitoring object (2) with a

Negative quantity (9) the surveillance object is extrapolated using the positive set (8) and a depth line (6) is generated.

6. image processing apparatus (13) according to claim 5, characterized in that the monitoring object (2) in a vertical direction of the monitored area

(1) is extrapolated.

7. image processing device (13) according to claim 5 or 6, characterized in that the position information of the depth line (6) is used for calibration of the depth map.

8. A method of calibrating a depth map and / or masking geometry in a surveillance area (1) monitored by a video surveillance system, the surveillance area (1) having at least one surveillance object (2) and at least one masking geometry (3),

wherein a set of subarea detectors (7 ae) is applied to a surveillance object (2), the subarea detectors (7 ae) detecting different subareas of the surveillance object (2),

wherein the set of subarea detectors (7 a - e) into a positive set (8) of subarea detectors which the associated subarea of the surveillance object

(2) have been detected and subdivided into a negative set (9) of sub-area detectors that have not recognized the associated sub-area of the surveillance object, and wherein the depth map and / or the masking geometry for the monitoring area (1) is calibrated on the basis of a detected monitoring object (2) with a negative quantity (9).

9. image processing device (13) and / or method according to any one of the preceding claims, characterized in that the set of partial area detectors (7 a - e) for detecting a person (2) is formed.

10. image processing device (13) and / or method according to any one of the preceding claims, characterized in that the amount of sub-area detectors (7 a - e) has at least one cardinality of four.

Computer program with program code means for carrying out all the steps of the method according to one of the claims 8 to 10, when the program is executed on a computer and / or an image processing device (13) according to one of the preceding claims.