JP2010128727A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce erroneous detection by effectively detecting an intruding object in an image processor that detects an object which fulfills a prescribed condition based on an image. <P>SOLUTION: The first lines A1, A2 are set for the image that is the processing subject according to an operation carried out by a user by a first line setting means in the image processor. A second line setting means sets the second lines B1, B2-1, B2-2 according to the first lines set by the first line setting means which are separated from the first lines for the image that is the processing subject. The object detection means detects the object that crosses the first lines set by the first line setting means and crosses the second lines set by the second line setting means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing device such as a CCTV (Closed Circuit Television) device for monitoring an intruder (or an intruder, which may be the same hereinafter) based on an image (video), and more particularly, The present invention relates to an image processing apparatus that effectively detects (detects) an intruder.

FIG. 4 shows an example of a flowchart of an intruder detection condition setting method performed by the image processing apparatus.
FIG. 5 shows an example of an intruder monitoring target area.
A camera is installed at a monitoring place, and a monitoring video 201 captured by the camera is given as an input video of the image processing apparatus. The monitoring video 201 includes an in-site area 202 and an off-site area 211. Consider a case where it is desired to detect an object moved from the off-site area 211 to the on-site area 202.

Detection conditions are set according to the procedure shown in FIG.
First, in a direction detection line / detection direction setting step (step S11), a direction detection line D1 and a detection direction (determination direction) E1 are set. Similarly, in the direction detection line / detection direction setting step (step S12), the direction detection line D2 and the detection direction (determination direction) E2 are set.
The setting of these steps S11 and S12 is artificially performed by a user (person) operation.

The image processing apparatus performs detection based on the set conditions.
That is, when the locus 222-1 of the object 221-1 crosses the direction detection line D1 in the set detection direction E1, it is detected.
However, even if the locus 222-2 of the object 221-2 on the site enters the site for a moment due to the influence of noise or the like, and the direction detection line D1 crosses the set detection direction E1, it is detected. End up.
As described above, in the detection method of the present example, an object that straddles the direction detection line is detected even for a moment, and thus there are many false detections.

JP 2001-155263 A JP-A-2005-57743 JP 2008-176768 A

As described above, in an image processing apparatus, when detecting an intruder using a direction detection line, for example, immediate detection is required, but as a result, an object that crosses the direction detection line is detected even for a moment. For this reason, there is a problem that it may lead to misinformation (false detection).
The present invention has been made to solve such a conventional problem, and provides an image processing apparatus that can effectively detect an intruder, for example, can reduce erroneous detection. With the goal.

In order to achieve the above object, in the present invention, an image processing apparatus that detects an object that satisfies a predetermined condition based on an image has the following configuration.
That is, the first line setting means sets the first line for the image to be processed in accordance with the operation performed by the user (person). According to the first line set by the first line setting means, the second line setting means sets a second line separated from the first line to the image to be processed. Set for The object detecting means detects an object that crosses the first line set by the first line setting means and crosses the second line set by the second line setting means. In this case, the predetermined condition is “a condition for detecting an object that crosses the first line and crosses the second line”, but any other condition is also taken into consideration.

  Therefore, when the image processing is performed based on the image and an object satisfying a predetermined condition (an object on the image) is detected, when the first line is set by the user, the second line is set, An object that crosses both of these lines is detected, for example, an object that crosses only the first line is not detected, so that an intruder can be effectively detected (detected), and false detection can be reduced. it can.

Here, various images (video) may be used, and for example, a moving image can be used.
Further, as an object to be detected, for example, an arbitrary object such as an object, a person, or an animal may be used, and only a predetermined object based on size, speed, etc. (for example, only a person) May be detected.
Various conditions for detecting the object may be used, or a plurality of conditions may be used in combination.
Moreover, various things may be used as a 1st line or a 2nd line, for example, a straight line (line segment), a broken line, a curve, etc. can be used.

Various methods may be used as a method for setting the second line in accordance with the first line. For example, a predetermined distance (for example, an actual distance or a distance from the first line) may be used. , Apparent distance on the image, etc.) can be used to set the second line at a position separated in parallel, and for example, the moving direction of the object to be detected is set In such a case, it is possible to set a second line shifted in that direction with respect to the first line.
The method of setting the second line according to the first line is, for example, set and stored in advance in the memory of the image processing apparatus. As another configuration example, the second line such as a separation distance is used. An aspect may be used in which part or all of information necessary for setting the line is received from the user.

[Other configuration examples]
Other configuration examples will be shown below.
(1) The predetermined condition includes a condition of the moving direction of the object. In this case, an object moving in a direction suitable for the set moving direction condition is detected. The moving direction may be one direction or both directions. In addition, as a moving direction, for example, when crossing a line, a direction (a direction crossing the line) as to which side of the line is crossed from which side (opposite side) can be used.
In this case, for example, the second line is set at a position away from the first line by a predetermined distance in a direction that matches the set moving direction condition. One is set for one direction, and two is set for both directions.
Further, in the case of both directions, for example, a condition for detecting an object if the first line and one second line are crossed between the two second lines, or the first line is used. If the line and all of the two second lines are crossed, a condition for detecting an object can be used.

(2) The predetermined condition includes an area condition. For example, the condition of the exclusion area does not detect an object that exists in the exclusion area.
As a specific example, a first area setting unit that sets a first area for an image to be processed according to an operation performed by a user, and a first area set by the first area setting unit And a second area setting means for setting a second area obtained by reducing the area (first area) for the image to be processed, and a second area set by the second area setting means. Object detection means for detecting an object using a predetermined condition (for example, an exclusion area condition) based on the second area.
Here, various methods may be used as a method for setting the second area obtained by reducing the first area. For example, other areas of the first area (excluded area as an example) may be used. It is possible to set a second area in which the boundary that touches (as an example, an area not excluded) is reduced (separated from other areas).

(3) For example, an object can be detected when both the line condition and the area condition (combination) are satisfied.
In addition, when there are a plurality of line conditions, area conditions, and the like, the user can set a combination of conditions to be used for each condition.
[End of other configuration examples]

  As described above, according to the image processing apparatus of the present invention, an intruder can be detected effectively, and for example, erroneous detection can be reduced.

Embodiments according to the present invention will be described with reference to the drawings.
FIG. 6 shows a configuration example of the image processing apparatus 1 according to an embodiment of the present invention.
The image processing apparatus 1 of this example includes an input unit 11, an output unit 12, a storage unit 13, and a control unit 14.
The input unit 11 has a function of inputting a video (image) to be monitored from an external camera or the like, and a function of keys and a mouse for inputting various instructions (control) and information by a user (person) operation. And a function of inputting various instructions (control) and information signals from an external device.
The output unit 12 has a function of displaying video and various information on the screen to the user, a function of outputting various sounds (voices) to the user, and various instructions (control) to an external device. And a function for outputting information signals.

The storage unit 13 is configured using a memory, and has a function of storing various types of information such as images to be processed and detection conditions, a function of storing programs used for various types of processing, and the like. Yes.
The control unit 14 has a function of controlling various processes performed in the image processing apparatus 1. The control unit 14 has, for example, a CPU (Central Processing Unit), and executes various processes by executing various programs.

An example of the intruder detection process performed by the image processing apparatus 1 of the present example is shown.
FIG. 1 shows an example of a flowchart of an intruder detection condition setting method performed by the image processing apparatus 1 of this example.
FIG. 2 shows an example of an intruder monitoring target area.
In this example, a process of automatically duplicating the direction detection line used when detecting an object moved in a specific movement direction is performed.

A camera is installed at a monitoring place, and a monitoring video 101 captured by the camera is given to the image processing apparatus 1 as an input video.
In the monitoring video 101, only an object moved from the off-site area 111 to the in-site area Z1 is detected, and an object existing in the on-site area Z1 or an object moved from the on-site area Z1 to the off-site area 111 is not detected. Under such conditions, an intruder (as described above, or may be an intruder, and so on) is detected.

First, in a direction detection line / detection direction setting step (step S1), a direction detection line A1 and a detection direction (determination direction) C1 for detecting an object moving from the off-site area 111 to the in-site area Z1 are set. The detection direction C1 is one direction. When set, the determination direction detection line calculation step (step S2) is automatically executed, and the determination direction detection line B1 is set.
Here, the determination direction detection line B1 is several meters behind from the start point and end point of the set direction detection line A1 in the detection direction C1 (upward direction of the set direction detection line A1 in this example). It is obtained by calculating the position (for example, 1 m deep). In this example, the method of setting the determination direction detection line B1 (for example, a calculation formula) is set and stored in advance in the apparatus, and may be specified by the user as another configuration example.

The determination direction detection line B1 is calculated by, for example, one or more preset camera installation conditions (eg, camera depression angle, camera height, horizontal viewing angle, CCD size, focal length, etc. (See, for example, Patent Document 2, Patent Document 3 and Japanese Patent Application No. 2007-176039). In other words, the appearance of the image captured by the camera changes depending on the installation conditions of the camera, and is different from the actual arrangement and size (length, width, distance, etc.) of the object to be imaged (what is reflected in the image) Therefore, for example, the determination direction detection line B1 can be set in consideration of these differences (coordinate differences).
The calculation for converting the actual length such as m (meter) and the number of pixels on the screen can be performed in the same manner (see, for example, Patent Document 2 and Japanese Patent Application No. 2007-176039). ).

  Similarly, in the direction detection line / detection direction setting step (step S3), the direction detection line A2 and the detection direction (determination direction) C2 are set. The detection direction C2 is both directions. When set, the determination direction detection line calculation step (step S4) is automatically executed, and two determination direction detection lines B2-1 and B2-2 are set. Here, the determination direction detection lines B2-1 and B2-2 are calculated by the same method as the above-described determination direction detection line B1, but the detection direction C2 of the direction detection line A2 is both directions. Therefore, determination method detection lines B2-1 and B2-2 are automatically set on both sides (for example, left and right) of the set direction detection line A2.

  Next, in order to surely exclude objects existing in the site area Z1, in the detection exclusion area setting step (step S5), the detection exclusion area Z1 (in this example, the same area as the site area Z1) is set. To do. For example, it is possible to use a determination method in which an object detected in the detection exclusion area Z1 is not detected even when it crosses the direction detection line A1 and the determination direction detection line B1.

  The detection exclusion area Z1 is set, for example, by designating a plurality of points by the user. As a specific example, the detection exclusion area Z1 is set by designating points 131a, 131b, 131c, 131d, and 131e in order. In this example, an inner area surrounded by the plurality of points is set as the detection exclusion area Z1. When set, the determination exclusion area calculation step for determination (step S6) is executed, and among the points 131a, 131b, 131c, 131d, and 131e constituting the detection exclusion area Z1, points adjacent to the off-site area 111 With respect to 131c, 131d, and 131e, points 132c, 132d, and 132e at a position several m deep (for example, 1 m deep) are calculated, and the detection exclusion area Z2 for determination is set. In this example, the inner area surrounded by the points 131a and 131b where the new position is not calculated in the back and the points 132c, 132d and 132e where the new position is calculated is determined as the detection exclusion area Z2 for determination. Set as.

Here, the reason why the point 132c, the point 132d, and the point 132e are set to be several meters deep is to reduce detection leakage when the foot position of the object existing in the off-site area 111 is not accurately obtained. In this example, a method of setting such points (points constituting the detection exclusion area Z2 for determination) (for example, a calculation formula) is set and stored in advance in the apparatus. As another configuration example, It may be specified by the user.
In addition, as a calculation method of the point 132c, the point 132d, and the point 132e, for example, a method similar to the calculation method of the determination direction detection lines B1, B2-1, and B2-2 (for example, how the camera looks and how it is actually arranged) And the like) can be used.
The settings of steps S1, S3, and S5 are artificially performed by a user (person) operation, and the calculations of steps S2, S4, and S6 (creation of lines and areas) are automatically performed by the apparatus.

In the image processing apparatus 1, an intruder is detected based on the set condition by performing image processing on the monitoring video 101.
That is, in the monitoring video 101, when the trajectory 122-1 of the object 121-1 crosses both the direction detection line A1 and the determination direction detection line B1 in the set detection direction C1, an intruder (in this example, , Intruder) is detected. In addition, the trajectory 122-2 of the object 121-2 in the site area Z1 crosses the direction detection line A1 in the set detection direction C1, but does not cross the direction detection line B1 for determination. Not detected.

Next, a setting method when determining by combining the detection area and the direction detection line will be described.
FIG. 3 shows an example of a screen 141 for setting detection condition combinations.
First, in the detection setting area 151, whether or not each line or area is used as a detection condition is set by selecting ON or OFF (ON / OFF). Here, ON indicates that detection is performed, and OFF indicates that detection is not performed.

In this example, both the direction detection line A1 and the direction detection line A2 are set to ON in order to detect an object moving from the off-site area 111 to the in-site Z1. For this reason, when any of the direction detection lines A1 and A2 is crossed (including the determination direction detection line in a predetermined direction), the detection is performed.
In this example, the detection exclusion area Z1 is set in order to reliably exclude objects in the site, and even if there is an object in the detection exclusion area Z1 (in this example, the detection exclusion area Z2 for determination). Since detection is not performed, the detection condition is set to off.

Conditions that are determined in combination can be set in the detection condition combination setting area 152 for those that are set to ON in the detection setting area 151. For example, “undetected” is set in the detection exclusion area Z1 as a condition to be used in combination with the direction detection line A1. By performing this setting, an object that has crossed the direction detection line A1 (including the determination direction detection line in a predetermined direction) has detected in the past the detection exclusion area Z1 (in this example, the detection detection exclusion area Z2). If it is detected by, the object is not detected. In addition, an object that has crossed the direction detection line A1 (in a predetermined direction, including the direction detection line for determination) has not been detected in the detection exclusion area Z1 (in this example, the detection exclusion area Z2 for determination) in the past. If it is detected, it is detected as an intruder. Thereby, the object (for example, thing or person) which exists in the site area Z1 can be reliably excluded from the detection target.
In addition, in this example, “−” is set for the direction detection line A <b> 2 because a condition for use in combination is not provided.

  Here, whether or not each object exists in the detection exclusion area Z1 (in this example, the detection exclusion area Z2 for determination) can be detected by performing image processing of the monitoring video 101, for example. Each object can be identified based on, for example, its size and shape. In this example, information on an object that has previously existed in the detection exclusion area Z1 (in this example, the detection exclusion area Z2 for determination) is stored in the memory.

As described above, in the image processing apparatus 1 of this example (or the image processing software provided therein), for example, video signals obtained from a plurality of television cameras that capture images in the monitoring area are processed and designated. When detecting an intruder that moves in the specified direction, it has a function of automatically doubling the direction detection line and providing hysteresis.
In addition, the image processing apparatus 1 of this example (or the image processing software provided therein) has a plurality of detection conditions such as a detection area and a direction detection line. It has a function of making a final determination by combining the detection / non-detection states.

Therefore, in the image processing apparatus 1 of this example (or the image processing software provided therein), when the direction detection line is set when performing detection based on the moving direction, the direction detection line is automatically set. By performing the detection (determination) with duplexing and having hysteresis, it is possible to reduce erroneous detection of an object and to perform stable determination.
As a specific example, when a direction detection line or detection direction is set, a direction detection line for determination is automatically calculated at a position several meters away from the detection direction based on the set direction detection line or detection direction. By setting and detecting an object that crosses (exceeds) both of these two direction detection lines, it is possible to reduce false detections. For example, it is possible to cross one direction detection line for a moment. Object false detection can be prevented.
In addition, it is possible to perform settings that are more suited to the site by performing detection by combining detection conditions (detection / non-detection settings) using areas, direction detection lines, and the like.

  In the image processing apparatus 1 of this example, the first line setting means is configured by the function of setting the first line (in this example, the direction detection lines A1 and A2) according to the operation performed by the user. The second line setting means is configured by the function of setting the second line (in this example, the determination direction detection lines B1, B2-1, B2-2) according to the first line, The object detection means is configured by a function of detecting an object on the image (an object shown in the image) using both the first line and the second line.

Here, the configuration of the system and apparatus according to the present invention is not necessarily limited to the configuration described above, and various configurations may be used. The present invention can also be provided as, for example, a method or method for executing the processing according to the present invention, a program for realizing such a method or method, or a recording medium for recording the program. It is also possible to provide various systems and devices.
The application field of the present invention is not necessarily limited to the above-described fields, and the present invention can be applied to various fields.
In addition, as various processes performed in the system and apparatus according to the present invention, for example, the processor executes a control program stored in a ROM (Read Only Memory) in hardware resources including a processor and a memory. A controlled configuration may be used, and for example, each functional unit for executing the processing may be configured as an independent hardware circuit.
The present invention can also be understood as a computer-readable recording medium such as a floppy (registered trademark) disk or a CD (Compact Disc) -ROM storing the control program, and the program (itself). The processing according to the present invention can be performed by inputting the program from the recording medium to the computer and causing the processor to execute the program.

[Hereinafter, a part of disclosure content of Japanese Patent Application No. 2007-176039]
Hereinafter, as a reference, a part of the disclosure content of Japanese Patent Application No. 2007-176039 (some portions of the application content are not as they are) is shown. The following contents are mainly the coordinate conversion between the video viewed from the camera and the actual video (what is shown in the video), and the setting of the user's area, etc. (for example, lines and dots in this embodiment). It is a matter regarding. It can be used for the present invention as needed.
It should be noted that the following description is a reference, and the content of the present invention is considered to be sufficiently understood and implemented by the above description, but the following content may be referred to if necessary. Good.

First, the concepts (1) to (3) of the technical idea are shown.
(1) In an image processing apparatus that processes an image, an image of a first coordinate system (referred to as a camera coordinate system in this example) that is a coordinate system at the time of imaging by an imaging apparatus (for example, a camera) Display that displays the image (second coordinate system image) converted to an image of a second coordinate system (referred to as a scene coordinate system in this example) provided on a reference plane based on the arrangement Means for accepting setting information relating to the image displayed by the display means from the user, and setting information received by the accepting means becomes a reference that does not depend on the arrangement of the imaging device from the information in the second coordinate system. An image processing apparatus comprising: coordinate system conversion means for converting into information of a third coordinate system (referred to as a world coordinate system in this example) provided on a plane.
Therefore, the user (person) inputs setting information by looking at the image of the second coordinate system in which an image corresponding to the arrangement of each imaging device can be seen and there is no difference in apparent size depending on perspective. Thus, the setting information can be easily and intuitively understood (for example, visually) for the user, and the second coordinate system setting information does not depend on the arrangement of the imaging device. By converting into system setting information, it is possible to share setting information even in imaging apparatuses having different arrangements.
Thus, for example, even when a plurality of imaging devices having different arrangements are used or when the arrangement of the imaging devices is changed, the burden of work required for setting information such as an area for an image captured by the imaging device Can be reduced. In addition, the setting information can be efficiently set even when an imaging device is added, or when setting information is added or changed.

(2) In the image processing apparatus according to (1), the coordinate system conversion unit is a setting information of a third coordinate system acquired based on an image captured by one imaging apparatus having a predetermined arrangement. Is converted into information of a second coordinate system corresponding to an arrangement different from the predetermined arrangement, and the image processing apparatus converts the information converted by the coordinate system conversion means (setting information of the second coordinate system). Conversion information setting means for setting an image obtained by an imaging device different from the one imaging device having an arrangement different from the predetermined arrangement or an image obtained by the one imaging device having a changed arrangement. A featured image processing apparatus.
Therefore, by acquiring setting information of the third coordinate system and converting the information into information of the second coordinate system corresponding to an arbitrary arrangement (arrangement of the imaging device), for example, a plurality of pieces having different arrangements Even when the image pickup apparatus is used or when the arrangement of the image pickup apparatus is changed, it is possible to reduce a burden of work required for setting information such as a region for an image picked up by the image pickup apparatus.

(3) In the image processing apparatus according to (1) or (2), at least one of information specifying a region to be monitored and information regarding a detection target is used as the setting information regarding the image. An image processing apparatus.
Therefore, these pieces of information can be shared between the same or different imaging devices having different arrangements.
Note that various types of information may be used as the information regarding the detection target. For example, threshold information regarding the size, width, height, length, and the like of the target may be used. As an example, when the size or the like of the target is equal to or greater than the threshold (or may be equal to or smaller than the threshold or within a certain range (threshold indicating the range)), a process for detecting the target may be performed. it can.

Here, as a first coordinate system (camera coordinate system) that is a coordinate system at the time of imaging, for example, a coordinate system viewed from the viewpoint of an imaging device (for example, a camera) is used.
Further, as the second coordinate system (scene coordinate system) provided on the reference plane, for example, the ground plane is used as a reference plane, and the coordinates viewed from directly above the plane (ground) are used. A system is used. The second coordinate system may be different depending on the arrangement of the imaging device.
In addition, as a third coordinate system (world coordinate system) provided on the reference plane, for example, the ground plane is used as a reference plane, and coordinates viewed from directly above the plane (ground) are used. A system is used. The third coordinate system is constant without depending on the arrangement of the imaging device.

Note that the reference plane coordinate system (second coordinate system or third coordinate system) is set in advance by the user and stored in the memory of the image processing apparatus, for example. As a reference plane, for example, the ground plane viewed from directly above the ground is used. However, the ground plane is not completely flat or may not necessarily be completely ground plane. For some reasons, there may be a deviation in a practically effective level.
As a specific example, the second coordinate system is an initial viewpoint direction (or a predetermined direction such as a pan head direction) with the installation position as the origin and the height direction as the z-axis for each imaging device. The third coordinate system is composed of a predetermined orthogonal X-axis and Y-axis with the height direction as the Z-axis. .

The coordinate conversion between the first coordinate system (camera coordinate system) and the second coordinate system (scene coordinate system) uses, for example, parameters according to the installation status and imaging status of the imaging device (for example, camera). Can be done.
Further, since the second coordinate system (scene coordinate system) and the third coordinate system (world coordinate system) are provided on the same reference plane, for example, these coordinate transformations can be performed by rotation or parallel movement. Can be performed. Specifically, in order to correct the difference (positional difference) between the origin of the second coordinate system and the origin of the third coordinate system, the coordinate axis on the plane of the second coordinate system (for example, In order to correct the difference (angle difference) between the coordinate axes on the plane of the third coordinate system and the orthogonal coordinate axes x and y). When the second coordinate system and the third coordinate system have different reference planes and are parallel (that is, only the height direction is different), a process for moving in the height direction is added. These coordinate transformations can be performed.
Further, if necessary, coordinate conversion may be directly performed between the first coordinate system and the third coordinate system.

Specifically, in the image processing apparatus, for example, the arrangement of a plane coordinate system (second coordinate system or third coordinate system) serving as a set reference and an installed imaging apparatus (for example, a camera) It is possible to perform coordinate conversion with each other directly or indirectly using a conversion formula (formula) or the like with a coordinate system (first coordinate system) based on a focal length or the like.
Information such as the arrangement and focal length of the imaging device may be set in the image processing device by a user, or may be detected (automatically) based on information from the imaging device or the like. Alternatively, the image processing apparatus may control the arrangement and focal length of the imaging device, and information such as the arrangement and focal length of the imaging device realized by the control may be used.

  In addition, as an aspect of receiving information input by the user, for example, the user can operate the operation unit while viewing an image of the second coordinate system (or another coordinate system as another configuration example) displayed on the screen. By using a mode in which information can be input by operating, or a mode in which information such as area, size, length, and numerical values received from the user is displayed on the screen, convenience is provided to the user. It is possible to increase.

Next, a specific example will be shown. Note that the monitoring device is, for example, a device constituting the image processing device.
With reference to FIGS. 7A to 7F, the flow of region setting processing (which can also be applied to line and point setting processing, etc.) will be described.
FIG. 7A shows an example of the input image 1121 captured by the first imaging device in the image input processing step. An example of a polygonal monitoring area (dotted line) 1122 is shown.
FIG. 7B shows an example of an input image 1131 captured by the second imaging device in the image input processing step. An example of a polygonal monitoring area (dotted line) 1132 is shown.
FIG. 7C shows an example of the input image 1141 captured by the third imaging device in the image input processing step. An example of a polygonal monitoring area (dotted line) 1142 is shown.

In FIG. 7D, the input image 1121 is parallel to the ground on which the imaging device is installed, based on information such as the position of the first imaging device, the imaging angle, and the focal length of the first monitoring device. An example of a scene image 1151 of a scene coordinate system projected on a simple plane is shown. An example of the monitoring area (dotted line) 1152 is shown.
In FIG. 7E, the input image 1131 is parallel to the ground on which the imaging device is installed, based on information such as the position of the second imaging device, the imaging angle, and the focal length of the second monitoring device. An example of a scene image 1161 of a scene coordinate system projected on a simple plane is shown.
In FIG. 7F, for the third monitoring device, the input image 1141 is parallel to the ground on which the imaging device is installed based on information such as the position, imaging angle, and focal length of the third imaging device. An example of a scene image 1171 of a scene coordinate system projected on a simple plane is shown.

In this example, in the region setting process, first, input images 1121, 1131, and 1141 expressed in a camera coordinate system having different coordinate axes depending on the position and imaging angle of each imaging device are used. Are converted into scene images 1151, 1161, and 1171 expressed in the scene coordinate system depending on the image, and a desired monitoring area 1152 is set on the scene image (for example, the scene image 1151) expressed on the scene coordinate system. It will be realized. In this case, for each imaging device, the monitoring area can be set using an image that can be seen according to its position and imaging angle. There is no difference in the apparent size due to perspective, making it easy to set. In this case, the monitoring area setting operation is performed, for example, while a person watches a scene image of the scene coordinate system displayed on the screen of the display device.
In this case, the area information set in the scene coordinate system is converted into information in the world coordinate system and stored.

  As another example, in the region setting process, first, input images 1121, 1131, and 1141 expressed in a camera coordinate system having different coordinate axes depending on the position and imaging angle of each imaging device are used. The scene image can be expressed in the world coordinate system (corresponding to the scene images 1151, 1161, 1171) which is always constant without depending on the coordinate axis, and the scene image expressed on the world coordinate system (for example, This is realized by setting a desired monitoring area (corresponding to the monitoring area 1152) on the scene image 1151). In this case, the monitoring area setting operation is performed, for example, while a person watches a scene image of the world coordinate system displayed on the screen of the display device.

  Here, any input image captured under any conditions can be expressed in the same world coordinate system if converted to a scene image in the scene coordinate system or the world coordinate system. Alternatively, there is an advantage that a monitoring area set on the world coordinate system (for example, one corresponding to the monitoring area 1152) can be applied to all monitoring apparatuses that capture even a part of the same area.

Next, an example of a conversion processing method between the coordinates of the camera coordinate system and the coordinates of the world coordinate system will be described. In this example, this conversion process is performed by, for example, an image processing processor (CPU or the like).
In this example, the direct coordinate conversion from the camera coordinate system to the world coordinate system may be complicated in some cases, and the scene coordinate system may be useful for setting the area. A coordinate system obtained by rotating or translating the coordinate system and converting from the camera coordinate system to a scene coordinate system having the intersection of the vertically downward vector of the imaging device 21 and the ground as the origin, and then changing from the scene coordinate system to the world coordinate system The process of converting to
That is, in this example, conversion between the camera coordinate system and the scene coordinate system and conversion between the scene coordinate system and the world coordinate system are performed.

Specifically, in this example, a scene image that can be expressed in a world coordinate system projected on a plane parallel to the ground from an input image 1121 expressed in a camera coordinate system imaged by the imaging device 21. The conversion process to (corresponding to the scene image 1151) will be described. In addition, it is also possible to convert from the coordinates of the world coordinate system to the coordinates of the camera coordinate system by performing inverse conversion of this conversion.
In this example, conversion from the input image 1121 in the camera coordinate system to the scene image 1151 is performed by coordinate conversion from the camera coordinate system to the scene coordinate system. Further, since the scene coordinate system is an orthogonal coordinate system obtained by simply rotating or translating the world coordinate system, the scene image 1151 can be expressed on the world coordinate system by performing inverse transformation of the rotation or translation. Can do.

FIG. 8A shows an example of a state when the imaging device 21 is viewed in a direction horizontal to the ground (from the side).
FIG. 8B shows an example of a state when the imaging device 21 is viewed in a direction perpendicular to the ground (from directly above).
8A and 8B are expressed in the scene coordinate system. In this example, the scene coordinate system takes the x-axis and the y-axis on the ground plane as the point where the origin O intersects the reference plane (ground) vertically downward (z direction) of the imaging device 21, It is an orthogonal coordinate system taking the z axis in the vertical direction. In addition, an r-axis is provided in the line-of-sight direction (optical axis direction) in which the imaging device 21 faces.

In this example, represents the height of installation of the imaging device 21 are expressed on H, the depression angle of the imaging device 21 are expressed on theta _T, the viewing angle in the vertical direction of the image pickup device 21 in theta _V.
Also, it represents the viewing angle in the lateral direction of the image pickup device 21 in theta _H. Incidentally, the viewing angle theta _V in the vertical direction of the image pickup apparatus 21 can be calculated by the aspect ratio of the image pickup device (aspect ratio) from the viewing angle theta _H in the lateral direction of the image pickup device 21.
Further, in the field of view of the image captured by the imaging device 21, the horizontal distance from the imaging device 21 to the nearest place is represented by _LN , and the horizontal distance from the imaging device 21 to the farthest place is represented by _LF . Represents.

First, for example, the current direction of the electric pedestal (camera head) for changing the position and direction of the imaging device 21 (for example, the pan angle θ _P and the tilt angle θ _T with the front of the head as the origin) is acquired. The current focal length f of the imaging lens of the imaging device 21 is acquired. Based on the acquired information, the position of the imaging range is calculated.
A method for calculating the position of the imaging range will be described. In this example, in order to simplify the description, it is assumed that the monitoring target area is a plane and has no unevenness on the ground.
The angle of view θ _H that is the viewing angle in the horizontal direction of the imaging device 21 is obtained by (Equation 1).

Here, w is the horizontal width of, for example, a CCD element that is an image pickup element of the image pickup device 21. As an example, when an image pickup element of 1/3 inch (element size 4.8 mm × 3.6 mm) is used, w = 4.8 mm. When a 1/3 inch imaging device is used and the focal length of the imaging lens is f = 8.00 mm, the angle of view of the imaging device 21 is θ _H = 33.4 °. In other words, the field of view of the imaging device 21 has a range of 33.4 ° in the lateral direction.

Usually, the imaging device 21 is often installed at a higher position than the monitoring target area. Therefore, depending on the current direction theta _T of the camera platform, in the area beneath the imaging device 21 is a region that can not occur be imaged. This region appears in the range of the line-of-sight direction _LN from the imaging device 21. A predetermined region beyond that (the region between the distance L _N and the distance L _F ) enters the field of view of the imaging device 21.
The distances L _N and L _F will be described.
The angle of view θ _V that is the viewing angle in the vertical direction of the imaging device 21 is obtained by (Expression 2).

Here, h is the vertical width of, for example, a CCD element, which is an image pickup element of the image pickup device 21. For example, when a 1/3 inch image pickup element is used, h = 3.6 mm. In addition, when a 1/3 inch imaging device is used and the focal length of the imaging lens is f = 8.00 mm, the angle of view of the imaging device 21 is θ _V = 25.4 °. That is, the field of view of the imaging device 21 has a range of 25.4 ° in the vertical direction.
The distance L _N and the distance L _F are obtained by (Equation 3).

As a result, a range that can be imaged by the imaging device 21, and the internal region of the triangle surrounded by the terms of the point and the distance L _F of the imaging device 21 and the distance L _N shown in FIG. 8 (a), 8 The inside of the trapezoidal region surrounded by the points P1, P2, P3, and P4 shown in (b).
As an example, using a 1/3 inch imaging device, the focal length of the imaging lens is f = 8.00 mm, the current direction of the camera platform is θ _T = 30 °, and the installation height of the imaging device 21 is H = 5.0 m, L _N = 5.42 m and L _F = 16.1 m.

As described above, the position of the field-of-view range of the imaging device 21 is calculated by (Expression 1) to (Expression 3).
Based on this result, each position (x ′, y ′) on the image of the camera coordinate system included in the visual field range can be converted to the (x, y) coordinate of the scene coordinate system.
For example, when the pixel represented by (x ′, y ′) on the image is an image of the position of the point C in FIG. 8A, if the number of pixels in the vertical direction of the input image 1121 is 480 pixels, The depression angle θ _y for imaging the point C is expressed by (Equation 4).

In addition, the pixel (x ′, y ′) in the camera coordinates images, for example, the position of the point D in FIG. 8B (the spatial position is the same as the position of the point C in FIG. 8A). If the input image 1121 has a horizontal pixel count of 640, the rotation angle θ _{x at} which the point D is imaged is represented by (Equation 5).

Here, θ _P is the pan angle of the imaging device 21, and in this example, indicates the angle formed by the x axis and the optical axis of the imaging device 21. Therefore, a pixel at an arbitrary position represented by (x ′, y ′) on the input image 1121 in the camera coordinate system is expressed as (x, y) in the scene image 1151 in the scene coordinate system according to (Expression 6). Desired.

That is, the scene image 1151 is obtained by performing coordinate conversion of the input image 1121 according to (Expression 1) to (Expression 6). Note that the coordinates of the position of the visual field range can be changed by panning, tilting, or zooming of the imaging device 21. Further, the image converted into the scene coordinate system can be returned to the camera coordinate system by performing inverse conversion of the conversion expressed by (Expression 4) to (Expression 6).
Here, the first imaging device (imaging device 21) has been described, but the same coordinates as in (Equation 1) to (Equation 6) also apply to input images 1131 and 1141 in camera coordinates taken by other imaging devices. By performing the conversion, the image can be converted into scene images 1161 and 1171 expressed on the scene coordinate system.

Here, since the scene coordinate system obtained by the coordinate transformation method described above is a coordinate system having an origin at the intersection of the vertical direction from the position of each imaging device and the ground, each scene image 1151, 1161, 1171 The coordinate system is different.
For this reason, in order to view the scene images 1151, 1161, 1171 obtained from all the monitoring devices (all the imaging devices) in a unified coordinate system, the world coordinate system which is the only coordinate system from each scene coordinate system. Coordinate conversion to is required.

For example, suppose that the world coordinate system is an orthogonal coordinate system that takes the z _w axis vertically upward from the ground plane and the x _w axis and the y _w axis on the ground plane with respect to the origin O _w . Since each scene coordinate system is an orthogonal coordinate system that takes the x-axis and y-axis on the ground plane and takes the z-axis in the vertical direction of each imaging device, each scene coordinate system rotates or parallels the world coordinate system. It becomes the moved coordinate system. For this reason, the coordinates (x, y, z) of each scene coordinate system are represented by the position vector of the origin O ′ of the scene coordinate system on the world coordinate system as O _w O ′ = (q _x , q _y ) ^T Assuming that the rotation angle between the coordinate system and the scene coordinate system is θ, the coordinates (x _w , y _w , z _w ) of the world coordinate system are expressed by (Expression 7).

By (Expression 7), as shown in FIG. 9A, the scene image 1151 can be expressed on the world coordinate system.
9A shows a scene image 1151 represented on the world coordinate system (scene image 1181) and a polygonal monitoring area (dotted line) 1182 (corresponding to the monitoring area 1152 in FIG. 7D). ) Is shown.

In particular, the scene image 1181 on the world coordinate system (and the scene image 1151 on the scene image system) obtained in this way has a similar relationship with the map of the monitoring area, and for example, roads are displayed as parallel lines. Is done. In addition, the size of the plant and the size of the sign appearing in the image are proportional to the actual size (for example, metric units).
For example, in the conventional area designation method using an image of the camera coordinate system, there is a problem that the apparent size differs depending on the position on the image when an area of a certain distance from the building is designated. When specifying an area using a scene image (in an individual scene coordinate system for each imaging device or in a world coordinate system common to all imaging devices) as in this example, the position on the image The problem of different apparent sizes does not occur.

Next, a method for setting a monitoring area using scene coordinates (or world coordinates) by each method described above and a method for sharing monitoring area information between monitoring apparatuses will be described.
In order to set and share the monitoring area, first, the input images 1121, 1131, and 1141 captured by each monitoring device are converted into scene images 1151 and 1161 of the scene coordinate system by (Expression 1) to (Expression 6), respectively. , 1171. Next, for example, in one monitoring apparatus, an area desired to be monitored is set by, for example, a polygon 1152 on the scene image 1151 of the scene coordinate system using the input device. Then, according to (Expression 7), the coordinate system of the scene images 1151, 1161, 1171 and the set monitoring area 1152 of each scene coordinate system is converted into the world coordinate system.

FIG. 9B shows scene images 1191, 1192, 1193 (corresponding to scene images 1151, 1161, 1171) and a monitoring area (dotted line) 1194 (corresponding to the monitoring area 1152) expressed on the world coordinate system. An example is shown.
In the monitoring device, information of the monitoring area 1194 expressed in the world coordinate system is stored so that it can be shared with other monitoring devices, for example, by storing it in a storage device. And the monitoring apparatus transmits the information of the world coordinate of the monitoring area | region 1194 defined on the said monitoring apparatus to another monitoring apparatus via a network as needed.

  In this example, the case where the world coordinate information of the monitoring area 1194 is transmitted via a network connected by an external I / F circuit, for example, is shown. However, as another configuration example, the information is connected to an external I / F circuit or the like. It is also possible to perform transmission using a portable medium such as a CD-RAM.

Next, for example, each monitoring apparatus is instructed to set a monitoring area based on the received world coordinate information of the monitoring area 1194 using an input device or the like. In response to this, each monitoring apparatus performs area setting based on the received information of the monitoring area 1194 by the image processor included in each monitoring apparatus.
Specifically, since the monitoring area 1194 and the scene images 1192 and 1193 can be expressed as shown in FIG. 9B on the world coordinate system, the respective scene images 1192 and 1193 and the monitoring area 1194 overlap. The coordinates of the area to be set are set as the monitoring area in each monitoring device. That is, a monitoring area equivalent to the monitoring area 1194 determined on one monitoring apparatus (corresponding to the monitoring area 1152) can be shared and set by other monitoring apparatuses. Further, the setting information of the monitoring area on the world coordinate system can be set by converting into the information on the scene coordinate system and the information on the camera coordinate system corresponding to each monitoring device.

For example, when setting a monitoring area for a plurality of monitoring devices by a conventional method, an input device such as a mouse is used for each of the images captured by all the imaging devices to be monitored, It is necessary to set the monitoring area directly and manually like the polygons 1122, 1132, and 1142 in FIGS. 7A, 7B, and 7C, for example. The more it took, the more time and effort it took to set the area.
On the other hand, in the case of this example, the scene image of the scene coordinate system (or the world coordinate system) obtained by one monitoring device, for example, a polygon 1152 in FIG. If the monitoring area is set, regardless of the number of monitoring apparatuses, it is possible to automatically and instantaneously set a monitoring area equal to the monitoring area for all other monitoring apparatuses.
[Part of the disclosure of Japanese Patent Application No. 2007-176039]

It is a figure which shows an example of the process sequence of the detection condition setting method which concerns on one Example of this invention. It is a figure which shows an example of the monitoring object area which concerns on one Example of this invention. It is a figure which shows an example of the screen for the detection condition combination setting which concerns on one Example of this invention. It is a figure which shows an example of the process sequence of the detection condition setting method which concerns on background art. It is a figure which shows an example of the monitoring object area which concerns on background art. It is a figure which shows the structural example of the image processing apparatus which concerns on one Example of this invention. (A) is a figure which shows an example of the input image imaged by the 1st imaging device, (b) is a figure which shows an example of the input image imaged by the 2nd imaging device, (c) is It is a figure which shows an example of the input image imaged by the 3rd imaging device, (d) is a figure which shows an example of the scene image by a 1st monitoring apparatus, (e) is a scene by a 2nd monitoring apparatus. It is a figure which shows an example of an image, (f) is a figure which shows an example of the scene image by a 3rd monitoring apparatus. (A) is a figure for demonstrating coordinate transformation, (b) is a figure for demonstrating coordinate transformation. (A) is a figure for demonstrating the method of area | region designation | designated, (b) is a figure for demonstrating the method of area | region designation | designated.

Explanation of symbols

1 .. Image processing device 11.. Input section 12.. Output section 13.. Storage section 14.
101 .. surveillance video, 111 .. off-site area, 121-1, 121-2 .. object, 122-1, 122-2 .. trajectory of object, 131 a to 131 e, 132 c to 132 d .. point, A 1, A2 ... Direction detection line, B1, B2-1, B2-2 ... Determination direction detection line, C1, C2 ... Detection direction (judgment direction), Z1 ... Site area (in this example, detection exclusion) Same as the area), Z2 ...
141 .. screen, 151 .. detection setting area, 152 .. detection condition combination setting area,
201 .. surveillance video, 202 .. site area, 211 .. off-site area, 221-1, 221-2 .. object, 222-1, 222-2 .. object trajectory, D1, D2 .. direction Detection line, E1, E2, ... Detection direction (judgment direction),

Claims (1)

In an image processing apparatus that detects an object that satisfies a predetermined condition based on an image,
First line setting means for setting a first line for an image to be processed in accordance with an operation performed by a user;
Second line setting means for setting a second line separated from the first line according to the first line set by the first line setting means for the image to be processed;
Object detecting means for detecting an object that crosses the first line set by the first line setting means and crosses the second line set by the second line setting means;
An image processing apparatus comprising:

Images