JP4976939B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that sets information such as a region for an image captured by an imaging apparatus, and particularly, for example, when a plurality of imaging apparatuses having different arrangements are used or the arrangement of imaging apparatuses is changed. Even in such a case, the present invention relates to an image processing apparatus that reduces the burden of work required for setting information such as areas.

  For example, an image (input image) obtained by imaging a monitoring area by an imaging device equipped with a solid-state imaging device such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) is processed by an image processing function. Monitoring devices that automatically detect a predetermined object have been widely used.

As an example of a method for automatically detecting a predetermined object in such an input image, a method called a difference method has been conventionally used.
With reference to FIG. 12, the basic process of the difference method will be described.
FIG. 12 shows an example of an outline of the object detection method based on the difference method.
First, there are an input image 301 and a background image 302 prepared in advance. The differentiator 303 calculates a difference for each pixel between the input image 301 and the background image 302 and calculates a difference image 304. Here, a humanoid object (object to be detected) 311 shown in the input image 301 is detected as a region 312 where a difference has occurred in the difference image 304. Next, the binarizer 305 compares the value of each pixel of the difference image 304 with a predetermined threshold value (for example, a threshold value 20 assuming that one pixel is 8 bits), and is equal to or higher than the threshold value. The pixel value of “255” is set to “255” while the pixel value less than the threshold value is set to “0”. Thereby, the binarized image 306 is obtained. Here, a region 312 in which a difference is generated in the difference image 304 is detected as a binarized pixel (detected object) 313, and a binarized pixel block is numbered by a process such as labeling. As a result, a region 314 where the detected object exists is obtained.

  In such a monitoring apparatus, it is necessary to set in advance information (region information) as to whether an object existing in which region is to be detected in the captured input image. In the conventional monitoring device, in order to perform such area setting, an input image is displayed on a display device, and for example, a person uses a pointing device such as a mouse to set a polygon vertex on the display screen. Specify the desired area.

With reference to FIG. 13A and FIG. 13B, the state of such region setting will be described.
FIG. 13A shows an example of an input image 321 by the imaging apparatus, and a building 322 is shown.
FIG. 13B shows a polygonal monitoring area 331 designated and set in the input image 321 shown in FIG.
For example, when an object approaching the building 322 in the input image 321 is detected, a predetermined pointing device is used for the input image 321 as shown in FIG. A region (a dotted line portion in FIG. 13B) is designated. An object approaching the building 332 can be detected by determining whether or not an object is detected in the polygon 331 specified in this way by a method such as the difference method described above.

In addition, a monitoring method that improves the accuracy of monitoring and expands the monitoring range by arranging a plurality of the above-described monitoring devices under conditions where the positions and angles of the imaging devices are different has been conventionally used.
Such a monitoring method will be described with reference to FIGS. 1 and 5A to 5C. 1 and 5 (a) to 5 (c) are referred to in examples according to the present invention described later, but there is no intention to unnecessarily limit the present invention here.
FIG. 1 shows an example of a sketch 101 in which the monitoring area is looked down from a direction perpendicular to the ground.
In this example, it is assumed that an object entering the polygonal monitoring area 114 is monitored using three monitoring devices. A region is imaged using the imaging device 111, the imaging device 112, and the imaging device 113 arranged as shown in FIG.

5A, 5B, and 5C show examples of video images (input images) 121, 131, and 141 captured by the imaging device 111, the imaging device 112, and the imaging device 113, respectively. It is shown.
In such a monitoring method, for each video (input image), area information is designated as polygons 122, 132, 142, and automatic detection of an object that enters the designated area by the difference method is performed for each monitoring apparatus. To do. By observing the detection results of the respective monitoring devices, it is possible to realize wide-area monitoring by reducing the blind spots in the entire monitoring area, rather than monitoring by one monitoring device.

JP 2006-211696 A

However, in the operation of designating a desired area by the above-described method, since the appearance of the area on the image varies depending on the location and angle of the imaging device, the monitoring device to be used is added or monitored. There is a problem that a new area has to be designated every time the location or angle of the apparatus is changed.
In the area setting operation, for example, the apparent distance of a building or the like is smaller in a place farther from the imaging device (for example, the upper portion of the image) than in a location closer to the imaging device (for example, the lower portion of the image). The work for setting the place away from the imaging device is a work that requires skill and time, such as a detailed and difficult work. It is a heavy burden on the user of the monitoring device that such work is required in association with increasing the number of monitoring devices (imaging devices) or changing the position of the imaging device.

As described above, in the conventional monitoring apparatus and area setting method, area setting is required for the input image obtained by the imaging apparatus. However, the setting operation is performed on an image whose apparent size changes depending on the distance from the camera. In addition, it is a difficult task to set one or more area information, and when monitoring using a plurality of monitoring devices, it is necessary to set the same region information for all the monitoring devices. As a result, there was a problem that the amount of work increased significantly. Further, even when the operation method of the monitoring device is changed, such as changing the location or angle of the imaging device, there is a problem that the region setting operation must be performed again.
Further, the information to be set is not limited to the information on the area, but includes information on an object to be detected, for example, and the burden of setting work is large.

  The present invention has been made in view of such conventional circumstances. For example, even when a plurality of imaging devices having different arrangements are used or when the arrangement of the imaging devices is changed, imaging is performed by the imaging device. An object of the present invention is to provide an image processing apparatus capable of reducing the work load required for setting information such as an area for an image to be processed.

In order to achieve the above object, in the present invention, an image processing apparatus for processing an image has the following configuration.
In other words, the display means has the second coordinates provided on the plane on which the image of the first coordinate system (camera coordinate system), which is the coordinate system at the time of imaging by the imaging device, is based on the arrangement of the imaging device. The image (second coordinate system image) converted into an image of the system (scene coordinate system) is displayed. The accepting means accepts setting information related to the image displayed by the display means from the user. The coordinate system converting means converts the setting information received by the receiving means from the information of the second coordinate system to a third coordinate system (world coordinate system) provided on a reference plane that does not depend on the arrangement of the imaging device. System) information.

Therefore, the user (person) inputs setting information by looking at the image of the second coordinate system in which an image corresponding to the arrangement of each imaging device can be seen and there is no difference in apparent size depending on perspective. Thus, the setting information can be easily and intuitively understood (for example, visually) for the user, and the second coordinate system setting information does not depend on the arrangement of the imaging device. By converting into system setting information, it is possible to share setting information even in imaging apparatuses having different arrangements.
Thereby, for example, even when a plurality of imaging devices having different arrangements are used or when the arrangement of the imaging devices is changed, the burden of work required for setting information such as an area for an image captured by the imaging device Can be reduced. Also, the setting information can be set efficiently when an imaging device is added, or when setting information is added or changed.

  Here, various configurations may be used as the configuration of the image processing device. For example, when a plurality of imaging devices are installed in respective arrangements, different processing devices (for example, each imaging device) The image processing apparatus can be configured by using the plurality of processing apparatuses, or the processing apparatus can be configured as a configuration in which the plurality of imaging apparatuses are managed by one processing apparatus. It can be regarded as a processing device. In addition to the imaging device and its processing device (for example, a monitoring device), a storage device separate from the processing device, a display device separate from the processing device, a server device separate from the processing device, etc. Can be included in the image processing apparatus.

In addition, when the image processing apparatus is composed of a plurality of apparatuses, each apparatus may be provided with an arbitrary functional unit. When it is necessary to exchange information between different apparatuses, The information is communicated (input / output).
In addition, as an imaging device which images an image, for example, the image processing device may be provided, or an image processing device provided outside the image processing device and input by the imaging device is input to the image processing device. Also good.

  In addition, various types of processing may be performed as image processing. For example, processing for detecting a predetermined target in an image can be performed. Such image processing may be performed in an arbitrary coordinate system. As an example, when setting information and arrangement information of an imaging device are displayed on the world coordinate system, when there are a plurality of setting information and a plurality of imaging devices, it is easy for a person to grasp them visually and collectively. can do.

Further, the arrangement of the imaging device includes, for example, an installation position and a viewpoint direction (for example, an imaging angle in the installation state).
Also, as position information, for example, absolute position information such as latitude and longitude can be used, or relative position information is used with reference to the position of one imaging device or the like. You can also.

Further, as a first coordinate system (camera coordinate system) that is a coordinate system at the time of imaging, for example, a coordinate system viewed from the viewpoint of an imaging device (for example, a camera) is used.
Further, as the second coordinate system (scene coordinate system) provided on the reference plane, for example, the ground plane is used as a reference plane, and the coordinates viewed from directly above the plane (ground) are used. A system is used. The second coordinate system may be different depending on the arrangement of the imaging device.
In addition, as a third coordinate system (world coordinate system) provided on the reference plane, for example, the ground plane is used as a reference plane, and coordinates viewed from directly above the plane (ground) are used. A system is used. The third coordinate system is constant without depending on the arrangement of the imaging device.

Note that the reference plane coordinate system (second coordinate system or third coordinate system) is set in advance by the user and stored in the memory of the image processing apparatus, for example. As a reference plane, for example, the ground plane viewed from directly above the ground is used. However, the ground plane is not completely flat or may not necessarily be completely ground plane. For some reasons, there may be a deviation in a practically effective level.
As a specific example, the second coordinate system is an initial viewpoint direction (or a predetermined direction such as a pan head direction) with the installation position as the origin and the height direction as the z-axis for each imaging device. The third coordinate system is composed of a predetermined orthogonal X-axis and Y-axis with the height direction as the Z-axis. .

The coordinate conversion between the first coordinate system (camera coordinate system) and the second coordinate system (scene coordinate system) uses, for example, parameters according to the installation status and imaging status of the imaging device (for example, camera). Can be done.
Further, since the second coordinate system (scene coordinate system) and the third coordinate system (world coordinate system) are provided on the same reference plane, for example, these coordinate transformations can be performed by rotation or parallel movement. Can be performed. Specifically, in order to correct the difference (positional difference) between the origin of the second coordinate system and the origin of the third coordinate system, the coordinate axis on the plane of the second coordinate system (for example, In order to correct the difference (angle difference) between the coordinate axes on the plane of the third coordinate system and the orthogonal coordinate axes x and y). When the second coordinate system and the third coordinate system have different reference planes and are parallel (that is, only the height direction is different), a process for moving in the height direction is added. These coordinate transformations can be performed.
Further, if necessary, coordinate conversion may be directly performed between the first coordinate system and the third coordinate system.

Specifically, in the image processing apparatus, for example, the arrangement of a plane coordinate system (second coordinate system or third coordinate system) serving as a set reference and an installed imaging apparatus (for example, a camera) It is possible to perform coordinate conversion with each other directly or indirectly using a conversion formula (formula) or the like with a coordinate system (first coordinate system) based on a focal length or the like.
Information such as the arrangement and focal length of the imaging device may be set in the image processing device by a user, or may be detected (automatically) based on information from the imaging device or the like. Alternatively, the image processing apparatus may control the arrangement and focal length of the imaging device, and information such as the arrangement and focal length of the imaging device realized by the control may be used.

The means for receiving information input by the user can be configured using an operation unit such as a mouse or a keyboard operated by the user, for example.
In addition, as an aspect of receiving information input by the user, for example, the user can operate the operation unit while viewing an image of the second coordinate system (or another coordinate system as another configuration example) displayed on the screen. By using a mode in which information can be input by operating, or a mode in which information such as area, size, length, and numerical values received from the user is displayed on the screen, convenience is provided to the user. It is possible to increase.

Moreover, as a means to display an image, it can comprise using the display apparatus which displays and outputs an image on a screen, for example.
Various information may be used as setting information received from the user. For example, numerical values such as parameters used when processing an image and information on various conditions can be used.

The image processing apparatus according to the present invention has the following configuration as one configuration example.
That is, the coordinate system conversion unit corresponds to the setting information of the third coordinate system acquired based on the image captured by one imaging device having a predetermined arrangement, in an arrangement different from the predetermined arrangement. Conversion into information of the second coordinate system.
Further, in the image processing apparatus, the conversion information setting unit is configured to convert the information (second coordinate system setting information) converted by the coordinate system conversion unit into the one image pickup having an arrangement different from the predetermined arrangement. The setting is made with respect to an image obtained by an imaging device different from the device or the one imaging device whose arrangement is changed.

  Therefore, by acquiring setting information of the third coordinate system and converting the information into information of the second coordinate system corresponding to an arbitrary arrangement (arrangement of the imaging device), for example, a plurality of pieces having different arrangements Even when the image pickup apparatus is used or when the arrangement of the image pickup apparatus is changed, it is possible to reduce a burden of work required for setting information such as a region for an image picked up by the image pickup apparatus.

  Here, as an aspect of setting setting information acquired based on an image captured by a certain imaging device with respect to images obtained by the same or different imaging devices having different arrangements, setting information equivalent to the setting information is included. For example, the setting information equivalent to the setting information is set in a processing device (for example, a monitoring device) that processes the image.

In addition, here, the setting information of the third coordinate system is converted into the setting information of the second coordinate system and set. However, as another configuration example, the setting information of the third coordinate system is ( For example, an aspect may be used in which the information is converted into the information of the first coordinate system and set (via the second coordinate system).
Usually, when setting information of the second coordinate system is set, an image of the second coordinate system is processed using the setting information, and setting information of the first coordinate system is set The image of the first coordinate system is processed using the setting information. Further, the third coordinate system image may be processed using the setting information of the third coordinate system.

The image processing apparatus according to the present invention has the following configuration as one configuration example.
That is, at least one of information specifying a region to be monitored and information regarding a detection target is used as the setting information regarding the image.
Therefore, these pieces of information can be shared between the same or different imaging devices having different arrangements.

Here, various areas may be used as the area to be monitored. For example, an area for detecting an object shown in an image can be used.
As a detection target, for example, a predetermined object shown in an image can be used.
Various information may be used as the information regarding the detection target. For example, threshold information regarding the size, width, height, length, and the like of the target can be used. As an example, when the size or the like of the target is equal to or greater than the threshold (or may be equal to or smaller than the threshold or within a certain range (threshold indicating the range)), a process for detecting the target may be performed. it can.
Further, the setting information regarding the image is not limited to these, and various types of information may be used.

The present invention can also be provided as a method, a program, a recording medium, and the like.
In the method according to the present invention, each unit executes various processes in the apparatus or system.
The program according to the present invention is executed by a computer constituting the apparatus or system, and causes the computer to function as various means.
In the recording medium according to the present invention, a program to be executed by a computer constituting an apparatus or system is recorded so as to be readable by input means of the computer, and the program executes various processes (procedures) on the computer. Let

  As described above, according to the image processing device of the present invention, for example, even when a plurality of imaging devices having different arrangements are used or when the arrangement of the imaging devices is changed, images are taken by the imaging device. It is possible to reduce the work burden required for setting information such as an area for an image.

  Embodiments according to the present invention will be described with reference to the drawings.

A first embodiment of the present invention will be described.
In this example, an example of a monitoring system that performs monitoring using at least one (one or more) monitoring device is shown.
FIG. 1 shows an example of a layout diagram 101 of the monitoring area monitored by the monitoring system of this example and its surroundings. This sketch 101 is a view of the monitoring area looking down from a direction perpendicular to the ground.
Specifically, there are roads, buildings, and vacant lots, the arrangement (position and direction) of the first imaging device 111 provided in the first monitoring device, and the second imaging provided in the second monitoring device. The arrangement of the device 112, the arrangement of the third imaging device 113 provided in the third monitoring device, and the polygonal monitoring area 114 set in the vicinity of the building are shown.

FIG. 2 shows a configuration example of the monitoring system of this example.
The monitoring system 1 of this example includes three monitoring devices 11, 12, and 13 (a first monitoring device 11, a second monitoring device 12, and a third monitoring device 13). It is connected.
In the present example, each of the monitoring devices 11, 12, and 13 is a device that images a region to be monitored and automatically detects an object that enters the region. For example, TCP (Transmission Control Protocol) / IP (Internet Protocol). Are respectively connected via the network 14 using the protocol (1), and the monitoring devices 11, 12, and 13 are monitored by different positions, angles, focal lengths of the lenses, and the like. The imaging devices 111, 112, and 113 in each of the monitoring devices 11, 12, and 13 are arranged as shown in FIG.

FIG. 3 shows a configuration example of the monitoring device 11. In this example, the configuration and operation of the other monitoring devices 12 and 13 are the same as the configuration and operation of the monitoring device 11.
The monitoring device 11 of this example includes an imaging device 21 configured using, for example, a camera, a video input circuit 22, an image processing processor 23, a program memory 24, a work memory 25, and a network 14 An external I / F circuit 26 that is indirectly connected to an external device through a video output circuit 27, a display device 28 having a display screen, for example, and a keyboard or a mouse operated by a person An input device 29, an input circuit 30, a storage device 31 configured using a memory, for example, and a data bus 32 are provided. The video input circuit 22, image processor 23, program memory 24, work memory 25, external I / F circuit 26, video output circuit 27, input circuit 30, and storage device 31 are connected to a data bus 32.
The configuration of the imaging device 21 includes, for example, a conventionally known technique having components such as a device for controlling the imaging device 21 and various external recording devices. In order to simplify the description, illustration and detailed description thereof are omitted.

An example of the operation performed in the monitoring apparatus 11 of this example is shown.
The imaging device 21 images a region to be monitored. The video (input image) captured by the imaging device 21 is recorded in the work memory 25 via the video input circuit 22.
The image processor 23 processes the input image recorded in the work memory 25 according to the program recorded in the program memory 24 (for example, object detection processing), and displays the processing result via the video output circuit 27. Display on device 28. In addition to the object detection process, the image processor 23 executes various processes.

Also, the image processor 23 changes or corrects the parameters of the program based on an instruction input from the external I / F circuit 26 or an instruction input from the input device 29 via the input circuit 30. Perform image processing. Here, as such an instruction, for example, an instruction from an operator (person) using an external device having an input function connected via the input device 29 or the external I / F circuit 26 is used.
The storage device 31 stores information on a region desired to be monitored and stores information on a monitoring result output by the monitoring device 11. The area information or the like may be recorded in the work memory 25.

FIG. 4 shows an example of the processing flow of the program executed by the image processor 23.
In FIG. 4, in order to make the explanation easy to understand, in addition to the processing steps S <b> 1 to S <b> 9, the background image T <b> 1 and the region information T <b> 2 recorded in the work memory 25 (or the storage device 31), And an input device 29 (or an input function from outside the monitoring device 11 via the external I / F circuit 26) and a storage device 31 (or outside the monitoring device 11 via the external I / F circuit 26). It may be a storage device). In FIG. 4, arrows indicated by dotted lines represent the flow of data.

First, in the image input processing step, a process of transferring a video (image) captured by the imaging device 21 to the work memory 25 via the video input circuit 22 (step S1).
Next, in the setting determination processing step, it is determined whether or not there is an operator setting instruction from the input device 29. If there is such a setting instruction, the process branches to an area setting processing step (step S9). If there is no such setting instruction, the process branches to a difference processing step (step S3) (step S2).
Here, the instruction regarding the setting of the area from the input device 29 is realized by, for example, an operator specifying arbitrary information as the area information or referring to the area information recorded in advance in the storage device 31. To do.

In the difference processing step, the input image (input image 301 in FIG. 12) input in the image input processing step (step S1) and the background image T1 recorded in the work memory 25 (background image 302 in FIG. 12). And a difference image (difference image 304 in FIG. 12) is obtained (step S3).
Next, in the binarization processing step, the value of each pixel of the difference image is compared with a predetermined threshold value (for example, a threshold value 20 assuming 8 bits per pixel), Conversion is performed in which the pixel value is set to “255” and the pixel value less than the threshold value is set to “0” (step S4). Thereby, a binarized image (binarized image 306 in FIG. 12) is obtained.

Next, in the labeling processing step, a number is assigned to the cluster of pixels having the pixel value “255” in the binarized image so that each cluster can be distinguished and processed (step S5).
Next, in the background update processing step, the background image T1 recorded in the work memory 25 is updated using the input image obtained in the image input processing step (step S1) (step S6).
Here, as the update of the background image, for example, a new background image T1 can be created by using a method such as moving average for each pixel of the background image T1 and the input image. The background image T1 can always be kept up-to-date following the change in illuminance due to the above.

Next, in the area determination processing step, based on the area information T2 recorded in the work memory 25, for example, it is determined whether or not a binarized image labeling cluster exists in the specified area. If it exists, it is determined that the cluster is an intruding object (step S7).
Next, in the processing result display step, the determination result of the region determination processing step (step S7) is displayed, for example, on the screen of the display device 28 (step S8). When the processing result display step is completed, the process branches to the image input processing step (step S1).

On the other hand, in the above-described setting determination processing step (step S2), in the region setting processing step that is branched when it is determined that there is a setting instruction from the operator or the like by the input device 29, the region information T2 is set (step S9). ).
In this example, the setting of the region information T2 has been described. However, for example, other information related to object detection may be set.

Here, the flow of the region setting process will be described with reference to FIGS. 5 (a) to 5 (f).
FIG. 5A shows an example of the input image 121 captured by the first imaging device 111 (the imaging device 21 in FIG. 3) in the image input processing step (step S1). An example of a polygonal monitoring area (dotted line) 122 is shown.
FIG. 5B shows an example of the input image 131 captured by the second imaging device 112 (the imaging device 21 in FIG. 3) in the image input processing step (step S1). An example of a polygonal monitoring area (dotted line) 132 is shown.
FIG. 5C shows an example of the input image 141 captured by the third imaging device 113 (the imaging device 21 in FIG. 3) in the image input processing step (step S1). An example of a polygonal monitoring area (dotted line) 142 is shown.

FIG. 5D shows the input image 121 with respect to the ground on which the imaging device 111 is installed, based on information such as the position of the first imaging device 111, the imaging angle, and the focal length of the first monitoring device. An example of a scene image 151 of a scene coordinate system projected onto a plane parallel to each other is shown. An example of the monitoring area (dotted line) 152 is shown.
FIG. 5E shows an input image 131 with respect to the ground on which the imaging device 112 is installed, based on information such as the position, imaging angle, and focal length of the second imaging device 112 for the second monitoring device. An example of a scene image 161 of a scene coordinate system projected onto a plane parallel to the plane is shown.
In FIG. 5F, the input image 141 is displayed with respect to the ground on which the imaging device 113 is installed on the third monitoring device based on information such as the position, imaging angle, and focal length of the third imaging device 113. An example of a scene image 171 of a scene coordinate system projected onto a plane parallel to the plane is shown.

In this example, in the area setting process, first, the input devices 121, 131, and 141 expressed in the camera coordinate system having different coordinate axes depending on the positions and imaging angles of the imaging devices 111, 112, and 113 are converted into the imaging devices 111 and 112. , 113 are converted into scene images 151, 161, 171 expressed in the scene coordinate system depending on the position, imaging angle, etc., and on the scene image (for example, scene image 151) expressed on the scene coordinate system This is realized by setting a desired monitoring area 152. In this case, since the monitoring area can be set for each of the imaging devices 111, 112, and 113 using an image that can be seen according to the position and the imaging angle, it is easy for a person to easily understand the setting of the monitoring area. In addition, it is easy to set without any difference in apparent size due to perspective. In this case, the monitoring area setting operation is performed, for example, while a person watches a scene image of the scene coordinate system displayed on the screen of the display device 28.
In this case, the area information set in the scene coordinate system is converted into information in the world coordinate system and stored.

  As another example, in the region setting process, first, input images 121, 131, and 141 expressed in a camera coordinate system having different coordinate axes depending on the positions and imaging angles of the imaging devices 111, 112, and 113 are converted into the imaging devices 111, It is converted into a scene image (corresponding to the scene images 151, 161, 171) that can be expressed in the world coordinate system in which the coordinate axes are always constant without depending on the positions of 112, 113, the imaging angle, etc., and the world coordinate system This is realized by setting a desired monitoring area (corresponding to the monitoring area 152) on the scene image expressed above (for example, corresponding to the scene image 151). In this case, the monitoring area setting operation is performed, for example, while a person watches a scene image of the world coordinate system displayed on the screen of the display device 28.

  Here, any input image captured under any conditions can be expressed in the same world coordinate system if converted to a scene image in the scene coordinate system or the world coordinate system. Alternatively, there is an advantage that a monitoring area set on the world coordinate system (for example, one corresponding to the monitoring area 152) can be applied to all monitoring apparatuses that capture even a part of the same area.

Next, an example of a conversion processing method between the coordinates of the camera coordinate system and the coordinates of the world coordinate system will be described. In this example, this conversion process is performed by the image processor 23, for example.
In this example, the direct coordinate conversion from the camera coordinate system to the world coordinate system may be complicated in some cases, and the scene coordinate system may be useful for setting the area. A coordinate system obtained by rotating or translating the coordinate system and converting from the camera coordinate system to a scene coordinate system having an intersection point between the vertical downward vector of the imaging device 21 and the ground as the origin, and then changing from the scene coordinate system to the world coordinate system The process of converting to
That is, in this example, conversion between the camera coordinate system and the scene coordinate system and conversion between the scene coordinate system and the world coordinate system are performed.

Specifically, in this example, a scene image that can be expressed in a world coordinate system projected on a plane parallel to the ground from an input image 121 expressed in a camera coordinate system captured by the imaging device 21. The conversion process to (corresponding to the scene image 151) will be described. In addition, it is also possible to convert from the coordinates of the world coordinate system to the coordinates of the camera coordinate system by performing inverse conversion of this conversion.
In this example, the conversion from the input image 121 of the camera coordinate system to the scene image 151 is performed by coordinate conversion from the camera coordinate system to the scene coordinate system. Further, since the scene coordinate system is an orthogonal coordinate system obtained by simply rotating or translating the world coordinate system, the scene image 151 can be expressed on the world coordinate system by performing inverse transformation of the rotation or translation. Can do.

FIG. 6A shows an example of a state when the imaging device 21 is viewed in a direction horizontal to the ground (from the side).
FIG. 6B shows an example of a state when the imaging device 21 is viewed in a direction perpendicular to the ground (from directly above).
6A and 6B are expressed in the scene coordinate system. In this example, the scene coordinate system takes the x-axis and the y-axis on the ground plane as the point where the origin O intersects the reference plane (ground) vertically downward (z direction) of the imaging device 21, It is an orthogonal coordinate system taking the z axis in the vertical direction. In addition, an r-axis is provided in the line-of-sight direction (optical axis direction) in which the imaging device 21 faces.

In this example, represents the height of installation of the imaging device 21 are expressed on H, the depression angle of the imaging device 21 are expressed on theta _T, the viewing angle in the vertical direction of the image pickup device 21 in theta _V.
Also, it represents the viewing angle in the lateral direction of the image pickup device 21 in theta _H. Incidentally, the viewing angle theta _V in the vertical direction of the image pickup apparatus 21 can be calculated by the aspect ratio of the image pickup device (aspect ratio) from the viewing angle theta _H in the lateral direction of the image pickup device 21.
Further, in the field of view of the image captured by the imaging device 21, the horizontal distance from the imaging device 21 to the nearest place is represented by _LN , and the horizontal distance from the imaging device 21 to the farthest place is represented by _LF . Represents.

First, for example, the current direction of the electric pedestal (camera head) for changing the position and direction of the imaging device 21 (for example, the pan angle θ _P and the tilt angle θ _T with the front of the head as the origin) is acquired. The current focal length f of the imaging lens of the imaging device 21 is acquired. Based on the acquired information, the position of the imaging range is calculated.
A method for calculating the position of the imaging range will be described. In this example, in order to simplify the description, it is assumed that the monitoring target area is a plane and has no unevenness on the ground.
The angle of view θ _H that is the viewing angle in the horizontal direction of the imaging device 21 is obtained by (Equation 1).

Here, w is the horizontal width of, for example, a CCD element that is an image pickup element of the image pickup device 21. As an example, when an image pickup element of 1/3 inch (element size 4.8 mm × 3.6 mm) is used, w = 4.8 mm. When a 1/3 inch imaging device is used and the focal length of the imaging lens is f = 8.00 mm, the angle of view of the imaging device 21 is θ _H = 33.4 °. In other words, the field of view of the imaging device 21 has a range of 33.4 ° in the lateral direction.

Usually, the imaging device 21 is often installed at a higher position than the monitoring target area. Therefore, depending on the current direction theta _T of the camera platform, in the area beneath the imaging device 21 is a region that can not occur be imaged. This region appears in the range of the line-of-sight direction _LN from the imaging device 21. A predetermined region beyond that (the region between the distance L _N and the distance L _F ) enters the field of view of the imaging device 21.
The distances L _N and L _F will be described.
The angle of view θ _V that is the viewing angle in the vertical direction of the imaging device 21 is obtained by (Expression 2).

Here, h is the vertical width of, for example, a CCD element, which is an image pickup element of the image pickup device 21. For example, when a 1/3 inch image pickup element is used, h = 3.6 mm. In addition, when a 1/3 inch imaging device is used and the focal length of the imaging lens is f = 8.00 mm, the angle of view of the imaging device 21 is θ _V = 25.4 °. That is, the field of view of the imaging device 21 has a range of 25.4 ° in the vertical direction.
The distance L _N and the distance L _F are obtained by (Equation 3).

As a result, a range that can be imaged by the imaging device 21, and the internal region of the triangle surrounded by the terms of the point and the distance L _F of the imaging device 21 and the distance L _N shown in FIG. 6 (a), 6 The inside of the trapezoidal region surrounded by the points P1, P2, P3, and P4 shown in (b).
As an example, using a 1/3 inch imaging device, the focal length of the imaging lens is f = 8.00 mm, the current direction of the camera platform is θ _T = 30 °, and the installation height of the imaging device 21 is H = 5.0 m, L _N = 5.42 m and L _F = 16.1 m.

As described above, the position of the field-of-view range of the imaging device 21 is calculated by (Expression 1) to (Expression 3).
Based on this result, each position (x ′, y ′) on the image of the camera coordinate system included in the visual field range can be converted to the (x, y) coordinate of the scene coordinate system.
For example, when the pixel represented by (x ′, y ′) on the image is an image of the position of point C in FIG. 6A, if the number of pixels in the vertical direction of the input image 121 is 480 pixels, The depression angle θ _y for imaging the point C is expressed by (Equation 4).

In addition, the pixel (x ′, y ′) in the camera coordinates images, for example, the position of the point D in FIG. 6B (the spatial position is the same as the position of the point C in FIG. 6A). If the number of pixels in the horizontal direction of the input image 121 is 640, the rotation angle θ _x for capturing the point D is expressed by (Equation 5).

Here, θ _P is the pan angle of the imaging device 21, and in this example, indicates the angle formed by the x axis and the optical axis of the imaging device 21. Accordingly, a pixel at an arbitrary position represented by (x ′, y ′) on the input image 121 in the camera coordinate system is expressed as (x, y) in the scene image 151 of the scene coordinate system according to (Expression 6). Desired.

That is, the scene image 151 is obtained by coordinate-transforming the input image 121 according to (Expression 1) to (Expression 6). Note that the coordinates of the position of the visual field range can be changed by panning, tilting, or zooming of the imaging device 21. Further, the image converted into the scene coordinate system can be returned to the camera coordinate system by performing inverse conversion of the conversion expressed by (Expression 4) to (Expression 6).
Here, the first imaging device 111 (imaging device 21) has been described, but the input images 131 and 141 in the camera coordinates captured by the other imaging devices 112 and 113 are also expressed by (Expression 1) to (Expression 6). By performing coordinate conversion similar to the above, it is possible to convert into scene images 161 and 171 expressed on the scene coordinate system.

Here, since the scene coordinate system obtained by the coordinate transformation method described above is a coordinate system having an origin at the intersection of the vertical direction from the position of each of the imaging devices 111, 112, and 113 and the ground, each scene image 151 is obtained. 161, 171 have different coordinate systems.
Therefore, in order to view the scene images 151, 161, 171 obtained from all the monitoring devices 11, 12, 13 (all the imaging devices 111, 112, 113) in a unified coordinate system, each scene coordinate system The coordinate conversion from the world coordinate system, which is the only coordinate system, is required.

For example, suppose that the world coordinate system is an orthogonal coordinate system that takes the z _w axis vertically upward from the ground plane and the x _w axis and the y _w axis on the ground plane with respect to the origin O _w . Since each scene coordinate system is an orthogonal coordinate system that takes the x-axis and y-axis on the ground plane and takes the z-axis in the vertical direction of each imaging device, each scene coordinate system rotates or parallels the world coordinate system. It becomes the moved coordinate system. For this reason, the coordinates (x, y, z) of each scene coordinate system are represented by the position vector of the origin O ′ of the scene coordinate system on the world coordinate system as O _w O ′ = (q _x , q _y ) ^T Assuming that the rotation angle between the coordinate system and the scene coordinate system is θ, the coordinates (x _w , y _w , z _w ) of the world coordinate system are expressed by (Expression 7).

By (Expression 7), as shown in FIG. 7A, the scene image 151 can be expressed on the world coordinate system.
7A shows a scene image 151 represented on the world coordinate system (scene image 181) and a polygonal monitoring area (dotted line) 182 (corresponding to the monitoring area 152 in FIG. 5D). ) Is shown.

In particular, the scene image 181 on the world coordinate system (and the scene image 151 on the scene image system) obtained in this way has a similar relationship with the map of the monitoring area. For example, roads are displayed as parallel lines. Is done. In addition, the size of the plant and the size of the sign appearing in the image are proportional to the actual size (for example, metric units).
For example, in the conventional area designation method using the camera coordinate system image as shown in FIG. 13B, when an area of a certain distance from the building 322 is designated, the apparent size depends on the position on the image. However, as in this example, the area using the scene image (in the individual scene coordinate system for each imaging device or in the world coordinate system common to all imaging devices) is used. In the case of designating, the problem that the apparent size differs depending on the position on the image does not occur.

Next, a method for setting a monitoring area using scene coordinates (or world coordinates) by each method described above and a method for sharing monitoring area information between monitoring apparatuses will be described.
In order to set and share the monitoring area, first, the input images 121, 131, and 141 captured by the respective monitoring devices 11, 12, and 13 are respectively represented by (Expression 1) to (Expression 6) in the scene coordinate system. The scene images 151, 161, and 171 are converted. Next, for example, in one monitoring device 11, the input device 29 is used to set a region desired to be monitored on the scene image 151 of the scene coordinate system, for example, by a polygon 152. Then, according to (Equation 7), the coordinate system of the scene images 151, 161, 171 of each scene coordinate system and the set monitoring area 152 is converted into the world coordinate system.

FIG. 7B shows scene images 191, 192, 193 (corresponding to scene images 151, 161, 171 respectively) and a monitoring area (dotted line) 194 (corresponding to the monitoring area 152) expressed on the world coordinate system. An example is shown.
In the monitoring device 11, information on the monitoring area 194 expressed in the world coordinate system is stored in the storage device 31 so that it can be shared with other monitoring devices 12 and 13, for example. And the monitoring apparatus 11 transmits the information of the world coordinate of the monitoring area | region 194 defined on the said monitoring apparatus 11 with respect to the other monitoring apparatuses 12 and 13 via the network 14 as needed.

  In this example, the case where the world coordinate information of the monitoring area 194 is transmitted via the network 14 connected by the external I / F circuit 26, for example, is shown. However, as another configuration example, the external I / F circuit 26 is used. It is also possible to perform transmission using a portable medium such as a CD-RAM connected to the device.

Next, for example, the monitoring device 12 or 13 is instructed to set the monitoring region based on the received world coordinate information of the monitoring region 194 using the input device 29 or the like. In response to this, in each of the monitoring devices 12 and 13, the image processing processor 23 of each of the monitoring devices 12 and 13 performs area setting based on the received information on the monitoring area 194.
Specifically, since the monitoring area 194 and the scene images 192 and 193 can be expressed as shown in FIG. 7B on the world coordinate system, the respective scene images 192 and 193 overlap with the monitoring area 194. The coordinates of the area to be set are set as the monitoring area in each of the monitoring devices 12 and 13. That is, a monitoring area equivalent to the monitoring area 194 determined on one monitoring apparatus 11 (corresponding to the monitoring area 152) can be shared and set by the other monitoring apparatuses 12 and 13. In addition, the setting information of the monitoring area on the world coordinate system can be set by converting into the information of the scene coordinate system and the information of the camera coordinate system corresponding to each of the monitoring devices 12 and 13.

For example, when setting a monitoring area for a plurality of monitoring devices by a conventional method, an input device such as a mouse is used for each of the images captured by all the imaging devices to be monitored, The monitoring area needs to be set manually and directly like the polygons 122, 132, 142 in FIGS. 5A, 5B, and 5C, for example. The more it took, the more time and effort it took to set the area.
On the other hand, in the case of this example, for a scene image in the scene coordinate system (or world coordinate system) obtained by one monitoring device, for example, a polygon 152 in FIG. If the monitoring area is set, regardless of the number of monitoring apparatuses, it is possible to automatically and instantaneously set a monitoring area equal to the monitoring area for all other monitoring apparatuses.

  As described above, in the monitoring devices 11 to 13 of the present example, the imaging device 21 that images the monitoring region, the process of processing the input image captured by the imaging device 21 and detecting an object in the monitoring region, and the like. An image processing apparatus (in this example, a function of the image processing processor 23 and the like) to be performed, and a display device 28 that displays an output image based on a detection result of an object by the image processing apparatus are provided. The image processing apparatus has an image conversion function for converting an input image of the first coordinate system (camera coordinate system) obtained by imaging the monitoring area with the imaging device 21 into a scene image based on the second coordinate system (scene coordinate system); A setting function for setting information (for example, region information) necessary for processing the input image on the scene image obtained by the image conversion function, and necessary for processing the input image Based on the information, processing for detecting an object in the monitoring area is performed.

In addition, the image processing apparatus has a function capable of arbitrarily performing conversion between the camera coordinate system, the scene coordinate system, and the world coordinate system with respect to setting information such as image information or regions. In particular, the area information set in the scene coordinate system is converted into information in the world coordinate system and stored.
In addition, the monitoring devices 11 to 13 of the present example include a storage device 31 for holding (storing) region information and detection result information in each of the monitoring devices 11 to 13. An input device 29 for inputting region information and the like is provided.

Further, in the monitoring devices 11 to 13 of this example, information set on the scene image in the scene coordinate system (or the world coordinate system) is held in an internal or external storage unit, and a part of the monitoring area or With respect to at least one (one or more) monitoring devices that monitor objects in a region including all from different positions, angles, etc., refer to the information recorded in the storage unit and monitor the imaging device. Information (for example, area information) is set.
Further, in the monitoring apparatuses 11 to 13 of this example, information on a detection area (monitoring area) representing an area for detecting an object is used as information necessary for processing the input image.

In the monitoring devices 11 to 13 of this example, the second coordinate system (scene coordinate system) described above is a coordinate system similar to the map of the monitoring area.
In the monitoring devices 11 to 13 of this example, the third coordinate system uses a world coordinate system that is a common coordinate system in all the monitoring devices that monitor the monitoring area. The world coordinate system is also a coordinate system similar to the map of the monitoring area.

As a specific example, monitoring of a monitoring area is performed using one or more monitoring devices.
The image of the monitoring area captured by one or more monitoring devices is information that looks different depending on the imaging conditions of each monitoring device, that is, information that differs for each camera coordinate system. By converting the input image information expressed in the system so that it can be expressed in a common world coordinate system that can indicate the absolute position of the object, information such as the monitoring area is different between different monitoring devices and different monitoring conditions A situation that can be shared and handled between the two is realized. In addition, by converting the input image of the camera coordinate system into a scene image of the scene coordinate system for each imaging device and setting a desired monitoring area on the scene image of the obtained scene coordinate system, a human sense Thus, it is possible to effectively set a region or the like using a scene image in a scene coordinate system that is easy to understand. Note that an area or the like may be set in an image of the world coordinate system.

  Here, information such as a monitoring area shared by a plurality of monitoring devices or a plurality of monitoring conditions can be applied as it is in determining the area of other monitoring devices as long as it is on the world coordinate system. As described above, for example, an imaging device (monitoring device) that monitors the same monitoring region (part or all) by enabling sharing of information such as a monitoring region with a plurality of monitoring devices or a plurality of monitoring conditions. ) And the movement of the same imaging device (monitoring device), etc., it is possible to eliminate the need for new information setting work such as a monitoring area.

In addition, even when it is necessary to change information such as an area desired to be monitored, by changing information such as an area on the world coordinate system that can be shared in one monitoring device, the change can be made to all This can be easily reflected in the monitoring device.
In addition, the scene coordinate system and the world coordinate system can be a coordinate system similar to the map of the monitoring area as a coordinate system projected onto a plane parallel to the ground of the monitoring area. By using the system, information such as a region can be set intuitively and easily by a person.

  Therefore, in this example, the setting information such as the monitoring area used for distinguishing and detecting the subject for each area is different among a plurality of monitoring apparatuses having different positions and angles of the imaging apparatus, and the positions and angles of the imaging apparatuses are different. It can be shared between the same monitoring devices (when arrangement is changed). Then, effective monitoring is performed by monitoring an object shown in the captured input image based on the area information set in the scene coordinate system or the world coordinate system similar to the map of the area to be monitored. It can be carried out.

For example, conventionally, it is necessary to previously set information such as in which area in an input image to be captured an object to be detected for each input image obtained by an imaging device used in each monitoring device. For this reason, enormous work is required when adding a monitoring device or changing usage conditions of the monitoring device.
On the other hand, in the monitoring device and the monitoring information setting method as in the present example, the world coordinates are obtained by converting the input image captured by the imaging device into a scene image that can be expressed in a world coordinate system similar to a map. It is possible to express input image information and area information captured by all imaging devices in the system, and set a desired area in the scene coordinate system or the world coordinate system, and capture all the information such as the area. Can be shared by devices. As a result, it is possible to reduce the labor of setting work such as an area associated with addition of a monitoring device or change of use conditions, and it is possible to operate the monitoring device more efficiently, easily and easily.

Here, in this example, the case where information of a monitoring area is mainly set as information necessary for processing an input image has been shown, but various kinds of information may be used as such information. For example, it is possible to set threshold information related to the size, width, height, length, etc. of an object to be detected.
In this example, a polygon is used as the shape of the monitoring area, but any other shape may be used.

In this example, the image processing apparatus is configured by the plurality of monitoring apparatuses 11, 12, 13 and the entire network 14.
In the image processing apparatus of this example, for example, a display unit is configured by a function of displaying an image by the display device 28 provided in the monitoring devices 11 to 13, and the input device 29 provided in the monitoring devices 11 to 13. The receiving means is configured by the function of receiving setting information such as the area from the user (person), and the coordinate system conversion is performed by the function of converting the coordinate system of information by the image processor 23 provided in the monitoring devices 11 to 13. The conversion information setting means is configured by the function of setting the setting information acquired by the arrangement of a certain imaging apparatus in an imaging apparatus having another arrangement (same or different from the certain imaging apparatus). Yes.
In this example, for example, the setting information acquired by the own device for each of the monitoring devices 11 to 13 is converted into information of the world coordinate system, and the information of the world coordinate system is communicated between the monitoring devices 11 to 13. However, other aspects may be used.

A second embodiment of the present invention will be described.
Detailed description of the same configuration and operation as in the first embodiment will be omitted.
In the first embodiment described above, information represented in the world coordinate system of the monitoring area set in each of at least one monitoring device can be shared with other monitoring devices via a network, for example. Since this monitoring area information is shared by at least one monitoring device, it is possible to use a form in which an information server responsible for information storage management is provided. In this example, such a form will be described. .

FIG. 8 shows a configuration example of a monitoring system according to an embodiment of the present invention.
The monitoring system 41 of this example includes three or more monitoring devices 51, 52, and 53 (a first monitoring device 51, a second monitoring device 52, a third monitoring device 53,...), These are connected by a network 55, and an information server 54 connected to the network 55 is provided. For example, the information server 54 can be operated by a person, store various types of information in an internal or external storage device, and display various types of information on a screen.
Note that the number of monitoring devices may be arbitrary, and in this example, for convenience of explanation, the description will first be made on the assumption that there are three devices, and then one device is added.

The information server 54 includes information on areas desired to be monitored obtained from all or any of the monitoring devices 51, 52, 53, input / output information for the monitoring devices 51, 52, 53, and monitoring devices 51, 52, 53. Information such as the presence / absence of an intruding object obtained is retained, and further, the information is transmitted to each of the monitoring devices 51, 52, 53 as necessary. For example, when the operator corrects the monitoring area information on the scene image in the scene coordinate system for one of the monitoring devices 51, 52, and 53 (for example, the monitoring device 51), the corrected information becomes the world coordinates. The monitoring area information is converted to the system area information, and the monitoring area information stored in the information server 54 is updated based on the converted area information.
Here, in this example, the information server 54 holds information on the world coordinate system, but information on the scene coordinate system and camera coordinate system for each monitoring device may be held as necessary.

  Each of the monitoring devices 51, 52, and 53 accesses the information server 54 periodically (for example, every 10 minutes), and if the information of the monitoring area in the information server 54 has been updated, Based on this, the region information stored in the storage device 31 of each of the monitoring devices 51, 52, and 53 is updated, and the region information updated in this way is used in subsequent object detection processing and the like.

  FIG. 9A shows an example of a scene image expressed on the world coordinate system. Specifically, the scene image 201 obtained by the first monitoring device 51, the scene image 202 obtained by the second monitoring device 52, the scene image 203 obtained by the third monitoring device 53, A polygonal monitoring area (dotted line) 204 is shown on the world coordinate system.

In this example, for example, the information of the scene images 201, 202, and 203 in each of the monitoring devices 51, 52, and 53 is transmitted to the information server 54 via the network 55, and is used on the world coordinate system using the information server 54. Each scene image 201, 202, 203 can be expressed. Further, it is possible to draw and set the polygon 204 as a monitoring area by a human operation on such a world coordinate system, and the information on the world coordinates of the monitoring area 204 is, for example, the information server 54 or the like for the entire monitoring system. By making the information sharable, the area overlapping with the monitoring area 204 can be set as the area to be monitored for all the monitoring devices 51, 52, 53.
As in this example, by providing the information server 54 for storing and managing information between the monitoring devices 51, 52, and 53, the information in the monitoring area 204 can be collectively managed, Changes can be made in a batch.

Furthermore, the monitoring system 41 of this example is efficient when a new monitoring area is added.
FIG. 9B shows an example of a scene image represented on the world coordinate system. Specifically, the scene images 211, 212, and 213 (same as the scene images 201, 202, and 203) obtained by the three monitoring devices 51, 52, and 53, and the polygonal monitoring area that is already set ( A dotted line) 214 (the same as the monitoring area 204) and a newly set polygonal monitoring area (dotted line) 215 are shown on the world coordinate system.

  For example, when it is desired to add the polygon 215 shown in FIG. 9B as a new monitoring area and monitor the two monitoring areas 214 and 215, one of the monitoring devices 51, 52, 53 or the information server 54 by inputting the coordinates of the scene coordinate system or world coordinate system of the polygon 215 of the new monitoring area to the information server 54 by using an input device such as a mouse. Set. Then, all the monitoring devices 51, 52, 53 that monitor the monitoring area 215 from the information server 54 (if a new monitoring area is set by any monitoring device, the monitoring device may be excluded). Information of the world coordinates of the monitoring area 215 is transmitted to the monitoring apparatuses 51, 52, and 53 (in either monitoring apparatus, a new monitoring area is monitored so that both the monitoring areas 214 and 215 are monitored). The monitoring area may be added to the entire monitoring system 41 instantaneously by giving an instruction to the monitoring device 41).

For example, in the conventional method, it is necessary to additionally specify a monitoring area by manually drawing a polygon with an input device such as a mouse for each monitoring apparatus to be monitored. was there.
On the other hand, in this example, a monitoring area desired to be added is manually set by an input device such as a mouse using any one of the monitoring devices 51, 52, 53 or the information server 54, and the monitoring area By transmitting the world coordinate information from the information server 54 to each of the monitoring devices 51, 52, and 53, the additional region is instantaneously reflected to all the monitoring devices 51, 52, and 53 by setting the additional region once. It is possible to reduce the burden on the user who operates the monitoring devices 51, 52, and 53.

Moreover, in the monitoring system 41 of this example, addition of a monitoring device (imaging device) can be easily realized.
FIG. 10 shows an example of a monitor area 221 monitored by the monitoring system 41 of the present example and a sketch 221 around it. This sketch 221 is a view of the monitoring area looking down from a direction perpendicular to the ground.
Specifically, there are roads, buildings, and vacant lots, the arrangement (position and direction) of the first imaging device 231 provided in the first monitoring device 51, and the second provided in the second monitoring device 52. Of the second imaging device 232, the arrangement of the third imaging device 233 provided in the third monitoring device 53, and the fourth imaging device 234 provided in the fourth monitoring device (new monitoring device). The arrangement is shown.
For example, even when a new monitoring device P having an imaging device is added to a certain point (the roof of a building) 234 on the map shown in FIG. 10 for the purpose of improving the monitoring accuracy, the monitoring system 41 of this example. Then, it is possible to easily realize setting of information such as a monitoring area for the new monitoring device P.

FIG. 11A shows an example of the input image 241 captured by the imaging device 234 of the new monitoring device P, and also shows an example of the monitoring region 242 to be set.
FIG. 11B shows an example of a scene image 251 obtained by converting the input image 241 shown in FIG. 11A by (Equation 1) to (Equation 6) in the new monitoring device P. It is.
FIG. 11C shows an example of the scene image 261 on the world coordinate system obtained by converting the scene image 251 on the scene coordinate system shown in FIG. An example of a polygonal monitoring area 262 is shown on the world coordinate system.

  In the area setting for the new monitoring device P, first, the scene image 251 shown in FIG. 11B is expressed in the world coordinate system as shown in FIG. For example, the coordinates of the polygon 262 are transmitted as the world coordinates of the monitoring area that can be shared by the information server 54 or the like to the monitoring apparatus P, and the world coordinates of the monitoring area 262 are monitored by the new monitoring apparatus P. Give instructions to do so. Thereby, the monitoring area 262 is set like a polygon shown in FIG. 11C, and the monitoring area can be set instantaneously and automatically.

As a result, in the monitoring system 41, as shown in FIG. 11D, the same monitoring area is obtained by a plurality of monitoring devices (in this example, the monitoring devices 51, 52, and 53 and the new monitoring device P). 262 can be monitored in duplicate.
FIG. 11D shows a common scene image 271, 272, 273, and 274 obtained by four monitoring devices (in this example, the monitoring devices 51, 52, and 53 and the new monitoring device P). An example of the monitoring area 275 is shown on the world coordinate system.

For example, in the conventional method, when a monitoring area is monitored by adding a new monitoring device to the monitoring system, an input device such as a mouse is used for an input image obtained by the new monitoring device, It is necessary to manually set the polygon of the monitoring area, and it is necessary to manually add the monitoring area as the monitoring apparatus is added.
On the other hand, in this example, by converting the information of the monitoring area that can be shared by the monitoring system 41 into the world coordinate system, it can be set numerically in all the monitoring devices that monitor the monitoring area. Therefore, no matter how many monitoring devices are added, it is not necessary to manually set a monitoring area for monitoring the monitoring area, and the operator of the monitoring system 41 is very A monitoring device can be added with a small burden.

In this example, for example, even when the monitoring area is monitored by changing the position or angle of the imaging device in the monitoring apparatus, the monitoring area is manually changed as long as the same area as the monitoring area is monitored. There is no need to do.
For example, when the position, the angle, and the focal length of the imaging device in the monitoring device P that monitors the monitoring region 262 shown in FIG. 11C are changed, the input image that has been picked up as shown in FIG. Consider a case where 241 changes as in the input image 281 shown in FIG. 11E in accordance with the change in the position, angle, and focal length of the imaging device.
FIG. 11E shows an example of the input image 281 captured by the imaging device of the monitoring device P and an example of the monitoring area (dotted line) 282.

In this case, in the conventional method, in order to reset the area overlapping the monitoring area 242 shown in FIG. 11A as the monitoring area, for example, the area 282 that is desired to be monitored is input to the input image 281 with the mouse. It was necessary to set it manually by the input device.
On the other hand, in this example, the monitoring area of the monitoring device P before changing the setting of the imaging device is set on the world coordinate system like a polygon 262 shown in FIG. Keep the value. Thereby, even when the setting of the imaging device is changed and the input image changes as shown in FIG. 11E, the input image 281 is converted into a scene on the world coordinate system shown in FIG. Since it can be expressed in world coordinates by converting to the image 291, the coordinates of the monitoring area 262 determined before changing the setting of the imaging apparatus can be used as the monitoring area as it is after the setting of the imaging apparatus is changed. This can reduce the time and effort to newly set the monitoring area.
FIG. 11F shows an example of a scene image 291 in which the input image 281 shown in FIG. 11E is expressed on the world coordinate system and an example of the monitoring area 292 on the world coordinate system.

In addition, by using the feature that scene images expressed on the scene coordinate system and the world coordinate system are similar to the map of the monitoring area, it is possible to set an intuitive monitoring area or the like based on the map, or the monitoring system 41. Monitoring information can be shared between them.
For example, the scene images 201, 202, and 203 represented on the world coordinate system shown in FIG. 9A are similar to the floor plan 101 in which these areas shown in FIG. There is no difference in information such as the position of

  Therefore, the coordinates of the monitoring area 114 set as shown in FIG. 1 on the world coordinate system are calculated in consideration of the scale of the sketch, for example, and the world coordinates of the monitoring area 114 are calculated by the information server 54 or the like. By enabling sharing, an area overlapping with the monitoring area 114 can be instantaneously set for all monitoring apparatuses. As an example, when a monitoring area is set using a medium outside the monitoring device using a floor plan or a map, it is not necessary to use information based on the output result of the monitoring device such as a scene image. Therefore, even a person who is not skilled in using the monitoring device can intuitively specify a region or the like.

As described above, in the monitoring devices 51, 52, and 53 of this example, information (for example, area information) necessary for processing the input image is transmitted via the network 55 to the external storage device (in this example, the information server 54). ) To store and manage information. In addition, the monitoring system 41 of this example includes an information server 54 that transmits and receives the above information to and from the monitoring devices 51, 52, and 53 connected to the network 55.
Therefore, in this example, the information server 54 provided outside the monitoring devices 51, 52, 53 shares information related to the plurality of monitoring devices 51, 52, 53 and collectively stores, manages, and sets the information. Can do.

In this example, the image processing apparatus is configured by the plurality of monitoring apparatuses 51, 52, 53, the information server 54, and the network 55 as a whole.
In the image processing apparatus of this example, the function of a display unit that displays an image, the function of a reception unit that receives setting information from a user (person), the function of a coordinate system conversion unit that converts a coordinate system of information, A part or all of the function of the conversion information setting means for setting the setting information between the imaging devices having different arrangements is provided in the information server 54, and part or all of these functions are provided in the monitoring devices 51 to 53. Is also provided.
In this example, for example, the setting information acquired by the own device for each of the monitoring devices 51 to 53 is converted into information of the world coordinate system, and the world coordinate system between the monitoring devices 51 to 53 and the information server 54 is converted. Although other information may be communicated, other aspects may be used.

Here, the configuration of the system and apparatus according to the present invention is not necessarily limited to the configuration described above, and various configurations may be used. The present invention can also be provided as, for example, a method or method for executing the processing according to the present invention, a program for realizing such a method or method, or a recording medium for recording the program. It is also possible to provide various systems and devices.
The application field of the present invention is not necessarily limited to the above-described fields, and the present invention can be applied to various fields.
In addition, as various processes performed in the system and apparatus according to the present invention, for example, the processor executes a control program stored in a ROM (Read Only Memory) in hardware resources including a processor and a memory. A controlled configuration may be used, and for example, each functional unit for executing the processing may be configured as an independent hardware circuit.
The present invention can also be understood as a computer-readable recording medium such as a floppy (registered trademark) disk or a CD (Compact Disc) -ROM storing the control program, and the program (itself). The processing according to the present invention can be performed by inputting the program from the recording medium to the computer and causing the processor to execute the program.

It is a figure which shows an example of the sketch of a monitoring area | region, and an example of the position of the installed imaging device. It is a figure which shows an example of the monitoring system which concerns on 1st Example of this invention. It is a figure which shows the structural example of the monitoring apparatus which concerns on one Example of this invention. It is a figure which shows an example of the flow of the process of the program performed by the image processor. (A) is a figure which shows an example of the input image imaged by the 1st imaging device, (b) is a figure which shows an example of the input image imaged by the 2nd imaging device, (c) is It is a figure which shows an example of the input image imaged by the 3rd imaging device, (d) is a figure which shows an example of the scene image by a 1st monitoring apparatus, (e) is a scene by a 2nd monitoring apparatus. It is a figure which shows an example of an image, (f) is a figure which shows an example of the scene image by a 3rd monitoring apparatus. (A) is a figure for demonstrating coordinate transformation, (b) is a figure for demonstrating coordinate transformation. (A) is a figure for demonstrating the method of area | region designation | designated, (b) is a figure for demonstrating the method of area | region designation | designated. It is a figure which shows an example of the monitoring system which concerns on 2nd Example of this invention. (A) is a figure for demonstrating the method of area | region designation | designated, (b) is a figure for demonstrating the method of area | region designation | designated. It is a figure which shows an example of the sketch of a monitoring area | region, and an example of the position of the installed imaging device. (A) is a figure which shows an example of the input image imaged with an imaging device, (b) is a figure which shows an example of the scene image by a monitoring apparatus, (c) is for demonstrating the area | region designation | designated method (D) is a figure for demonstrating the method of area | region designation | designated, (e) is a figure which shows an example of the input image imaged with an imaging device, (f) is a method of area | region designation | designated. It is a figure for demonstrating. It is a figure which shows an example of the flow of the object detection process by a difference method. (A) is a figure which shows an example of the input image imaged with an imaging device, (b) is a figure for demonstrating the method of area | region designation | designated.

Explanation of symbols

1, 41 ... Monitoring system, 11-13, 51-53 ... Monitoring device, 14, 55 ... Network, 21 ... Imaging device, 22 ... Video input circuit, 23 ... Image processor, 24 ... Program memory, 25 ... Work memory, 26 ... External I / F circuit, 27 ... Video output circuit, 28 ... Display device, 29 ... Input device, 30 ... Input circuit, 31 ... Storage device, 32 ..Data bus 54.Information server
101, 221, sketch, 111 to 113, 231 to 234, imaging device 114, 122, 132, 142, 152, 182, 194, 204, 214, 215, 242, 262, 275, 282, 292,. Monitoring area 121, 131, 141, 241, 281 .. Input image, 151, 161, 171, 181, 191-193, 201-203, 211-213, 251, 261, 271-274, 291 ... Scene image ,
301, 321 ... Input image 302 ... Background image 303 ... Differencer 304 ... Difference image 305 ... Binarizer 306 ... Binarized image 311 ... Object 312 ... Area, 313 ... pixel, 314 ... area, 322 ... building, 331 ... monitoring area,
T1, ... background image, T2, ... area information,

Claims (3)

In an image processing apparatus that processes an image,
Displaying the image obtained by converting the image of the first coordinate system, which is the coordinate system at the time of imaging by the imaging device, into the image of the second coordinate system provided on the reference plane based on the arrangement of the imaging device Display means to
Accepting means for accepting setting information regarding the image displayed by the display means from a user;
Coordinate system conversion means for converting the setting information received by the receiving means from information of the second coordinate system to information of a third coordinate system provided on a reference plane independent of the arrangement of the imaging device;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
The coordinate system conversion means uses second information corresponding to an arrangement different from the predetermined arrangement for setting information of the third coordinate system acquired based on an image captured by one imaging device having a predetermined arrangement. To the coordinate system information of
The image processing apparatus uses the imaging apparatus different from the one imaging apparatus having an arrangement different from the predetermined arrangement or the one imaging apparatus whose arrangement has been changed, by converting the information converted by the coordinate system conversion unit. Provided with conversion information setting means for setting the obtained image;
An image processing apparatus. The image processing apparatus according to claim 1 or 2,
As the setting information related to the image, at least one of information specifying an area to be monitored or information related to a detection target is used.
An image processing apparatus.

Images