KR101313134B1 - Apparatus for combing image in surveillance camera system and method thereof - Google Patents

Abstract

An apparatus and method for synthesizing an image in a surveillance camera system according to the present invention are disclosed.
An apparatus for synthesizing an image according to the present invention includes an image input unit for receiving a compressed image from a plurality of cameras in a monitored area; An image transformation unit for reconstructing the received compressed images and transforming the reconstructed images by using a predefined transformation matrix; And an image synthesizing unit for synthesizing the transformed images to generate one synthesized image, wherein the transformation matrix is a matrix defined to transform coordinates in three-dimensional space into coordinates in a two-dimensional image.
Through this, the present invention can not only acquire the image of the entire area regardless of the position or installation angle of the camera, but also the monitor can intuitively monitor the whole area, and can easily track and monitor the moving object moving in the whole area. Can be.

Description

Apparatus and method for synthesizing images in surveillance camera system {APPARATUS FOR COMBING IMAGE IN SURVEILLANCE CAMERA SYSTEM AND METHOD THEREOF}

The present invention relates to an image synthesizing apparatus. In particular, an image of a three-dimensional space plane obtained from a plurality of cameras is transformed into an image of a two-dimensional space plane by using a predefined transformation matrix, and the modified images are synthesized. An apparatus and method for synthesizing an image in a surveillance camera system for generating a composite image of the present invention.

Surveillance camera systems typically use multiple cameras to monitor specific areas. Images from each camera are output to a separate monitor so that the observer can monitor the entire area using that individual monitor.

Since each camera is installed at various angles and directions, it is difficult to observe the image output from each monitor to determine the relationship between the image displayed on the actual monitor and the physically existing area.

Therefore, the monitor cannot easily think of the image of the entire area to be monitored by checking only the image output on the monitor, so not only the monitoring of the entire area is easy, but also the monitor is easily tired.

In addition, when continuously monitoring a specific moving object in a specific area to be monitored, it is difficult to continuously monitor the moving object in a specific area because the image of the entire area to be monitored cannot be easily identified.

Therefore, to solve the problems of the prior art, an object of the present invention is to transform the image of the three-dimensional space plane obtained from a plurality of cameras to the image of the two-dimensional space plane using a predefined transformation matrix, the deformation An apparatus and method for synthesizing an image in a surveillance camera system for synthesizing the captured images to generate a single composite image are provided.

However, the objects of the present invention are not limited to those mentioned above, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In order to achieve the above objects, an apparatus for synthesizing an image according to an aspect of the present invention includes an image input unit for receiving a compressed image from a plurality of cameras in the area to be monitored; An image transformation unit for reconstructing the received compressed images and transforming the reconstructed images by using a predefined transformation matrix; And an image synthesizing unit for synthesizing the transformed images to generate one synthesized image, wherein the transformation matrix is a matrix defined to transform coordinates in three-dimensional space into coordinates in a two-dimensional image.

Preferably, the image transformation unit may transform the reconstructed three-dimensional coordinate system images into two-dimensional coordinate system images, respectively, by using the predefined transformation matrix.

Preferably, the image transformation unit may transform each of the reconstructed images by using the transformation matrix calculated by calculating a value of the transformation matrix based on at least one coordinate previously known in each of the reconstructed images. have.

Preferably, the image synthesizing unit may synthesize the overlapping portions between the images synthesized in the process of synthesizing the images into one using a preset blending technique.

Preferably, the image synthesizing unit may combine the overlapping portions between the images synthesized in the process of synthesizing the images into one using an average value of the overlapping images.

Preferably, the image synthesizing unit may synthesize one overlapping portion between the images synthesized in the process of synthesizing the images by using a preset weight according to a distance from each of the overlapping images.

According to another aspect of the present invention, a method for synthesizing an image includes: (a) receiving a compressed image from a plurality of cameras in a monitored area; (b) restoring the received compressed images and transforming the restored images using a predefined transformation matrix; And (c) synthesizing the plurality of modified images to generate a single composite image, wherein the transformation matrix is a matrix defined to transform coordinates in three-dimensional space into coordinates in a two-dimensional image. It features.

Preferably, in the step (b), the reconstructed three-dimensional coordinate system images may be transformed into two-dimensional coordinate system images, respectively, by using the predefined transformation matrix.

Preferably, the step (b) transforms each of the reconstructed images by using the transformation matrix calculated by calculating a value of the transformation matrix based on at least one coordinate previously known in each of the reconstructed images. can do.

Preferably, in the step (c), the overlapping portions between the images synthesized in the process of synthesizing the images may be synthesized into one using a preset blending technique.

Preferably, in the step (c), the overlapping portions between the synthesized images may be synthesized into one using the average value of the overlapping images.

Preferably, in the step (c), the overlapping portions between the images synthesized in the process of synthesizing the images may be synthesized into one using a weight set in advance according to the distance from each of the overlapping images.

Through this, the present invention transforms the image of the three-dimensional space plane obtained from a plurality of cameras to the image of the two-dimensional space plane using a predefined transformation matrix and synthesized the modified images to generate a single composite image In addition, there is an effect of obtaining an image of the entire area regardless of the position or installation angle of the camera.

In addition, the present invention has the effect that the monitor can intuitively monitor the entire area because the image is obtained for the entire area regardless of the position or installation angle of the camera.

In addition, the present invention has an effect that can easily track and monitor the moving object moving in the entire area because the monitor can intuitively monitor the entire area.

1 is a view showing a surveillance camera system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a detailed configuration of the image synthesizing apparatus 200 shown in FIG. 1.
3 is a diagram showing a point in three-dimensional space projected on a two-dimensional plane.
4 is a diagram for describing a transformation of an image, according to an exemplary embodiment.
5 is a diagram for describing synthesis of an image, according to an exemplary embodiment.
6 is a diagram illustrating a method for synthesizing an image according to an embodiment of the present invention.

Hereinafter, an apparatus and method for synthesizing an image in a surveillance camera system according to an exemplary embodiment of the present invention will be described with reference to FIGS. 1 to 6. The present invention will be described in detail with reference to the portions necessary for understanding the operation and operation according to the present invention. Like reference numerals in the drawings denote like elements throughout the specification. In the following description, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Particularly, in the present invention, the image of the three-dimensional space plane obtained from a plurality of cameras is transformed into the image of the two-dimensional space plane using a predefined transformation matrix, and the modified images are synthesized to generate one composite image. Propose a new solution.

1 is a view showing a surveillance camera system according to an embodiment of the present invention.

As shown in FIG. 1, the surveillance camera system according to the present invention may include a plurality of cameras 100 installed in a surveillance area (eg, a bridge, a tunnel, etc.), and an image synthesizing apparatus 200. have.

The camera 100 may be arranged to observe all the areas (eg, bridges, tunnels, etc.) to be monitored. At this time, the position and direction of the camera 100 are irrelevant, but should be arranged to obtain an image that can secure all the entire area to be monitored.

The camera 100 may compress the obtained image and transmit the compressed image to the image synthesizing apparatus 200 through a wired or wireless network.

The image synthesizing apparatus 200 restores the compressed image received from the camera, transforms the restored image using a predefined transformation or transformation matrix, and synthesizes the modified images to generate one composite image. can do.

FIG. 2 is a diagram illustrating a detailed configuration of the image synthesizing apparatus 200 shown in FIG. 1.

As shown in FIG. 2, the image synthesizing apparatus 200 according to the present invention includes an image input unit 210, an image transformation unit 220, an image synthesis unit 230, an image storage unit 240, and the like. Can be configured.

The image input unit 210 may receive a compressed image from a camera by wire or wirelessly.

The image transformation unit 220 may restore the compressed image and transform the restored image using a predefined transformation matrix, wherein the transformation matrix transforms coordinates in three-dimensional space into coordinates in a two-dimensional image. Coordinates on a two-dimensional image can be transformed into coordinates on a three-dimensional space.

This conversion matrix is described in detail with reference to FIGS. 3 to 5 as follows.

3 is a diagram showing a point in three-dimensional space projected on a two-dimensional plane.

As shown in FIG. 3, assuming that a point (X, Y, Z) in three-dimensional space can be projected to a point (x, y) in a two-dimensional plane through a camera, the coordinate in three-dimensional space ( X, Y, Z) and the relationship between the coordinates (x, y) on the two-dimensional plane can be expressed as Equation 1 below.

[Equation 1]

Since (x, y) is a coordinate on the camera sensor, changing this to the pixel coordinate of the image can be expressed as Equation 2 below.

&Quot; (2) "

Here, (u, v) may represent coordinates on the image, and (u ₀ , v ₀ ) may represent coordinates where the camera optical axis is projected on the image. And k _u , k _v may represent the ratio between the coordinates on the camera sensor and the image coordinates for the x and y axes, respectively.

If Equation 2 is expressed as a matrix, it can be expressed as Equation 3 below.

&Quot; (3) "

Since the coordinates in the three-dimensional space are rotated and moved, the equation (3) can be expressed by the following equation (4) in consideration of the rotation and the movement in the three-dimensional space.

&Quot; (4) "

로 이루어지고, 이동은 이동 행렬

로 이루어진다.Where rotation is the rotation matrix

Is made up of a moving matrix

.

(z축)

(y축),

(x축)이라 하면, 회전 행렬 R은 다음의 [수학식 5]와 같이 나타낼 수 있다.The three-dimensional rotation matrix R is a matrix representing rotations about the x, y, and z axes in three-dimensional space.

(z-axis)

(y-axis),

In the case of (x-axis), the rotation matrix R can be expressed as Equation 5 below.

&Quot; (5) "

In Equation 5, the relationship between coordinates (X, Y, Z) in three-dimensional space and coordinates (u, v) in two-dimensional images is multiplied by multiplying the weekly matrix, as shown in Equation 6 below. Can be represented.

&Quot; (6) "

가 모두 포함되어 있는 행렬로, 3차원 물체가 2차원 평면상에 투영되어 변화한다면 임의의 형태로 변형되는 것이 아니라 상기 [수학식 6]에 의하여 형태의 변화가 제한받게 된다.In Equation 6, the matrix P is a camera internal variable, f, k _u , k _v , u ₀ , v ₀ and a variable related to the rotation and movement of the camera.

Is a matrix that contains all, if the three-dimensional object is projected on the two-dimensional plane and is not transformed into any shape, but the change in form is limited by the equation (6).

In practice, it is almost impossible to measure the camera internal and external variables in order to calculate the matrix P. Therefore, the coordinates in the actual three-dimensional space and the coordinates in the two-dimensional image must be measured and calculated. The following Equation 7 can be derived from the relationship between the coordinates in the three-dimensional space and the coordinates in the two-dimensional image.

[Equation 7]

,

,

,

,

Using Equation 7, the transformation matrix P can be calculated from the coordinates in the three-dimensional space and the coordinates in the two-dimensional image corresponding thereto. If we assume that the coordinates in the three-dimensional space are on the same plane, it can be simplified as shown in [Equation 8].

[Equation 8]

If the coordinates in the three-dimensional space do not exist on the same plane, it is necessary to know the height information of the three-dimensional space object to synthesize the image. However, since the surveillance camera is installed at a very high level, the height of the vehicle is negligible compared to the distance from the camera to the ground, so it can be assumed that the image coming from the camera is on the same plane as shown in [Equation 8].

Assuming that the coordinates in the three-dimensional space are on the same plane, a point in the three-dimensional space and four corresponding points on the image are required to calculate the matrix P.

4 is a diagram for describing a transformation of an image, according to an exemplary embodiment.

As shown in FIG. 4, the transformation of the image is performed by using a 3x3 transformation matrix P as shown in (a). That is, the image of the 3D coordinate system may be transformed into the image of the 2D coordinate system or the image of the 2D coordinate system may be transformed into the image of the 3D coordinate system.

Here, if in the image coming from the camera to know the coordinates of the four points are 4 points can be from calculating the transformation matrix P, by using the transformation matrix P in the reverse may be converted to any point on the camera image to a three-dimensional coordinate system image .

In this case, it is assumed that at least four coordinates in the camera image are known in advance. Such four coordinates may be predefined mark points or well-known landmarks.

In (b), the image of the actual 3D coordinate system is transformed into the image of the 2D coordinate system.

The image synthesizer 230 may generate one composite image by synthesizing the modified images.

5 is a diagram for describing synthesis of an image, according to an exemplary embodiment.

As shown in FIG. 5, when there are two cameras, a conversion matrix P for an image photographed by each individual camera is calculated and synthesized in a two-dimensional plane.

In the process of synthesizing the images, a portion where the images overlap with the images may occur. How smoothly this part is combined ensures the quality of the synthesized image. Smooth blending of the images at the overlapping portion is called image blending.

There are various methods for such a blending technique, for example, a method using an average value of two images and a method using a preset weight according to a distance from each image. Of course, the present invention is not necessarily limited thereto.

The image storage unit 240 may periodically store the generated synthetic image. The stored synthetic images may be provided to a monitor or a manager as needed.

6 is a diagram illustrating a method for synthesizing an image according to an embodiment of the present invention.

As illustrated in FIG. 6, an apparatus for synthesizing an image according to the present invention (hereinafter, referred to as an “image synthesizing apparatus”) may receive a compressed image from a camera by wire or wirelessly (S610).

Next, the image synthesizing apparatus may reconstruct the compressed images (S620) and transform the reconstructed images using the predefined transformation matrix (S630). That is, the image synthesizing apparatus may transform the reconstructed image of the 3D coordinate system into an image of the 2D coordinate system.

Next, the image synthesizing apparatus may generate one synthetic image by synthesizing the modified images (S640). In this case, the image synthesizing apparatus may combine the overlapping portions between the images synthesized in the process of synthesizing the images into one using an average value of the overlapping images. In the process of synthesizing the images, the composite imaging apparatus may combine the overlapping portions between the synthesized images into one by using a weight set in advance according to the distance from each of the overlapping images.

The above-described embodiments of the present invention can be embodied in a general-purpose digital computer that can be embodied as a program that can be executed by a computer and operates the program using a computer-readable recording medium. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (e.g., ROM, floppy disk, hard disk, etc.), optical reading medium (e.g., CD ROM,

Those skilled in the art of synthesizing the image in the surveillance camera system according to the present invention and its method will be capable of various modifications and variations without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

100: camera
200: video synthesis device
210: video input unit
220: image transformation unit
230: video synthesis unit
240: image storage unit

Claims (12)

An image input unit configured to receive a compressed image from a plurality of cameras in an area for monitoring a bridge or tunnel;
Reconstruct the received compressed images into three-dimensional coordinate system images and calculate a value of a predetermined transformation matrix based on four predefined marker points or coordinates of well-known landmarks in each of the restored three-dimensional coordinate system images. An image transformation unit for transforming the reconstructed 3D coordinate system images into 2D coordinate system images using the calculated transformation matrix; And
And an image synthesizer for synthesizing the transformed images to generate one synthesized image, wherein the transform matrix is a 3 × 3 matrix for allocating a zero value '0' to a height (z-axis). Device for synthesizing. delete delete The method according to claim 1,
Wherein the image synthesizer comprises:
And a portion overlapping the synthesized images in the process of synthesizing the images is synthesized into one using a predetermined blending technique. 5. The method of claim 4,
Wherein the image synthesizer comprises:
And an overlapping portion of the synthesized images in the synthesizing of the images is synthesized into one using an average value of the overlapping images. 5. The method of claim 4,
Wherein the image synthesizer comprises:
And an overlapping portion between the synthesized images in the synthesizing of the images is synthesized into one using a weight set in advance according to a distance from each of the overlapping images. (a) receiving a compressed image from a plurality of cameras in an area for monitoring a bridge or tunnel;
(b) reconstructing the received compressed images into three-dimensional coordinate system images, and based on the coordinates of four marker points or well-known landmarks defined in each of the restored three-dimensional coordinate system images, Transforming the reconstructed 3D coordinate system images into 2D coordinate system images using a calculated transformation matrix after calculating a value; And
(c) synthesizing the plurality of modified images to generate a single composite image,
Wherein the transformation matrix is a 3 × 3 matrix that assigns a zero value ('0') to a height (z-axis). delete delete The method of claim 7, wherein
The step (c)
The overlapping portions of the synthesized images in the synthesizing of the images are synthesized into one using a predetermined blending technique. The method of claim 10,
The step (c)
And the overlapping portions of the synthesized images are synthesized into one by using the average value of the overlapping images. The method of claim 10,
The step (c)
The overlapping part between the synthesized images in the synthesizing of the images is synthesized into one by using a weight set in advance according to the distance from each of the overlapping images.

Images