KR101020921B1 - Controlling Method For Rotate Type Camera - Google Patents

Abstract

Embodiments relate to a method of controlling a rotatable camera.

A control method of a rotatable camera according to an embodiment may include generating a pan and tilt lookup table for an image input to a fixed camera; Calculating measurement information of an object area from camera information and image conversion of an image input through the fixed camera; And automatically expanding and aligning the rotatable camera to the object area.

Calibration, Fixed Camera, Rotating Camera

Description

Controlling Method For Rotate Type Camera}

Embodiments relate to a method of controlling a rotatable camera.

Surveillance using fixed cameras and rotating cameras such as pans, tilts and zoom cameras can be classified into two types, depending on the camera installation location. A fixed camera and a rotating camera are installed on the same axis, and the control value for rotating camera control is looked up by converting the rectangular coordinate system of the fixed camera into the cylindrical coordinate system of the rotating camera and by adjusting the coordinates between the cameras at different positions. After storing it in a look-up table, a rotation camera control is performed with reference to this value.

Such a prior art requires a step of manually designating a position to be enlarged with respect to an image input from a fixed camera by a mouse or the like, and in the step of automatically aligning the center of the rotatable camera with respect to a point designated by the user. When generating the lookup table, the magnification is applied to fit the size of the defined object.

Figure 1a is an image of a fixed camera according to the prior art, Figure 1b is an image of a rotary camera according to the prior art.

However, such a prior art requires the operator's intervention for automatic magnification alignment when operating the system using the corresponding technology, and the size of the object for applying the automatic alignment of the rotating camera based on the fixed camera must be fixed and not variable. do.

For example, according to the related art, as shown in the case of photographing the license plate 20 of the fixed vehicle 10 as shown in FIGS. 1A and 1B, the seal size of the fixed camera may be applied only to a fixed object.

This method has a problem that it is not possible to auto-align for objects of various sizes that can appear in the image, it is difficult to provide the image information that can be determined by the important operator in the object tracking system.

Embodiments provide a control system and a control method of a rotating camera capable of providing an image that can be determined by an operator regardless of the size and distance of an object as a subject.

A control method of a rotatable camera according to an embodiment may include generating a pan and tilt lookup table for an image input to a fixed camera; Calculating measurement information of an object area from camera information and image conversion of an image input through the fixed camera; And automatically expanding and aligning the rotatable camera to the object area.

The embodiment uses a rotating camera installed adjacent to a fixed camera and performs coordinate correction between the two cameras, and then the actual distance and size value of the object to be tracked through the projective transformation of the image obtained from the fixed camera. By controlling the zoom value of the rotatable camera can provide an image that can be determined by the operator regardless of the size and distance of the object in the center of the image.

In addition, according to an embodiment, when tracking an object by interlocking a fixed camera and a rotating camera, the limitation of physical position can be minimized, and the rotating camera is controlled by applying a width and height value of a real object in controlling the rotating camera. Even without manual operation, the size of objects such as people and vehicles inside the camera view does not deviate, and the operator can automatically arrange the image into a size that can be judged.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

(Example)

2A is a conceptual diagram of a rotatable camera control system according to an embodiment.

A control method of a rotatable camera according to an embodiment may include generating a pan and tilt lookup table for an image input to a fixed camera; Calculating measurement information of an object area from camera information and image conversion of an image input through the fixed camera; And automatically expanding and aligning the rotatable camera to the object area.

The embodiment may further include removing the vanishing point by a projective transform on an image input to the fixed camera before calculating the measurement information of the object region.

The calculating of the measurement information of the object area may calculate the measurement information by obtaining a measurement value per unit pixel based on the image converted image.

Automatically expanding and aligning the rotatable camera 50 to the object area may include a height ratio of the height and width of the object area extracted from the image of the fixed camera 40 in the vertical direction of the image sensor of the rotatable camera 50. If it is larger than the ratio of size and width, it can be sorted by height, and if it is small, it can be sorted by width. The image sensor of the rotatable camera 50 may be a charge-coupled device (CCD) image sensor or a CIS (COMS Image Sensor), but is not limited thereto.

The rotary camera 50 of the embodiment may be a pan, tilt, zoom camera, but is not limited thereto.

The embodiment uses the rotating camera 50 installed adjacent to the fixed camera 40 and performs coordinate correction between the two cameras, and then tracks through the projective transformation of the image obtained from the fixed camera 40. By calculating a real distance and a size value of an object to be controlled, a zoom value of the rotatable camera 50 may be controlled to provide an image that an operator can determine in the center of the image regardless of the size and distance of the object.

Further, according to the embodiment, when the object is tracked by interlocking the fixed camera 40 and the rotating camera 50, the limitation of the physical position can be minimized, and the width of the real object is controlled in the rotating camera 50. By applying the height value and the height of the object, such as a person, a vehicle in the camera view without manual operation of the rotatable camera 50 can be automatically sorted into an image of the size that can be determined by the operator.

The embodiment is a rotating camera 50 through a coordinate correction in a technique of tracking an object using a rotating camera 50 installed at a different position from a fixed camera 40 having the same resolution, for example, a pan, tilt, and zoom camera. A look-up table is constructed by obtaining a pan and tilt value of 50, and according to the size value (width, height) of the real object according to the projected transformation of the fixed camera 40 image and the input of the sample object size, By applying a zoom control value for an object such as a vehicle differentially, an image that can be determined by an operator may be provided to the rotatable camera 50, for example, a pan / tilt / zoom camera. In this embodiment, the focus value may use the camera's autofocus function.

2B is a flow chart of a control system for a rotatable camera 50 according to an embodiment.

An embodiment may include the following steps. First, the height, the tilt value of the fixed camera 40, the distance value with the rotatable camera 50, the size value of the image sensor, and the wide-angle focus distance of the rotatable camera 50 may be used as initial input values. Thereafter, the measured distance lookup table, the pan, and the tilt lookup table may be generated. Subsequently, a projective transform may be performed on the image of the fixed camera 40. Thereafter, the input of the sample object size value and the calculation of the length value per unit cell can be performed. Thereafter, the telephoto focal length value calculation and the object alignment method of the rotatable camera 50 may be selected. Thereafter, control and tracking of the rotatable camera 50 may be performed.

2B is a flowchart illustrating a method of controlling a rotatable camera according to an embodiment.

Referring to FIG. 2B, the rotating camera control method according to the embodiment includes a lookup table generation step S210, a moving object detection step S220, an object silk machine and a real distance calculation step S230, an object's center of gravity, and an alignment criterion. The calculating step S240, the horizontal / receiving angle and the zoom magnification value calculating step S250, and the rotating camera control step S260 may be included.

In addition, the lookup table generation step S210 may include the following steps.

Using the camera information from the image captured by the fixed camera 40, the measured distance lookup table S211 of the X and Y axes may be generated for all regions within the image.

Subsequently, the calculated X-axis and Y-axis angles and diagonal length lookup tables S212 of the rotating camera may be generated for all regions within the image using the calculated measurement distance table.

In addition, the embodiment may include the following steps as in the flowchart of FIG. 2B.

An image is input to the fixed camera (S205), and a moving object detection step (S220) of detecting a moving object from the image captured by the fixed camera 40.

Thereafter, the detected object may calculate a silk machine of the height and width of the object using the lookup table S211 (S230), and the image of the object through the projective transform on the image of the fixed camera 40. It may include the step of calculating the measured size.

Subsequently, the method may include calculating an alignment criterion by using a ratio of the position of the center of gravity cell of the object extracted in the image and the horizontal / vertical size of the object (S240).

Then, the position and alignment criteria of the center of gravity cell of the calculated object are extracted with reference to the look-up table S212 to extract the horizontal / vertical angle at which the detected object is located from the rotatable camera 50. In step S250, the zoom magnification value is used.

Subsequently, by automatically controlling the rotatable camera 50 at the calculated horizontal / vertical angle, the object is aligned in the center of the image of the rotatable camera 50, and the object is positioned in the image using the calculated zoom magnification value. It includes a rotary camera 50 automatic control step (S260) to enlarge and align all included.

Specifically, the embodiment may include the following steps. The measured distance lookup table of the X-axis and the Y-axis is calculated for all regions inside the image acquired from the fixed camera 40. Subsequently, the X-axis, Y-axis angle, and diagonal length lookup table of the rotating camera may be calculated for all regions within the image using the calculated measurement distance table. Subsequently, the silk machine of the height and width of the object may be calculated using the calculated measured distance lookup table. Subsequently, the silk size of the object may be calculated by calculating a measured size per unit cell by applying camera calibration to the input image acquired from the fixed camera 40. Thereafter, the zoom magnification value may be calculated using the calculated silk machine. Thereafter, the object may be tracked through the automatic rotation camera alignment using the calculated angle table and the calculated zoom magnification value.

The embodiment uses, for example, Inigo's distance measurement method or the like, using the actual distance information from the camera for each pixel in the image and the distance information of two cameras away from each other, such as pan (P), tilt (T), and zoom ( Z), the focus (F) value can be calculated, but is not limited thereto. Inigo's proposed three-dimensional actual distance calculation method for each point in a two-dimensional image includes the focal length of the camera, the size of the CCD, the angle of view for the X and Y axes, the height of the camera (h _y ), and the bottom of the image. If the distance Dminy is known, the actual distance of all points in the image acquired from the fixed camera 40 can be obtained.

For example, the look-up table may be generated as shown in Table 1 below by obtaining the actual X-axis distance and the Y-axis distance for all regions inside the fixed camera 40 image.

Area coordinates in the image X axis distance (dx) Y-axis distance (dy) (0,0) 2000 5000 (0,1) 2000 4980 (0,2) 2000 4960

Next, a pan and tilt lookup table for the image input to the fixed camera 40 is generated. For example, a pan / tilt lookup table for all points of the fixed camera 40 may be generated using the formula of Inigo, but is not limited thereto.

The embodiment finds the pan P and tilt T values of the rotatable camera 50 at any point C (x, y). The pan P value corresponds to the horizontal angle of the rotatable camera 50 and corresponds to θ, and the tilt T value corresponds to the longitudinal angle of the rotatable camera 50 and corresponds to φ. The pan value θ and the tilt value φ may be calculated by using Equation 1 below to calculate the angle between the X and Y axes of the rotating camera 50, but is not limited thereto. In Equation 1, D is a distance between the fixed camera 40 and the rotary camera 50.

After extracting the measured distances dx and dy of C (x, y) from the calculated measured distance lookup tables, the distance h _x from the rotating camera 50 to the subject is obtained using Equation 2 below.

By using Equation 2, a lookup table may be calculated as shown in Table 2 by obtaining the angles and the diagonal distances of the X-axis and the Y-axis with respect to the rotating camera 50 for all regions in the image.

Area coordinates in the image Horizontal rotation angle (θ) Vertical rotation angle (φ) Distance from camera (h _x ) (0,0) 10 80 6000 (0,1) 10 79 5980 (0,2) 10 78 5960

Next, the embodiment may further include removing the vanishing point by a projective transform on the image input to the fixed camera 40 before calculating the measurement information of the object region.

The calculating of the measurement information of the object area may calculate the measurement information by obtaining a measurement value per unit pixel based on the image converted image.

That is, the embodiment may proceed with the step of generating the 2D vanishing point removing image through the projective transformation on the input image of the fixed camera 40.

3A is an example of an image from the stationary camera 40 in the rotating camera 50 control system according to the embodiment, and FIG. 3B is a view of the stationary camera 40 in the rotating camera 50 control system according to the embodiment. Is an illustration of the vanishing point.

A general surveillance camera is installed at a constant height to form an angle of view downward. In this installation environment, the image input from the camera expresses the 3D image in 2D. In this process, parallel lines of the real world meet at one vanishing point as shown in FIGS. 3A and 3B.

In particular, when the vertical angle of the camera is large, the perspective of the image due to the vanishing point is severe. In this case, the silk error of the estimated object may increase.

Therefore, the embodiment further includes the step of eliminating the vanishing point by the projective transform on the image input to the fixed camera 40, thereby reducing the silk error of the step of calculating the measurement information of the object area. There is this.

That is, the embodiment calculates the measured value per unit pixel based on the image converted image in the step of calculating the measured information of the object region and calculates the measured information, thereby reducing the silk error. In the automatic image magnification sorting step, images may be magnified and aligned based on accurate measured data.

In a typical camera installation environment, one or more vanishing points exist in the image, and the distance value per unit cell of the image is different due to the perspective generated due to the vanishing point. In order to solve this problem, the image is converted to FIG.

In the image of FIG. 3C, four points are set and input based on a path generated by perspective, and four points are converted and input again in the form of an image to be converted.

This removes the image distortion caused by the vanishing point through the projective transformation.

4 is a conceptual diagram of an image conversion image of the image of the fixed camera 40 in the control system for the rotating camera 50 according to the embodiment.

Equation 3 is used for projection to different planes in the same coordinate system using the center point as shown in FIG. 4.

In Equation 3, Matrix H is called a homogeneous matrix. Here, the 2D image inputted from the image can be matched with the real world coordinate system as shown in FIG. 4, and the point Ｘ (ｘ, ｙ) of the Π plane in the inhomogeneous coordinate system is the Ｘ ＇(ｘ＇, ｙ ＇) of the Π＇ plane. ). In the inhomogeneous coordinate system, projective transformation is represented by Equation 4.

If Equation 4 is expressed as two polynomials, it is expressed as Equation 5.

By using Equation 5, elements of the projective matrix are determined using four pairs of eight different positions corresponding to each other, such as X (x, y) and X ＇(x＇, y ＇) of FIG. 4. Obtain and convert every point of the image acquired by the camera.

Next, the measurement information of the object region may be calculated from the pixel information of the sample object region with respect to the image input through the fixed camera 40.

5A and 5B illustrate examples of calculating a length value per unit cell and calculating a silk value of an object through input of a sample object in the control system of the rotatable camera 50 according to the embodiment.

After extracting the coordinates of X ₁ (x ₁ , y ₁ ) and X ₂ (x ₂ , y ₂ ) of the extracted object, the silk width of the object width can be obtained using the silk lookup table. Using the following Equation 6 it can be obtained a silk group (x) per unit pixel.

The actual height of an object may be obtained using the silk group x per unit pixel.

In addition, in the embodiment, as shown in FIG. 5B, per unit cell in the horizontal direction by inputting length value information such as a road lane length, a street lamp height, or an actual value of an object having a size value known to a user, in an image converted image. The length value (m / pixel) can be obtained. Based on this value, the actual size (width, height) of the object appearing in the image can be obtained.

For example, in FIG. 5B, the object has a width of 0.65 m and a height of 1.75 m.

Next, a step of automatically expanding and aligning the image of the rotatable camera 50 in the object area may be performed.

For example, calculating a zoom magnification value through an input of a wide-angle focal length of the image sensor such as 1/4 ", 1/2", etc. and rotating the camera 50, and automatically aligning and enlarging the object to an object. You can proceed.

Using the look-up table obtained above, when the distance between the object, the size of the object, and the size of the image sensor is known from the fixed camera 40 and the rotating camera 50, the telephoto focus is input by inputting the wide-angle focal length of the given camera. The distance (focal length of the lens where the subject in the image fits the entire image size) can be obtained. Using the two focal length values, the zoom magnification can be determined through the telefocal length / wide-angle focal length.

6 is an exemplary view of zoom magnification determination in a control system for a rotatable camera 50 according to an embodiment.

In Figure 6, the following proportional formula is established,

,

,

Here, the size (V) of the image in the camera can be obtained by Equation 7 below.

7A is an exemplary diagram for determining a telephoto focal length in a rotating camera 50 control system according to an embodiment, and FIG. 7B is an enlarged exemplary view of an object in a rotating camera 50 control system according to an embodiment.

The telephoto focal length as shown in FIG. 7B can be obtained through Equation 8 below.

In an exemplary embodiment, the size of the image may be calculated as the total size of the CCD so that the subject may enter the full size in the image, but is not limited thereto.

When the focal length of the telephoto rotating camera 50 obtained through Equation 8 is divided by the wide-angle focusing distance, the zoom magnification value of the rotating camera 50 can be obtained.

In addition, the embodiment may obtain the size V2 of the subject in the rotatable camera 50 by using the distance between the subject and the two cameras, the image size V1 of the fixed camera 40, and the following proportional expression.

D1: V1 = D2: V2

At this time, D1 is the distance between the stationary camera 40 and the subject, D2 is the distance between the rotatable camera 50 and the subject, V1 is the size of the subject in the stationary camera 40, V2 is the rotary camera ( 50) is the size of the subject. The value of D2 can be obtained using the distance value between D1 and the two cameras.

Using the size (V2) of the subject in the rotatable camera 50, by using the equation (8) to obtain the focal length of the rotatable camera 50 when telephoto, through this zoom of the rotatable camera 50 Magnification values can also be obtained.

8A and 8B illustrate an example of an object auto-aligning image in the control system of the rotatable camera 50 according to the embodiment.

In the embodiment, the step of automatically expanding and aligning the rotatable camera 50 to the object region may include the ratio of the height and width of the object region extracted from the image of the fixed camera 40 and the rotation of the rotatable camera 50. By comparing the ratio between the vertical and horizontal sizes of the image sensor, it can be adaptively enlarged based on the height or width.

When the ratio of the height / width of the object region extracted from the image of the fixed camera 40 is greater than the ratio of the vertical size / horizontal size of the CCD of the rotating camera 50, the arrangement is based on the height. You can sort by width.

For example, the ratio of the height / width of the object extracted through the image processing of the fixed camera 40 is calculated, and the ratio of the vertical / horizontal size of the CCD of the rotating camera 50 as shown in FIG. 8A is greater than 3/4. In the case of having a large value, it is arranged based on the height value, and in the case of having a value smaller than 3/4, it is arranged in the image of the rotating camera 50 with respect to objects of various sizes by aligning by the width as shown in FIG. Can be.

In addition, the embodiment may extract the center of gravity coordinates after extracting the object from the fixed camera 40. On the other hand, the zoom can be controlled by adjusting the horizontal and vertical angles of the rotatable camera 50 by using the pan / tilt lookup table and calculating the zoom magnification value. As this process is repeated, the object extracted from the fixed camera 40 continues to be tracked by the rotating camera 50.

The embodiment uses the rotating camera 50 installed adjacent to the fixed camera 40 and performs coordinate correction between the two cameras, and then, through the projective transformation of the image obtained from the fixed camera 40. By calculating the real distance and size of the object to be tracked, the zoom value of the rotatable camera 50 is controlled to provide an image that can be determined by the operator regardless of the size and distance of the object in the center of the image. .

Further, according to the embodiment, when the object is tracked by interlocking the fixed camera 40 and the rotating camera 50, the limitation of the physical position can be minimized, and the width of the real object is controlled in the rotating camera 50. By applying the height value and the height of the object, such as a person, a vehicle in the camera view without manual operation of the rotatable camera 50 can be automatically sorted into an image of the size that can be determined by the operator.

The present invention is not limited to the described embodiments and drawings, and various other embodiments are possible within the scope of the claims.

Figure 1a is an image of a fixed camera according to the prior art.

Figure 1b is an image of a rotatable camera according to the prior art.

2A is a conceptual diagram of a rotatable camera control system according to an embodiment.

2B is a flowchart of a rotatable camera control system according to an embodiment.

3A is an illustration of an image in a stationary camera in a rotatable camera control system according to an embodiment.

3B is an illustration of a vanishing point in a stationary camera in a rotatable camera control system according to an embodiment.

3C is an illustration of an image of a stationary camera in a rotatable camera control system according to an embodiment.

3D illustrates an image conversion image for an image of a fixed camera in the rotating camera control system according to the embodiment.

4 is a conceptual diagram of an image conversion image for an image of a fixed camera in a rotating camera control system according to an embodiment.

5A and 5B illustrate examples of calculating a length value per unit cell and calculating a silk value of an object through input of a sample object in the rotatable camera control system according to the embodiment.

6 is an exemplary view of zoom magnification determination in a rotatable camera control system according to an embodiment.

7A is an illustration of a telephoto focal length determination in a rotatable camera control system according to an embodiment.

7B is an enlarged illustration of a subject in a rotatable camera control system according to an embodiment.

8A and 8B are diagrams illustrating an object auto-aligning image in the rotatable camera control system according to the embodiment.

Claims (5)

Generating a pan and tilt lookup table for an image input to the fixed camera; Removing vanishing points by a projective transform on an image input to the fixed camera; After the removing of the vanishing point, calculating camera information on an image input through the fixed camera and measurement information of an object region from the image transformation; And Automatically expanding the image of the rotating camera to the object area; Control method of a rotating camera comprising a. delete According to claim 1, Computing the measurement information of the object area, And calculating measurement information by obtaining an actual measurement value per unit pixel based on the image converted image. Generating a pan and tilt lookup table for an image input to the fixed camera; Calculating measurement information of an object area from camera information and image conversion of an image input through the fixed camera; And And automatically expanding and aligning the rotatable camera to the object area. Automatically expanding and aligning the rotatable camera to the object area may include: Comparing the ratio of the height and width of the object region extracted from the image of the fixed camera and the ratio of the vertical size and the horizontal size of the image sensor of the rotatable camera, the entire object area is expanded by adaptively expanding based on the height or width. Rotating camera control method to enlarge and align. Generating a pan and tilt lookup table for an image input to the fixed camera; Calculating measurement information of an object area from camera information and image conversion of an image input through the fixed camera; And And automatically expanding and aligning the rotatable camera to the object area. Automatically expanding and aligning the rotatable camera to the object area may include: If the ratio of the height / width of the object region extracted from the image of the fixed camera is larger than the ratio of the vertical size / horizontal size of the image sensor of the rotating camera to sort by the height, if small by the width Control method of rotating camera.

Images