KR20130120914A - Control method of pan/tilt/zoom camera apparatus using stereo camera - Google Patents

Control method of pan/tilt/zoom camera apparatus using stereo camera Download PDF

Info

Publication number
KR20130120914A
KR20130120914A KR1020120044150A KR20120044150A KR20130120914A KR 20130120914 A KR20130120914 A KR 20130120914A KR 1020120044150 A KR1020120044150 A KR 1020120044150A KR 20120044150 A KR20120044150 A KR 20120044150A KR 20130120914 A KR20130120914 A KR 20130120914A
Authority
KR
South Korea
Prior art keywords
camera
tilt
stereo camera
image
zoom
Prior art date
Application number
KR1020120044150A
Other languages
Korean (ko)
Inventor
김중구
강전구
유재형
김종호
Original Assignee
(주) 비전에스티
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by (주) 비전에스티 filed Critical (주) 비전에스티
Priority to KR1020120044150A priority Critical patent/KR20130120914A/en
Priority to PCT/KR2013/003194 priority patent/WO2013162198A1/en
Publication of KR20130120914A publication Critical patent/KR20130120914A/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Image Analysis (AREA)

Abstract

A stereo camera monitors an entire observation area two-dimensionally and three-dimensionally (Two-dimensional level monitoring means that images are obtained from both cameras and three-dimensional monitoring means that three-dimensional depth information is calculated in real time.) and people are searched for based on the monitoring result. With regard to the people found, enlarged individual images including each person’s face are obtained by controlling pan, tilt and zoom cameras using the three-dimensional depth information calculated by the stereo camera. [Reference numerals] (109) Detect human, Calculate position of human, Calculate pan, tilt and zoom;(110) Detect face and transmit face image;(111) Store total image and face image

Description

Control method of pan / tilt / zoom camera apparatus using stereo camera

The present invention relates to a pan / tilt / zoom camera apparatus and a control method using a stereo camera.

A stereo camera is a camera that converts a target object into an image and calculates the image as 3D depth information using two cameras on the left and right sides.

The pan, tilt and zoom cameras consist of pans moving in the left and right directions, tilts moving in the vertical direction and one zoom camera.

The 3D depth information of the viewing area obtained from the stereo camera is transmitted to the pan / tilt / zoom camera to properly control the pan, tilt and zoom of the pan / tilt / zoom camera and obtain an enlarged image of the target object. The present invention relates to a pan / tilt / zoom camera apparatus and a control method using a stereo camera.

The security camera is used to acquire an observation area image through a camera arranged toward the observation area and to analyze the image when necessary.

However, because the viewing area is wide and the resolution of the camera is fixed, it is very difficult to closely monitor the whole viewing area. The wider field of view requires a higher resolution camera, and the higher resolution camera requires more data processing and therefore a higher specification system. After all, high-end systems require a lot of cost, which creates a cost problem in building a security camera.

Even if the resolution is increased to HD level, when the viewing area is enlarged, the approximate figure of a person can be stored, but it is difficult to obtain a clear image enough to identify a face.

As the observation area becomes wider, this problem becomes more serious. Consequently, even if a high-end system is built at a high cost, there is a problem that a user cannot build a clear image as much as the user wants.

When the area to be observed using the camera is wide and the resolution of the camera is constant, an image of the entire observation area can be obtained, but it is difficult to secure the face of the person at each position clearly.

In order to solve this problem, a zoom camera can be used to enlarge each area to obtain a clear image, but in this case, there is a problem of missing the entire image.

While acquiring the entire image of the observation area, in order to acquire an image in which each person in the observation area is clearly and clearly enlarged, an enlarged image of each person in the entire area must be separately acquired.

In addition, in order to obtain an enlarged image including a face part for each person, the zoom ratio of the zoom camera should be properly controlled.

As a method of calculating 3D depth information of an observation region in real time, Patent Application No. 10-2003-0062150 (Image Matching Method Using Multiple Image Lines and Its System) is used.

The technical problem to be solved by the present invention is to obtain the entire image of the observation area, while acquiring an individual image of each person in the observation area, a pan / tilt / zoom camera using a stereo camera that can obtain the image including the face portion An apparatus and a control method are provided.

In accordance with an aspect of the present invention, a pan / tilt / zoom camera apparatus and a control method using a stereo camera according to the present invention include a zoom camera for obtaining an image by enlarging or reducing a target object, one side of which is connected to the zoom camera, and a zoom camera. A fan motor operative to rotate left and right, a tilt motor connected to one side of the fan motor and operative to rotate the zoom camera up and down, and a stereo camera facing the same direction as the zoom camera, wherein the fan motor, The tilt motor and the zoom camera are electrically connected to the stereo camera, and control and photograph the pan motor, the tilt motor, and the zoom camera by using 3D position information of the target object detected by the stereo camera. Pan / Tilt / Zoom camera device and control method using stereo camera .

The pan / tilt / zoom camera apparatus and control method using the stereo camera according to the present invention monitor the entire observation area using the stereo camera. Find the person by referring to the 3D depth information about the area where the movement occurs during the monitoring by the stereo camera, and then transfer the 3D position information about each person to the pan, tilt and zoom camera to enlarge the upper body including the face of each person. Get There is an advantage in that an effective system can be constructed by storing the image of the upper body including the face in a storage device.

1 is a block diagram of a pan / tilt / zoom camera device and a control method using a stereo camera according to the present invention.
2 is an operation flowchart of a pan / tilt / zoom camera apparatus and a control method using a stereo camera according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a block diagram of a pan / tilt / zoom camera device and a control method using a stereo camera according to the present invention.

1, the zoom camera 101 is installed on the fan motor 102, the fan motor 102 is installed on the tilt motor 103. The position of the zoom camera, pan motor and tilt motor can be changed in some cases.

The stereo camera 104 is a camera composed of two left and right cameras, and acquires an image through both cameras and simultaneously calculates 3D depth maps of the observation area 105 in real time.

Both the zoom camera 101 and the stereo camera 104 face the viewing area 105 and are used to recognize people 106, 107, 108 and the like.

A person is detected by an image output from a stereo camera, a position of the person is calculated, a value for moving the pan, tilt, and zoom camera is calculated (109), and the image is sent to the pan, tilt, and zoom camera for control.

The human face is detected by an image obtained using a pan, tilt and zoom camera, and the facial image is transmitted to a recording medium such as a PC or a DVR (110).

Finally, the transmitted full image and face image are stored 111.

2 is an operation flowchart of a pan / tilt / zoom camera apparatus and a control method using a stereo camera according to the present invention.

Briefly referring to an operation flowchart of a pan / tilt / zoom camera apparatus and a control method using a stereo camera according to the present invention, the stereo camera 104 uses 3D depth information 203 (or display) using both images 201 and 202. Print a disparity map. One of the images 201 or 202 is used to monitor the entire image of the observation area 105. In operation 204, a portion in which a motion occurs in the entire monitored image is detected. The presence or absence of people 106, 107, and 108 is determined by referring to 3D depth information 203 (or disperity map) corresponding to the detected area (205). Areas identified as people 106, 107, and 108 refer to 206 3D location information (x, y, z) of the stereo camera 104 and control the pan, tilt and zoom cameras to point to that location ( 207). The camera is directed toward the object by moving the pan and tilt motors 103 in the x and y directions. By moving the zoom motor by the distance z, images of people 106, 107, and 108 at each position are individually acquired (208). In order to obtain an upper body image including a face from the acquired persons 106, 107, and 108 images, the image is stored for the case where the image is recognized as the image 209 including the face by performing a face presence process based on the obtained image. do. If not, the image is acquired again.

Looking at the operation sequence of the pan / tilt / zoom camera device and a control method using a stereo camera according to the present invention in detail,

The pan / tilt / zoom camera apparatus using a stereo camera is composed of a stereo camera 104, a pan motor, a tilt motor, and a zoom camera.

The stereo camera 104 acquires both images of the entire observation region 105. At the same time, the 3D depth information 203 (or disperity map) is calculated. The stereo camera 104 outputs 3D depth information 203 (or disperity map) of the image simultaneously with the images obtained through both cameras. Through the 3D depth information 203 (or disperity map), 3D location information of each point of the image may be calculated by the following equation.

Figure pat00001

Figure pat00002

Figure pat00003

In Equations 1 to 3, B is the distance between both cameras of the stereo camera 104, F is the lens focal length of the stereo camera 104, Dp is the disparity value, Pz is used in the stereo camera 104 Pixel horizontal size of the sensor, W is the horizontal length of the left image of the stereo camera 104, H is the vertical length of the left image of the stereo camera 104. In addition to the left image, the right image can be used.

It is not necessary to use the stereo camera 104 to calculate the 3D depth information 203 (or disparity map). The same effect can also be obtained by using a measuring device that obtains other types of 3D depth information 203 (or disperity map). However, when the stereo camera 104 is not used, a separate camera for monitoring the entire observation area should be mounted.

The pan, tilt and zoom cameras are composed of a pan motor 102, a tilt motor 103, and a zoom camera 101. The fan motor 102 allows the camera to move left and right. The tilt motor 103 moves the camera up and down. The zoom motor included in the zoom camera 101 may acquire an enlarged image through zoom in and zoom out.

The left image and the 3D depth information 203 (or disperity map) are used among the outputs of the stereo camera 104. The left image monitors the entire observation area. The motion area is detected in the entire image. Motion detection is obtained by storing the acquired image as a reference image when there is no motion for a predetermined time and using the difference between the stored reference image and the acquired image. The same result can be obtained by using the right image in addition to the left image.

In addition, referring to the 3D depth information 203 (or disperity map) of the stereo camera 104 corresponding to the region where the movement occurs, the person 106 may be utilized by utilizing the area information of the person 106, 107, 108 according to the distance. , 107, 108) Improves the accuracy of discriminating presence or absence. Since people in the near distance (106, 107, 108) and people in the distance (106, 107, 108) have an area difference in the image, it can be used to increase the accuracy of recognition of the people (106, 107, 108). have.

That is, the presence or absence of people 106, 107, and 108 may be more accurately determined by examining the distance information obtained through the stereo camera 104 and the area corresponding to the motion area obtained through the motion detection.

After using the stereo camera 104 to find the people 106, 107, 108 existing in the observation area 105, 3D location information for each of the people 106, 107, 108 found is calculated. When the left and right directions are the x-axis, the up and down direction is the y-axis, and the distance from the camera to the z-axis, the 3D position information (Dx, Dy) of the object from the stereo camera 104 is referred to. , Dz) is calculated as follows.

Figure pat00004

Figure pat00005

Figure pat00006

In Equations 4 to 6, B is a distance between both cameras of the stereo camera 104, F is a lens focal length of the stereo camera 104, Dp is a disparity value, Pz is a sensor used in the stereo camera 104 The horizontal width of the pixel, W is the horizontal length of the left image of the stereo camera 104, H is the vertical length of the left image of the stereo camera 104. In addition to the left image, the right image can be used.

Through the above equations, the individual 3D location information of the people 106, 107, and 108 in the observation area 105 can automatically obtain a magnified image of each person 106, 107, and 108. Passed to pan, tilt and zoom camera.

The pan, tilt, and zoom cameras all have three motors. That is, the fan motor 102 rotates in the left and right directions, the tilt motor 103 rotates in the vertical direction, and the zoom motor that controls the zoom of the camera. Based on the 3D location information (Dx, Dy, Dz) of each person 106, 107, 108 found in the stereo camera 104, the respective cameras 106, 107, 108 is obtained (208). The control of the pan, tilt and zoom motors is performed by the following equation.

Figure pat00007

Where Px is the rotation angle of the fan motor 102 (-90 degrees to 90 degrees), Pmax is the maximum rotation angle of the fan motor (-90 degrees or 90 degrees), Pmin is the minimum rotation angle of the fan motor 102 (0 degrees) , MaxDx is the maximum position X value that the object can exist, Dx is the position X value of the object calculated by the stereo camera 104.

Figure pat00008

Where Ty is the rotation angle of the tilt motor 103 (-90 degrees to 90 degrees), Tmax is the maximum rotation angle of the tilt motor (-90 degrees or 90 degrees), Tmin is the minimum rotation angle of the tilt motor (0 degrees), MaxDy is the maximum position Y value at which the object may exist, and Dy is a position Y value of the object calculated by the stereo camera 104.

Figure pat00009

Where Z is the zoom motor magnification value, zoom motor maximum magnification value, Zmin is the zoom motor minimum magnification value, MaxDz is the maximum position Z value at which the object can exist, and Dz is the position Z value of the object calculated by the stereo camera 104. to be.

The pan, tilt, and zoom cameras receive the object (or human) 3D position (Dx, Dy, Dz) information from the stereo camera 104, and rotate the pan, tilt, and zoom motors, respectively, through the above equations to obtain the final zoom. Acquire an image.

The pan, tilt, and zoom cameras may acquire an enlarged upper body image of the person 106, 107, or 108 by using 3D location information about an individual, but may not include a face part. Therefore, it is necessary to determine whether a face is included in the acquired image. As a method of determining whether a face exists in the acquired image, various face detection algorithms generally known to date can be used.

When the face is detected 209 in the acquired image, the acquired image is stored. If not detected, it is recorded that the face image is not obtained. Reshooting is performed until a face image is obtained.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope of the present invention.

101: zoom camera 102: fan motor
103: tilt motor 104: stereo camera
105: observation area 106, 107, 108: human

Claims (7)

A zoom camera for obtaining an image by enlarging or reducing a target object;
A fan motor connected to one side of the zoom camera and configured to rotate the zoom camera left and right;
A tilt motor connected to one side of the fan motor and operative to rotate the zoom camera up and down; And
A stereo camera facing the same direction as the zoom camera;
The pan motor, the tilt motor and the zoom camera are electrically connected to the stereo camera,
The pan / tilt / zoom camera apparatus using a stereo camera, characterized in that by using the 3D position information of the target object detected by the stereo camera to control the pan motor, the tilt motor and the zoom camera to shoot. Monitoring the entire image of the observation area using either image of both images of the stereo camera;
Detecting a movement of the target object detected by the stereo camera and determining whether there is a presence;
Calculating 3D location information when the target object exists;
Calculating and controlling a value for adjusting the pan motor, the tilt motor, and the zoom camera according to the 3D position information; And
Pan / Tilt / Zoom camera control method using a stereo camera, characterized in that it comprises the step of shooting the target object. The method of claim 1, wherein the detecting of the movement of the target object detected by the stereo camera and determining whether there is a presence,
The 3D depth information is calculated by acquiring the left and right images of the entire observation area in the stereo camera,
And detecting a movement using the one image and 3D depth information, and comparing the area of the target object according to distance to recognize whether the target object exists. The method of claim 3, wherein the step of calculating the 3D location information when the target object exists,
The method of controlling a pan / tilt / zoom camera using a stereo camera calculates 3D position information with respect to the detected target object by calculating the distance to the target object from left to right, up and down direction, based on the stereo camera. The method of claim 4, wherein calculating and controlling a value for adjusting the fan motor, the tilt motor, and the zoom camera according to the 3D position information comprises:
The target object is a human,
And controlling the pan motor, the tilt motor, and the zoom camera based on the 3D position information to obtain and adjust a value of the upper body image of the person. 6. The method of claim 5,
Analyzing the acquired upper body image of the person to determine whether the face of the person is detected,
And a method of controlling a pan / tilt / zoom camera using a stereo camera when the face of the person is detected. The method according to claim 6,
Analyzing the acquired upper body image of the person to determine whether the face of the person is detected,
And recording a failure to obtain a face image when the face of the person is not detected, and continuously photographing until the face of the person is detected.
KR1020120044150A 2012-04-26 2012-04-26 Control method of pan/tilt/zoom camera apparatus using stereo camera KR20130120914A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020120044150A KR20130120914A (en) 2012-04-26 2012-04-26 Control method of pan/tilt/zoom camera apparatus using stereo camera
PCT/KR2013/003194 WO2013162198A1 (en) 2012-04-26 2013-04-16 Surveillance system and method for controlling same

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020120044150A KR20130120914A (en) 2012-04-26 2012-04-26 Control method of pan/tilt/zoom camera apparatus using stereo camera

Publications (1)

Publication Number Publication Date
KR20130120914A true KR20130120914A (en) 2013-11-05

Family

ID=49483436

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020120044150A KR20130120914A (en) 2012-04-26 2012-04-26 Control method of pan/tilt/zoom camera apparatus using stereo camera

Country Status (2)

Country Link
KR (1) KR20130120914A (en)
WO (1) WO2013162198A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015167738A1 (en) * 2014-04-30 2015-11-05 Replay Technologies Inc. System and method of limiting processing by a 3d reconstruction system of an environment in a 3d reconstruction of an event occurring in an event space
KR102400007B1 (en) * 2021-12-17 2022-05-19 주식회사 굿링크 Surveillance camera system inculding lidar sensor

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09322053A (en) * 1996-05-24 1997-12-12 Nippon Hoso Kyokai <Nhk> Image pickup method for object in automatic image pickup camera system
JP2002101408A (en) * 2000-09-22 2002-04-05 Asahi Precision Co Ltd Supervisory camera system
KR100519782B1 (en) * 2004-03-04 2005-10-07 삼성전자주식회사 Method and apparatus for detecting people using a stereo camera
KR100691348B1 (en) * 2005-12-21 2007-03-12 고정환 Method for tracking moving target with using stereo camera based on pan/tilt contol and system implementing thereof

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015167738A1 (en) * 2014-04-30 2015-11-05 Replay Technologies Inc. System and method of limiting processing by a 3d reconstruction system of an environment in a 3d reconstruction of an event occurring in an event space
US9846961B2 (en) 2014-04-30 2017-12-19 Intel Corporation System and method of limiting processing by a 3D reconstruction system of an environment in a 3D reconstruction of an event occurring in an event space
US10063851B2 (en) 2014-04-30 2018-08-28 Intel Corporation System for and method of generating user-selectable novel views on a viewing device
US10477189B2 (en) 2014-04-30 2019-11-12 Intel Corporation System and method of multi-view reconstruction with user-selectable novel views
US10491887B2 (en) 2014-04-30 2019-11-26 Intel Corporation System and method of limiting processing by a 3D reconstruction system of an environment in a 3D reconstruction of an event occurring in an event space
US10567740B2 (en) 2014-04-30 2020-02-18 Intel Corporation System for and method of generating user-selectable novel views on a viewing device
US10728528B2 (en) 2014-04-30 2020-07-28 Intel Corporation System for and method of social interaction using user-selectable novel views
US11463678B2 (en) 2014-04-30 2022-10-04 Intel Corporation System for and method of social interaction using user-selectable novel views
KR102400007B1 (en) * 2021-12-17 2022-05-19 주식회사 굿링크 Surveillance camera system inculding lidar sensor

Also Published As

Publication number Publication date
WO2013162198A1 (en) 2013-10-31

Similar Documents

Publication Publication Date Title
JP4568009B2 (en) Monitoring device with camera cooperation
CN104902246B (en) Video monitoring method and device
CN105574847B (en) Camera system and image registration method thereof
WO2012023766A2 (en) Security camera tracking and monitoring system and method using thermal image coordinates
TWI520576B (en) Method and system for converting 2d images to 3d images and computer-readable medium
JP4929828B2 (en) Three-dimensional authentication method, three-dimensional authentication device, and three-dimensional authentication program
EP2717013A1 (en) Stereo camera device and computer-readable recording medium
JP4979525B2 (en) Multi camera system
EP2719160A2 (en) Dual-resolution 3d scanner
JP2005117542A (en) Monitoring system
JP5007863B2 (en) 3D object position measuring device
TW201044858A (en) Imaging apparatus, imaging control method, and recording medium
CN110136047B (en) Method for acquiring three-dimensional information of static target in vehicle-mounted monocular image
KR100793077B1 (en) Method for deciding the position of an observer, and an observer tracking 3D display system and method using that
JP2013115668A (en) Image processing apparatus, image processing method, and program
KR20130039844A (en) Intelligent camera appartus and intelligent surveillance metod using the same
JP2005142683A (en) Apparatus and method for camera control
JP4198536B2 (en) Object photographing apparatus, object photographing method and object photographing program
CN111726515A (en) Depth camera system
KR20130120914A (en) Control method of pan/tilt/zoom camera apparatus using stereo camera
JP2020088840A (en) Monitoring device, monitoring system, monitoring method, and monitoring program
TW201205449A (en) Video camera and a controlling method thereof
KR20160038297A (en) Apparatus and method for guiding correct posture of medical image system
JP6581280B1 (en) Monitoring device, monitoring system, monitoring method, monitoring program
JP5007786B2 (en) Video surveillance apparatus and method

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E601 Decision to refuse application