KR20140036824A - Monitoring apparatus and system using 3d images, and method thereof - Google Patents

Abstract

Provided are a monitoring device and a monitoring system using a 3D image and a method using the same. The monitoring device using a 3D image according to one embodiment of the present invention includes: an interface unit interlocked with a plurality of PTZ cameras; an image obtaining unit obtaining 2D images photographed by the PTZ cameras through the interface unit; and an image conversion unit composing at least two images among the 2D images obtained by the image obtaining unit and converting the composed image into the 3D image. [Reference numerals] (210) Interface unit; (220) Image obtaining unit; (230) Image conversion unit; (250) Location tracing unit; (300) Database; (340) Information processing unit

Description

Monitoring apparatus and system using 3D images and method using the same

The present invention relates to a surveillance apparatus and system using a 3D image, and a method using the same. More particularly, a 3D image is obtained by using a PTZ (Pan Tilt Zoom) camera linked to a network, and the 3D image is acquired. The present invention relates to a monitoring apparatus and system using 3D video and a method of using the same, which can share a camera in real time through a network and monitor a moving subject.

The general method for acquiring a 3D screen requires two cameras to capture left and right images, and synthesizes two different images according to the angle difference to complete the final 3D screen.

In tracking a moving object (e.g., viewer), the position of the subject to be tracked is calculated by calculating an amount of image change on a 2D screen, not a stereoscopic image, and moving the focus of the PTZ camera to the position. For example, the position of the viewer is detected using the received image frame, and the position of the viewer is detected using at least one of the position of the detected viewer and the position of the previously detected viewer. Calculate. The viewer position is estimated by using the detected viewer position and the calculated moving speed, and the amount of stereoscopic image change is calculated based on the estimated viewer position. Then, the stereoscopic image is displayed based on the calculated stereoscopic image change amount.

As described above, the conventional technology focuses on calculating a viewer's position and displaying the image in an environment having a stereoscopic image display. In operating two stereoscopic cameras, a PTZ (Pan Tilt Zoom) function is provided. There is a difficulty in implementing a more free stereoscopic image.

Specifically, Utility Model Registration 20-0350161 discloses a three-dimensional image security control device that provides a security function to block the access by detecting the movement of the access object. Hereinafter, a description will be given with reference to FIG. 1.

1 is a block diagram of a conventional stereoscopic image security control apparatus disclosed in the utility model registration 20-0350161.

Referring to FIG. 1, the 3D image security control apparatus 10 uses the communication modules 70 and 80 to connect the 3D image cameras 90 and the driving motor 100 to various places of a security facility 110. ) And a configuration for automatically managing security by connecting to the speaker 120 is disclosed. Here, the communication module (70, 80) has a problem that can not be connected to a universal interface because each interface is dedicated. In addition, a dedicated three-dimensional image camera 90 must be installed and there is a problem that can not track the omnidirectional object in a wide place because it is not a PTZ camera that can continuously track the subject in front.

In addition, the theoretical algorithm of the conventional object tracking device is a method of finding a moving object by calculating the difference between the previous screen and the current screen in a 2D state (still image). This is often missed when the object is hidden behind another object or when it crosses another object. In other words, if you are in an environment where the success rate of automatic object tracking is very low, you will miss the object being tracked.

Korean Patent Publication No. 10-2011-0050843

The present invention is to solve the above problems, to obtain a three-dimensional image using a PTZ (Pan Tilt Zoom) camera linked to the network, and can be tracked while omnidirectional monitoring a moving subject while sharing through the network with other cameras The present invention provides a surveillance apparatus and system using 3D images and a method using the same.

In addition, it is possible to acquire the coordinates of a moving subject more accurately by converting the distance / height information of the subject using all the PTZ functions, and by using the 3D image monitoring apparatus and system, It is to provide a method using this.

Problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

Surveillance apparatus using a 3D image according to an embodiment of the present invention for achieving the above object, Interface unit for interlocking a plurality of PTZ camera; An image obtaining unit obtaining each 2D image picked up by the plurality of PTZ cameras through the interface unit; And an image converter configured to synthesize at least two images from each of the 2D images obtained by the image acquirer, and convert the two images into 3D images.

Surveillance system using a 3D image according to an embodiment of the present invention for achieving the above object, in the 3D image acquisition system for tracking the object by obtaining a 3D image by interlocking a plurality of PTZ cameras, each of the PTZ camera The interface unit is connected to another PTZ camera through a network; An image acquisition unit for capturing an object to obtain a 2D image; And an image converter configured to synthesize the 2D image obtained by the image acquisition unit with the 2D image of another PTZ camera transmitted through the network and convert the 2D image into a 3D image.

Surveillance method using a 3D image according to an embodiment of the present invention for achieving the above object, the step of obtaining a 2D image by shooting with a plurality of PTZ camera; Transmitting the 2D image between a plurality of PTZ cameras connected through a network; Converting the transmitted 2D image into a 3D image by synthesizing the photographed 2D image; Extracting location information of the object from the 2D image and extracting height information of the object from the 3D image; And tracking the position of the object based on the position information and the height information.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, it is possible to reduce costs by using a fixed PTZ camera without installing a separate mobile expensive 3D camera.

In addition, when using evidence of crime identification, distance information and overlapping objects that can be missed on existing 2D screens can be identified and separated into a plurality of objects.In a PTZ camera with an automatic tracking device, the success rate is seamless without this method. High object tracking is possible.

In addition, by using a plurality of network PTZ cameras using the Internet Protocol (IP) to acquire a stereoscopic image it can significantly improve the performance of the existing object tracking device.

In addition, since the object can be identified by adding the height information of the object calculated from the stereoscopic image acquired by using the plurality of image input devices, the performance of the automatic tracking device can be maximized. Therefore, there is no need to build a dedicated communication module.

If the user wants to track an object in the changed stereoscopic image, since the object has height information, it can exhibit superior performance than the automatic object tracking device based on the conventional 2D screen change measurement method.

1 is a block diagram of a conventional stereoscopic image security control apparatus.
2 is a block diagram of a monitoring apparatus using a 3D image according to an embodiment of the present invention.
3 is a block diagram of a surveillance system using a 3D image according to an embodiment of the present invention.
4 is a flowchart illustrating a monitoring method using 3D images according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense commonly understood by one of ordinary skill in the art to which this invention belongs. Also, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

2 is a block diagram of a monitoring apparatus using a 3D image according to an embodiment of the present invention.

Referring to FIG. 2, the monitoring apparatus 200 using the 3D image according to an embodiment of the present invention obtains images captured by the plurality of PTZ cameras 100, processes them, and converts the images into 3D stereoscopic images. It tracks moving objects using 3D stereoscopic images. The data acquired by the monitoring apparatus 200 using the 3D image and the data processed by the monitoring apparatus 200 using the 3D image are stored in the database 300.

The monitoring apparatus 200 using the 3D image includes an interface unit 210, an image acquisition unit 220, and an image conversion unit 230. In addition, the information processing unit 240 and the location tracking unit 250 may be further included.

The interface unit 210 serves to link the plurality of PTZ cameras 100. Here, the interface unit 210 may interwork a plurality of PTZ cameras 100 using a network, and in detail, may interlock a plurality of PTZ cameras 100 using IP (Internet Protocol). Accordingly, a separate dedicated communication module or method is not required, and only the interworking inside the system due to the conventional design of the prior art can be designed to overcome the disadvantage that the flexible interface design is difficult and the scalability is poor.

In particular, the plurality of PTZ cameras 100 are fixedly arranged at a specific position. That is, since the PTZ camera 100 is installed in a state where a plurality of 100-1, 100-2, ..., 100-n are facing each other, the blind spot can be minimized, and the amount of change of the previous image into the acquired stereoscopic image. It is an advantage that the accuracy is much higher than the method of finding the corresponding position by calculating. In this case, data such as a pan angle, a tilt angle, a zoom scale, a focus length, and the like may be obtained from the PTZ camera 100. In general, in the case of the PTZ camera 100, the captured image is stored. Therefore, the camera parameters (i.e., the pan angle, the tilt angle, the zoom magnification, and the focal length, etc.) of the frame may be stored together with the frames coming into the storage space at regular intervals.

The image acquisition unit 220 acquires each 2D image captured by the plurality of PTZ cameras 100 through the interface unit 210. The obtained image may be converted into a compressed format in which the image is compressed to facilitate data transmission. Image data in the form of a compressed format may have various formats such as Moving Picture Experts Group (MPEG) -1 or MPEG-4. Each 2D image acquired by the image acquisition unit 220 is transmitted to the image conversion unit 230.

The image converter 230 synthesizes at least two images from each of the 2D images obtained by the image acquirer 220, and converts the two images into 3D images. For example, when synthesizing two images, plane coordinates of a subject in a 2D image by each camera may be acquired, and distances between the respective cameras may be obtained. By using this, 2D image information exchanged through IP may be changed to stereoscopic image.

In particular, the PTZ camera 100 for surveillance, which is fixed at a specific place and is capable of operating up / down / left / right (Pan / Tilt), is more effective because the above-described camera parameters can be known in real time. In addition, the PTZ camera 100 acquires an image at a fixed position, and thus there is an advantage that the input of the image (screen switching, etc.) does not come in irregularly. Pan angle by three-dimensional movement of the PTZ camera 100, such as three-dimensional movement such as Zoom-In / Zoom-Out, Pan, Tilt, Camera parameters such as tilt angle, zoom scale, and focus length are acquired. In addition, a 3D image obtained by synthesizing two 2D images may be obtained using a pan / tilt angle, a zoom magnification, and a focal length of the camera.

The information processor 240 extracts position information of the object from the 2D image, and extracts height information of the object from the 3D image. Here, the position information of the object may be extracted based on the frame and camera parameters of the 2D image obtained from each PTZ camera 100. The height information of the object may be calculated from a 3D image obtained by synthesizing at least two 2D images. The location information of the object and the height information of the object are stored in the database 300. In this way, the difference between the previous screen and the current screen is calculated in a 2D state (still image) to find a moving object. This prevents the object from being missed when it is hidden behind another object or crosses with another object. Can be. At this time, before storing the position information and the height information of the object in the external database 300, put it in a temporary buffer (not shown) of the monitoring device 200 using the 3D image as information for tracking the position of the object. You can also use it right away.

In addition, the information processor 240 may extract distance information between the object in the image and the PTZ camera 100. The reason for extracting the distance information is that the zoom function and the focal position can be determined due to the characteristics of the PTZ camera 100. That is, distance information between the object and the PTZ camera 100 is calculated based on the zoom scale and the focus length. In such a case, the distance information of the tracking object or the imaging body can be displayed on the screen or processed without the use of a three-dimensional image camera, and the object can be tracked without missing the end.

The location tracking unit 250 may track the location of the object based on the location information and the height information of the object. By acquiring the stereoscopic image using at least two linked PTZ cameras 100, the height information of the object can be acquired together in the method of calculating the amount of change of the image, thereby enabling more accurate object tracking. At this time, the blind spots can be minimized by installing the linked PTZ camera 100 in the state of looking at each other, and the accuracy can be much higher than the method of finding the corresponding position by calculating the amount of change of the previous image with the acquired stereoscopic image. have. That is, since the object can be identified by adding the height information of the object calculated from the stereoscopic image acquired by using the plurality of image input devices, the performance of the automatic tracking device can be maximized.

Surveillance device 200 using the above-described 3D image is physically separated from the PTZ camera 100 is configured as a separate module. Of course, it will be apparent to those skilled in the art that each component of the monitoring apparatus 200 using the 3D image may be configured inside the PTZ camera 100. Hereinafter, an embodiment in which a monitoring process using a 3D image is executed inside a camera will be described.

3 is a block diagram of a surveillance system using a 3D image according to an embodiment of the present invention.

Referring to FIG. 3, the surveillance system 50 using the 3D image tracks an object by interlocking a plurality of PTZ cameras 500 to obtain a 3D image. Here, each PTZ camera 500 transmits and receives information and data to and from another PTZ camera 500 through the network 55. As described above, the network 55 may use IP (Internet Protocol). That is, the PTZ camera 500 may be an IP network camera.

Each PTZ camera 500 includes an interface unit 510, an image acquisition unit 520, and an image conversion unit 530. The information processing unit 540 may further include a location tracking unit 550 and a storage unit 560.

The interface unit 510 connects each PTZ camera 500 with another PTZ camera 500 through the network 55. In this case, a separate dedicated communication module or method is not required, and the design is limited to the interworking of the system due to the conventional design of the prior art, so that the design of the flexible interface is difficult and the scalability may be overcome. Same as one.

The image acquirer 520 captures an object to obtain a 2D image. An image is acquired by capturing a predetermined region. Here, a charge coupled device (CCD) module or a complementary metal oxide semiconductor (CMOS) module may be used as the imaging device. In particular, the camera module constituting the image acquisition unit 520 may be a camera module that can be photographed at night so as to detect an accurate situation at night. That is, it may be a camera module including an infrared sensor module for detecting infrared rays. Preferably, the infrared sensor module is a near infrared sensor module that detects near infrared rays, but is not limited thereto, and may be a far infrared sensor module that detects far infrared rays. The obtained image may be converted into a compressed format in which the image is compressed to facilitate data transmission. Image data in the form of a compressed format may have various formats such as Moving Picture Experts Group (MPEG) -1 or MPEG-4. The image photographed by the image acquisition unit 520 is transmitted to the image conversion unit 530.

The image converter 530 synthesizes the 2D image obtained by the image acquisition unit 520 with the 2D image of another PTZ camera 500 transmitted through the network 55 and converts the 2D image into a 3D image. The 2D images acquired by the plurality of network PTZ cameras 500 may be processed into a stereoscopic image by the image converting unit 530 of the arbitrary camera 500, and the distance and height information of the subject may be transmitted back to the counterpart camera. Through this, it is possible to achieve higher accuracy by using a stereoscopic image without tracking the position of a moving object only with a change amount of the 2D screen.

The information processor 540 extracts location information of the object from the 2D image, extracts height information of the object from the 3D image, and extracts distance information based on the focus and zoom magnification of the PTZ camera 500. Since the specific function of the information processing unit 540 is the same as the information processing unit 240 of the monitoring apparatus 200 using the above-described 3D image, it will be omitted.

The location tracking unit 550 tracks the location of the object based on at least two pieces of information of the location information, the height information, and the distance information of the object extracted by the information processing unit 540. For example, when several PTZ cameras 500 are installed in the same space regardless of height, and each camera 500 exchanges information obtained with the other camera 500 and simultaneously tracks the same subject, By using at least two pieces of position information, height information, and distance information, the position of an object as a subject may be accurately calculated in a 3D coordinate system.

The storage unit 560 stores position information, height information, distance information, and the like of the object extracted by the information processing unit 540. Here, it will be apparent to those skilled in the art that the storage unit 560 may be a component included in the PTZ camera 500 or may be replaced by an external storage medium. When the storage unit 560 is replaced by an external storage medium, the storage unit 560 may be a buffer for temporarily storing various information acquired and processed by the PTZ camera 500.

In the surveillance system 50 using the 3D image, since the plurality of PTZ cameras 500 interoperate with each other through the network 55, the 3D image and the information processor converted by the image converter 530 through the network 55 ( Position information, height information, and distance information processed by 540 may be transmitted to another PTZ camera 500. In detail, the distance between the subject and the camera may be calculated by converting the focal length zoom ratio of the camera, and the information may be exchanged with the other camera in real time to track the subject, and the 2D acquired by the plurality of network PTZ cameras 500 may be used. By processing the image into a stereoscopic image from a specific camera, the distance and height information of the subject can be transmitted back to the other camera to track the subject, and multiple PTZ cameras 500 are installed in the same space regardless of height. The same subject can be tracked while exchanging the acquired information with the other camera. Therefore, it does not use a separate three-dimensional video camera to reduce the cost, it is easy to configure using a universally installed PTZ camera 500, connected to a network PTZ camera 500 having an IP interface 510 And it is easy to operate, and has a higher accuracy by using a stereoscopic image without tracking the position only with the change amount of the 2D screen.

4 is a flowchart illustrating a monitoring method using 3D images according to an embodiment of the present invention.

Referring to FIG. 4, in the surveillance method using a 3D image according to an embodiment of the present invention, a 2D image is obtained by photographing with a plurality of PTZ cameras (S10), and a 2D image between a plurality of PTZ cameras connected through a network. (S20), synthesize the transmitted 2D image with the photographed 2D image and convert it into a 3D image (S30), extract the position information of the object from the 2D image, and extract the height information of the object from the 3D image In operation S50, the position of the object is tracked based on the position information and the height information.

Through this, after the plurality of network PTZ cameras are installed, the interlocking operation of the cameras is performed through the network, and the 2D image information exchanged through the network is changed back to 3D image. By real-time sharing the three-dimensional images acquired through the network with the other party's camera and monitoring the moving subject by omnidirectional monitoring, it is possible to save money by using fixed PTZ camera without installing a separate moving expensive 3D camera. There are advantages to it. In addition, when using evidence of crime identification, distance information and overlapping objects that can be missed on existing 2D screens can be identified and separated into a plurality of objects.In a PTZ camera with an automatic tracking device, the success rate is seamless without this method. High object tracking is expected.

On the other hand, the surveillance method using the 3D image according to an embodiment of the present invention can be implemented as a module by software and hardware, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, It can be implemented in a general-purpose computer for operating the program using a recording medium that can be read. The computer-readable recording medium is implemented in the form of a carrier wave such as a ROM, a floppy disk, a magnetic medium such as a hard disk, an optical medium such as a CD or a DVD, and a transmission through the Internet. In addition, the computer-readable recording medium may be distributed to a network-connected computer system so that computer-readable codes may be stored and executed in a distributed manner.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

100: PTZ camera
200: surveillance device using 3D video
210: interface unit 220: image acquiring unit
230: image converter 240: information processor
250: location tracking unit
300: database
50: Surveillance system using 3D video
55: network
500: PTZ camera
510: interface unit 520: image acquisition unit
530: image converter 540: information processor
550: location tracking unit 560: storage unit

Claims (13)

An interface unit for interworking a plurality of PTZ cameras;
An image obtaining unit obtaining each 2D image picked up by the plurality of PTZ cameras through the interface unit; And
And an image converter configured to synthesize at least two images from each of the 2D images obtained by the image acquisition unit and convert the two images into 3D images. The method of claim 1,
The plurality of PTZ camera is fixed to a specific position, the monitoring device using a 3D image. The method of claim 1,
The interface unit, the monitoring device using the 3D video, which interlocks the plurality of PTZ cameras using the Internet Protocol (IP). The method of claim 1,
And an information processor for extracting position information of the object from the 2D image and extracting height information of the object from the 3D image. 5. The method of claim 4,
And the information processor extracts distance information between an object and a PTZ camera. 6. The method of claim 5,
And the information processor extracts the distance information based on the focus and zoom magnification of the PTZ camera. 5. The method of claim 4,
And a location tracking unit for tracking the location of the object based on the location information and the height information of the object. In the 3D image acquisition system for tracking the object by obtaining a 3D image by interlocking a plurality of PTZ cameras,
Each PTZ camera,
An interface unit connected to another PTZ camera through a network;
An image acquisition unit for capturing an object to obtain a 2D image; And
And a video converter configured to convert the 2D video obtained by the video capturing unit to 2D video of another PTZ camera transmitted through the network and convert the 2D video into a 3D video. The method of claim 8,
The apparatus may further include an information processor configured to extract position information of the object from the 2D image, extract height information of the object from the 3D image, and extract distance information based on the focus and zoom magnification of the PTZ camera. Surveillance system using video. The method of claim 9,
And a location tracking unit to track the location of the object based on at least two pieces of information of the location information, the height information, and the distance information of the object. The method of claim 9,
And a storage unit for storing location information, height information, and distance information of the object. The method of claim 9,
And a 3D image converted by the image conversion unit and the position information, the height information, and the distance information processed by the information processing unit to another PTZ camera via the network. Photographing with a plurality of PTZ cameras to obtain a 2D image;
Transmitting the 2D image between a plurality of PTZ cameras connected through a network;
Converting the transmitted 2D image into a 3D image by synthesizing the photographed 2D image;
Extracting location information of the object from the 2D image and extracting height information of the object from the 3D image; And
And tracking the position of the object based on the position information and the height information.

Images