KR20160090650A - Apparatus and method for estimation of user location over a configurable sensor network - Google Patents

Abstract

An apparatus for estimating user location according to an embodiment of the present invention may include the steps of: acquiring a first image including depth information by photographing a photographing target area by a first camera; acquiring a second image by magnifying and photographing a specific area in the photographing target area by a second camera; detecting a body of a user from the first image, and calculating a three-dimensional location of the body; detecting a face of the user from the second image by controlling a pose of the second camera on the basis of the three-dimensional location of the body, and calculating a three-dimensional location of the face; and estimating the three-dimensional location of the user on the basis of the three-dimensional location of the body and the three-dimensional location of the face. The apparatus of the present invention can calibrate a sensor network automatically.

Description

[0001] APPARATUS AND METHOD FOR ESTIMATION OF USER LOCATION OVER A CONFIGURABLE SENSOR NETWORK [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a user recognition field, and more particularly, to an apparatus and method for estimating a user's position using a plurality of images captured in a sensor network composed of different types of cameras.

As sensor technology and image processing technology are developed, techniques for recognizing a specific target and estimating a position from an image captured by a sensor such as a camera are used in many fields. For example, such a target recognition technique can be utilized to provide a user information based robot service. In addition, such a target recognition technology can be utilized in a video security management system such as a crime prevention and risk situation prevention system.

Devices that recognize these particular targets must be able to effectively recognize targets in various environments or at various distances. However, in the conventional target recognition apparatuses, a sensor network is constructed using only a single sensor or a sensor of the same type, and a method of recognizing a target using the sensor network is adopted. In this case, in a case where a target recognition area is narrow, The recognition rate of the target is lowered.

Patent Publication No. 10-2009-0022099

Accordingly, it is an object of the present invention to provide a user location estimation apparatus and method for recognizing a user and estimating a location of a user using a sensor network composed of different kinds of sensors.

It is also an object of the present invention to provide a user location estimation apparatus and method for automatically correcting a sensor network.

According to an aspect of the present invention, there is provided a method of capturing an image, comprising: capturing a region to be photographed by a first camera to obtain a first image including depth information; Acquiring a second image by enlarging an image of a specific area within the area to be imaged by a second camera; Detecting a user's body from the first image, calculating a three-dimensional position of the body based on the first pose parameter of the first camera and the depth information of the body; Wherein the face detection unit detects the face of the user from the second image by controlling the pose of the second camera based on the three-dimensional position of the body, and based on the first pose parameter of the second camera and the depth information of the face, Calculating a three-dimensional position of the face, the depth information of the face being estimated from depth information of the body; And estimating the three-dimensional position of the user based on the three-dimensional position of the body and the three-dimensional position of the face.

According to an aspect of the present invention, the first camera may be an RGB-D camera, and the second camera may be a pan-tilt-zoom (PTZ) camera.

According to an aspect of the present invention, the first pose parameter includes a parameter indicating a tilting angle of the RGB-D camera, and the second pose parameter includes a panning angle of the PTZ camera, tilting angle and zoom ratio.

According to an aspect of the present invention, the step of correcting the sensor pose parameters of the sensor network constituted of the RGB-D camera and the PTZ camera using the RGB-D camera may be performed before the step of acquiring the first image Wherein the sensor pose parameter comprises a parameter indicative of a height of the sensor network.

According to an aspect of the present invention, the step of calculating the three-dimensional position of the face may estimate the depth information of the face using the depth information of the body and the second pose parameter of the PTZ camera.

According to an aspect of the present invention, there is provided an image processing apparatus comprising: a first camera for photographing a region to be photographed to acquire a first image including depth information; A second camera for acquiring a second image by zooming in on a specific area within the shooting target area; And a controller for controlling the first camera and the second camera, wherein the controller detects a user's body from the first image, and detects a first pose parameter of the first camera and depth information of the body A first position estimation module for estimating a three-dimensional position of the body based on the three-dimensional position of the body; Wherein the face of the user is detected from the second image by controlling the pose of the second camera based on the three-dimensional position of the body, and based on the second pose parameter of the second camera and the depth information of the face, A second position estimation module for estimating a three-dimensional position of a face, the depth information of the face being estimated from depth information of the body; And a user position determination module for estimating the three-dimensional position of the user based on the three-dimensional position of the body and the three-dimensional position of the face.

According to an aspect of the present invention, the first camera may be an RGB-D camera, and the second camera may be a PTZ camera.

According to an aspect of the present invention, the first pose parameter includes a parameter indicating a tilting angle of the RGB-D camera, and the second pose parameter includes a panning angle of the PTZ camera, tilting angle and a parameter indicating a zooming ratio.

According to an aspect of the present invention, the control unit further includes a sensor pose parameter correcting unit for correcting sensor pose parameters of the sensor network constituted of the RGB-D camera and the PTZ camera using the RGB-D camera, The sensor pose parameter may include a parameter indicative of the height of the sensor network.

According to an aspect of the present invention, the second position calculation unit may estimate depth information of the face using the depth information of the body and the second pose parameter of the PTZ camera.

According to the present invention, the user location estimation apparatus and method recognize a user by using a sensor network composed of different types of sensors, thereby expanding the area of user recognition as compared to using a single sensor, The user can be recognized and the user recognition rate can be increased.

In addition, the user location estimation apparatus and method can automatically acquire system parameters of a sensor network regardless of the installed environment by automatically correcting the sensor network, thereby providing a user recognition service in a relatively wide indoor environment.

1 is a schematic diagram of a user location estimation apparatus according to an embodiment of the present invention.
2 is a block diagram of a user location estimation apparatus according to an embodiment of the present invention.
3 is a graph illustrating a change in focal length of a camera conversion matrix of a PTZ camera according to a zoom ratio of the PTZ camera according to an embodiment of the present invention.
4 illustrates a technique for estimating face depth information based on depth information of a user's body and pose parameters of a PTZ camera, according to an embodiment of the present invention.
FIG. 5 illustrates a method by which a user location estimation apparatus according to an embodiment of the present invention corrects a sensor pose parameter of a sensor network.
FIG. 6 is a table showing error rates of three-dimensional position estimation of the user's body and face by changing the dynamic parameters of the sensor network in the user location estimation apparatus according to an embodiment of the present invention.
FIG. 7 is a view showing three-dimensional position coordinates of a body, three-dimensional position coordinates of a face, and errors according to a user's movement on a sensor network according to an embodiment of the present invention.
8 is a flowchart of a method of estimating a user location according to an embodiment of the present invention.
9 is a flowchart of a method of estimating a user location according to another embodiment of the present invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings and the accompanying drawings, but the scope of the claims is not limited or limited by the embodiments.

As used herein, terms used in the present specification are selected from the general terms that are currently widely used, while taking into consideration the functions, but these may vary depending on the intention or custom of the artisan or the emergence of new techniques. Also, in certain cases, there may be a term selected by the applicant at will, in which case the meaning will be described in the description part of the corresponding specification. Therefore, it is intended that the terminology used herein should be interpreted based on the meaning of the term rather than on the name of the term, and on the entire contents of the specification.

In addition, the embodiments described herein may be wholly hardware, partially hardware, partially software, or entirely software. A "unit", "module", "device", "robot" or "system" or the like in this specification refers to a computer-related entity such as hardware, a combination of hardware and software, . For example, a component, a module, a device, a robot or a system may refer to software such as an application for driving the hardware and / or the hardware constituting part or all of the platform.

1 is a schematic diagram of a user location estimation apparatus according to an embodiment of the present invention. Referring to FIG. 1, the user location estimation apparatus 100 may include a first camera 110, a second camera 120, and a controller 130.

In this specification, the user location estimation apparatus 100 detects a user 10 using a first image and a second image respectively photographed by different types of cameras in a sensor network in a sensor network composed of different kinds of cameras, And estimates the position of the user 10. For example, the user location estimation apparatus 100 may be configured such that in a sensor network composed of a first camera 110 (e.g., an RGB-D camera) and a second camera 120 (e.g., a PTZ camera) (E.g., an RGB-D image) photographed by the first camera 120 and a second image (e.g., a PTZ image) photographed by the second camera 120, Dimensional position of the user 10 based on the detected body and the three-dimensional position of the face.

As shown in FIG. 1, the first camera 110 and the second camera 120 of the user location estimation apparatus 100 may be mounted on the same tripod to form one sensor network. In this case, the relative position between the first camera 110 and the second camera 120 can be fixed. Since the user location estimation apparatus 100 uses a sensor network composed of different types of cameras, it is necessary to precisely control dynamic and controllable pose parameters of each camera or sensor network in order to implement a user recognition system.

In the present specification, the pose parameter may include a first pose parameter of the first camera, a second pose parameter of the second camera, and a sensor pose parameter of the sensor network.

)일 수 있다. 또한, 제2 포즈 파라미터는 제2 카메라의 자세를 나타내는 파라미터로서, 예컨대, 제2 카메라의 패닝 각도, 틸팅 각도 및 줌 비율를 나타내는 파라미터들(

)일 수 있다. 제1 포즈 파라미터 및 제2 포즈 파라미터는 사용자 위치 추정 장치에 의한 각 카메라에 대한 모터제어에 대한 피드백을 통해 정확한 제1 및 제2 포즈 파라미터 값을 획득할 수 있다. Here, the first pose parameter is a parameter indicative of the attitude of the first camera, for example, a parameter indicating the tilting angle (e.g., 0 to 90 degrees) of the first camera

). The second pose parameter is a parameter indicating the pose of the second camera, for example, parameters indicating the panning angle, tilting angle and zoom ratio of the second camera

). The first pose parameter and the second pose parameter may obtain correct first and second pose parameter values through feedback to the motor control for each camera by the user position estimation device.

)일 수 있다. 일 실시예에서, 센서 포즈 파라미터는 미리 보정된(calibarated) 파라미터일 수 있다. 다른 실시예에서, 센서 포즈 파라미터는 보정되지 않은 파라미터일 수 있다. 이 경우, 사용자 위치 추정 장치(100)는 미리 설정된 방법에 따라 센서 포즈 파라미터를 보정할 수 있다. 이에 대하여는, 도 5를 참조하여 이하에서 상세히 설명하도록 한다. Here, the sensor pose parameter is a parameter indicating the posture of the sensor network, for example, a parameter indicating the height of the sensor network

). In one embodiment, the sensor pose parameter may be a calibrated parameter. In another embodiment, the sensor pose parameter may be an uncorrected parameter. In this case, the user location estimation apparatus 100 can correct the sensor pose parameter according to a predetermined method. This will be described in detail below with reference to Fig.

The first camera 110 may capture the first region of the three-dimensional image by photographing the region to be photographed. Here, the photographing target area may be an area including at least one object (for example, the user 10), which can be photographed by the camera. In one embodiment, the first camera 110 may operate in various poses depending on the first pose parameter of the first camera. The user position estimation apparatus 100 can adjust the pose of the first camera by adjusting the first pose parameter, thereby obtaining the first image photographed at various angles.

In one embodiment, the first camera 110 includes an RGB image composed of red (R), green (G), and blue (B) pixels, depth information of an object photographed by the first camera 110 And a distance from the first camera 110 to the object). However, the first camera 110 may be an arbitrary camera including a configuration for obtaining a color image such as a charge coupled device and a configuration for obtaining a depth image such as a laser sensor.

The second camera 120 can acquire a second image in a two-dimensional manner by zooming-photographing a specific region within the region to be photographed. In one embodiment, the second camera 120 may be a PTZ camera capable of panning, tilting, and zooming. However, this is an example, and the second camera 120 can be any camera capable of rotating and zooming in or out of the object to be photographed through zooming in or zooming out.

In one embodiment, the second camera 120 may operate in various poses depending on the second pose parameter of the second camera. The user location estimation apparatus 100 can adjust the pose of the second camera by adjusting the second pose parameter, thereby obtaining the second image photographed at various angles and various zoom ratios.

The control unit 140 may control the first camera 110 and the second camera 120. In one embodiment, the controller 140 may control the pose of the first camera 110 and the second camera 120. [ In this case, the control unit 140 can control the pose of each camera by driving a rotating motor, a zoom motor, and the like connected to each camera. In addition, the controller 140 can estimate the three-dimensional position of the user based on the first image captured by the first camera 110 and the second image captured by the second camera 120. [ The control unit 140 will be described in detail below with reference to FIG.

Hereinafter, embodiments will be described focusing on the case where the first camera is an RGB-D camera and the second camera is a PTZ camera. However, these embodiments are equally applicable to the first and second cameras of the other types described above.

2 is a block diagram of a user location estimation apparatus according to an embodiment of the present invention. Referring to FIG. 2, the user location estimation apparatus 100 may include a first camera 110, a second camera 120, and a controller 130. The control unit 110 may include a first position calculation module 131, a second position calculation module 132, and a user position determination module 133. In addition, the control unit 130 may further include a sensor pose parameter correcting unit 134 as an optional configuration. Since the first camera 110 and the second camera 120 have been described in detail with reference to FIG. 1, a detailed description thereof will be omitted.

The first position calculation module 131 may detect the user's body from the first image of the RGB-D camera. Here, the body refers to the entire body of the user or a part of the body (for example, the body excluding the face, arms, and legs) that can be identified as a person.

In one embodiment, the first position calculation module 131 detects an object in the first image based on the RGB image and the depth information of the first image, and determines whether the detected object is the body of the user, The body of the user can be detected from the image. For example, the first position calculation module 131 detects, based on the RGB image and the depth information of the first image, a portion of the first image that is larger than a predetermined size, as an object, It is possible to determine whether the detected object is the user's body by comparing the shape of the user with the shape of the user's body stored in advance. However, this is an exemplary one, and the first position calculation module 131 may detect an object in the first image using various methods and determine whether the detected object is the body of the user.

In addition, the first position calculation module 131 may calculate the three-dimensional position of the body based on the pose parameter of the first camera and the depth information of the body. A method of calculating the three-dimensional position of the body by the first position calculation module 131 will be described in detail with reference to the following equations.

The second position calculation module 132 can detect the user's face from the second image by controlling the pose of the PTZ camera based on the three-dimensional position of the body. In one embodiment, the second position calculation module 132 adjusts the pose parameters of the PTZ camera (e.g., the parameters representing the zoom ratio, panning angle, and tilting angle of the PTZ camera) based on the three- (For example, zooming in or focusing) the face region of the face image of the user to obtain the second image, and detect the face of the user from the second image. For example, the second position calculation module 132 may detect a face of a user by comparing an image of a two-dimensional face region of a second image with an image of a user's face stored in advance. However, this is exemplary and the second position calculation module 132 may detect the user's face in the second image using various methods.

In addition, the second position calculation module 132 may calculate the three-dimensional position of the body based on the pose parameter of the second camera and the depth information of the face. Here, the depth information of the face can be estimated based on the depth information of the body, as will be described in detail below with reference to Fig.

Hereinafter, with reference to the equations, a description will be made of a method in which the first position calculating module 131 and the second position calculating module 132 calculate the three-dimensional position of the user's body and face, respectively. First, a relationship between a two-dimensional coordinate (x) of a pixel in a camera environment and a corresponding three-dimensional coordinate (X) in a real environment can be expressed by the following equation (1).

Here, K denotes a camera conversion matrix, R denotes a rotation matrix, and t denotes a transitive vector. Each camera has its own camera conversion matrix, and the RGB-D camera and the PTZ camera have unique camera conversion matrices as shown in Equation (2) below.

를 3차원 지역좌표로 변환하는 식은 아래의 수학식 3과 같고, 사용자의 얼굴의 중심을 나타내는 픽셀의 2차원 좌표

를 3차원 지역좌표로 변환하는 식은 아래의 수학식 4와 같다.In this case, f 'denotes the focal length of the RGB-D camera, and f denotes the focal length of the PTZ camera. The camera conversion matrix according to the zoom ratio of the PTZ camera can be estimated in a two-dimensional manner through an experimental method. FIG. 3 shows that the focal distance of the camera conversion matrix of the PTZ camera changes according to the zoom ratio of the PTZ camera FIG. Through equations (1) and (2), two-dimensional coordinates of the pixel representing the center of the user's body

Into the three-dimensional local coordinates is expressed by Equation (3) below, and the two-dimensional coordinates of the pixel representing the center of the user's face

To the three-dimensional local coordinates is expressed by Equation (4) below.

는 몸체의 중심을 나타내는 픽셀의 깊이 정보를 나타내고,

는 얼굴의 중심을 나타내는 픽셀의 깊이 정보를 나타낸다. 여기에, RGB-D 카메라의 제1 포즈 파라미터를 적용하여 사용자의 몸체의 3차원 지역좌표로부터 3차원 글로벌좌표를 획득하는 식은 아래의 수학식 5와 같고, PTZ 카메라의 제2 포즈 파라미터를 적용하여 사용자의 얼굴의 3차원 지역좌표로부터 3차원 글로벌좌표를 획득하는 식은 아래의 수학식 6와 같다. here,

Represents the depth information of the pixel representing the center of the body,

Represents the depth information of the pixel representing the center of the face. Here, the equation for obtaining the three-dimensional global coordinates from the three-dimensional area coordinates of the user's body by applying the first pose parameter of the RGB-D camera is expressed by Equation (5) below and the second pose parameter of the PTZ camera is applied The equation for obtaining the three-dimensional global coordinates from the three-dimensional local coordinates of the user's face is shown in Equation (6) below.

는 상기 제1 파라미터로서, RGB-D 카메라의 틸팅 각도를 나타내는 파라미터이고,

는 상기 제2 파리미터로서, 각각 PTZ 카메라의 줌 비율, 패닝 각도, 틸팅 각도를 나타내는 파라미터들이다. 또한,

는 상기 센서 파리미터로서, RGB-D 카메라 및 PTZ 카메라로 구성된 센서 네트워크의 높이를 나타내는 파라미터이다. here,

Is a parameter indicating the tilting angle of the RGB-D camera as the first parameter,

Are parameters indicating the zoom ratio, panning angle, and tilting angle of the PTZ camera, respectively. Also,

Is a parameter indicating the height of the sensor network constituted by the RGB-D camera and the PTZ camera.

값은 제1 영상으로부터 직접 추정될 수 있으나, PTZ 카메라에 의해 촬영된 제2 영상은 각 픽셀의 깊이 정보를 포함하고 있지 않으므로

값을 제2 영상으로부터 직접적으로 정확히 추정하기는 어렵다. 이에, 도 4에서처럼, 제2 위치 산출 모듈(132)은 다음 식 5를 이용하여

값을 추정하고, 이를

값으로 이용할 수 있다. The first position calculation module 131 and the second position calculation module 132 can calculate the three-dimensional position of the body and the three-dimensional position of the face using the above-described equations, respectively. However, since the first image captured by the RGB-D camera includes depth information of each pixel

Value can be directly estimated from the first image, but the second image captured by the PTZ camera does not include the depth information of each pixel

It is difficult to accurately estimate the value directly from the second image. 4, the second position calculation module 132 uses the following Equation 5

Quot;

Value.

The user position determination module 133 can estimate the three-dimensional position of the user based on the three-dimensional position of the body and the three-dimensional position of the face. In one embodiment, the user position determination module 133 may estimate the user's three-dimensional position using the difference between the three-dimensional position of the body and the three-dimensional position of the face. Accordingly, the user location estimation apparatus can accurately estimate the three-dimensional position of the user, so that it can be utilized as high-reliability user information.

The sensor pose parameter correcting unit 134 can correct the sensor pose parameters of the sensor network composed of the RGB-D camera and the PTZ camera using the RGB-D camera. This will be described in detail below with reference to Fig.

FIG. 2 is a block diagram according to an embodiment of the present invention, wherein blocks separated and shown are logically distinguishing components of the apparatus. Therefore, the components of the above-described apparatus can be mounted as one chip or as a plurality of chips according to the design of the apparatus. Hereinafter, the control unit 140 may be described as controlling at least one component included in the user location estimation apparatus 100 or the user location estimation apparatus 100, and the control unit 140 and the user location estimation apparatus 100 may be referred to as " Can be explained.

FIG. 5 illustrates a method by which a user location estimation apparatus according to an embodiment of the present invention corrects a sensor pose parameter of a sensor network.

If the sensor pose parameter of the sensor network is not previously calibrated, as shown on the left side of FIG. 5, the user position estimation device controls the first pose parameter of the RGB-D camera so that the RGB- The tilting angle is 90 degrees). And, as shown in the middle of FIG. 5, the user position estimation device can extract the ground based on the plane model. 5, the user position estimating apparatus can correct the sensor pose parameters by estimating the average height of the ground and the RGD-D camera for a preset time. In one embodiment, the user location estimation apparatus can perform the above-described functions through the control unit.

 FIG. 6 is a table showing the error rates of three-dimensional position estimation of the user's body and face as the dynamic parameters of the sensor network are changed according to an embodiment of the present invention. Here, the dynamic parameter is a concept including the above-described first pose parameter, second pose parameter, and sensor pose parameter.

와 센서 네트워크의 높이를 나타내는 파라미터인

를 센서 네트워크 상에서 동적으로 변화시킨 경우, 고정된 사용자의 몸체의 3차원 위치와 얼굴의 3차원 위치를 실험적으로 구한 값이다. 이렇게 추정된 사용자의 위치 오차의 평균값은 몸체의 경우 0.19미터이고, 얼굴의 경우 0.25미터를 갖는다.
More specifically, FIG. 6 shows a parameter indicating the tilting angle of the RGBD camera among five pose parameters that can be controlled

And a parameter representing the height of the sensor network

Dimensional position of the body of the fixed user and the three-dimensional position of the face are experimentally obtained when the robot is dynamically changed on the sensor network. The average value of the estimated position error of the user is 0.19 m for the body and 0.25 m for the face.

FIG. 7 is a view showing three-dimensional position coordinates of a body, three-dimensional position coordinates of a face, and errors according to a user's movement on a sensor network according to an embodiment of the present invention. In FIG. 7, the blue "x" mark represents the three-dimensional position of the face, the red "*" represents the three-dimensional position of the body, the line connecting the two represents the three- Lt; / RTI > Referring to Fig. 7, the error of the face position is about 0.158 meters.

8 is a flowchart of a method of estimating a user location according to an embodiment of the present invention. In FIG. 8, a detailed description of the same or similar parts as those described in FIGS. 1 to 7 will be omitted.

Referring to FIG. 8, the user location estimation apparatus may acquire a first image in three dimensions by photographing a region to be photographed through a first camera (S10). Here, the photographing target area may be an area including at least one object (e.g., a user), which is an area that can be photographed by the camera. In one embodiment, the first camera may operate in various poses depending on the first pose parameter of the first camera. In step S10, the user position estimation device adjusts the first pose parameter to adjust the pose of the first camera, thereby obtaining the first image photographed at various angles.

The user location estimation apparatus can acquire a second image in a two-dimensional manner by enlarging a specific region within the region to be photographed through the second camera. In one embodiment, the second camera may operate in various poses depending on the second pose parameter of the second camera. In step S20, the user position estimation device adjusts the second pose parameter to adjust the pose of the second camera, thereby obtaining a second image photographed at various angles and various zoom ratios.

The user position estimation apparatus can detect the user's body from the first image and calculate the three-dimensional position of the body (S30). In step S30, the user position estimation apparatus detects the user's body from the first image of the first camera, calculates the three-dimensional position of the body based on the first pose parameter of the first camera and the depth information of the body can do.

The user location estimation apparatus can detect the user's face from the second image and calculate the three-dimensional position of the face (S40). In step S40, the user position estimation apparatus detects the user's face from the second image by controlling the pose of the second camera based on the three-dimensional position of the body, and detects the first pose parameter of the second camera and the face The three-dimensional position of the face can be calculated based on the depth information. In this case, the user location estimation apparatus can estimate the depth information of the face from the depth information of the body.

The user position estimation apparatus can estimate the three-dimensional position of the user based on the three-dimensional position of the body and the three-dimensional position of the face (S50).

9 is a flowchart of a method of estimating a user location according to another embodiment of the present invention. In FIG. 9, detailed description of the same or similar parts as those described in FIGS. 1 to 8 will be omitted. The description of steps S10 to S50 in FIG. 9 is the same as the description of steps S10 to S50 in FIG. 8, so that the description will be omitted.

Referring to FIG. 9, before the step S10 of obtaining the first image, the user location estimation apparatus corrects the sensor pose parameters of the sensor network constituted by the RGB-D camera and the PTZ camera using the RGB-D camera (S1). Here, the sensor pose parameter may be a parameter indicating the height of the sensor network.

In contrast, if the sensor pose parameter of the sensor network is not corrected in advance, the user position estimating apparatus controls the first pose parameter of the RGB-D camera, so that the RGB-D camera is perpendicular to the ground The tilting angle is 90 degrees). Then, the user location estimation apparatus can extract the ground based on the plane model. Then, the user location estimation apparatus can correct the sensor pose parameter by estimating the average height of the ground and the RGD-D camera for a predetermined time.

Such a user location estimation method may be implemented in an application or may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions that are recorded on a computer-readable recording medium may be those that are specially designed and constructed for the present invention and are known and available to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. A hardware device may be configured to operate as one or more software modules to perform processing in accordance with the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

In this specification, both the invention and the method invention are explained, and the description of both inventions can be supplemented as necessary.

100: user position estimation device 110: first camera
120: second camera 130:

Claims (10)

Capturing an area to be photographed by a first camera to obtain a first image including depth information;
Acquiring a second image by enlarging an image of a specific area within the area to be imaged by a second camera;
Detecting a user's body from the first image, calculating a three-dimensional position of the body based on the first pose parameter of the first camera and the depth information of the body;
Wherein the face detection unit detects the face of the user from the second image by controlling the pose of the second camera based on the three-dimensional position of the body, and based on the first pose parameter of the second camera and the depth information of the face, Calculating a three-dimensional position of the face, the depth information of the face being estimated from depth information of the body; And
Estimating the three-dimensional position of the user based on the three-dimensional position of the body and the three-dimensional position of the face. The method according to claim 1,
Wherein the first camera is an RGB-D camera and the second camera is a PTZ camera. 3. The method of claim 2,
Wherein the first pose parameter includes a parameter indicating a tilting angle of the RGB-D camera and the second pose parameter includes a panning angle, a tilting angle, and a zoom angle of the PTZ camera, And a parameter indicative of the ratio. 3. The method of claim 2,
Before the step of acquiring the first image,
Further comprising the step of correcting sensor pose parameters of a sensor network composed of the RGB-D camera and the PTZ camera using the RGB-D camera, wherein the sensor pose parameter includes a parameter indicating a height of the sensor network / RTI > 3. The method of claim 2,
Wherein the step of calculating the three-
And estimating the depth information of the face using the depth information of the body and the second pose parameter of the PTZ camera. A first camera for photographing a region to be photographed to acquire a first image including depth information;
A second camera for acquiring a second image by zooming in on a specific area within the shooting target area; And
And a controller for controlling the first camera and the second camera,
Wherein,
A first position estimation module that detects a user's body from the first image, estimates a three-dimensional position of the body based on the first pose parameter of the first camera and the depth information of the body;
Wherein the face of the user is detected from the second image by controlling the pose of the second camera based on the three-dimensional position of the body, and based on the second pose parameter of the second camera and the depth information of the face, A second position estimation module for estimating a three-dimensional position of a face, the depth information of the face being estimated from depth information of the body; And
And a user positioning module for estimating the three-dimensional position of the user based on the three-dimensional position of the body and the three-dimensional position of the face. The method according to claim 6,
Wherein the first camera is an RGB-D camera and the second camera is a PTZ camera. 8. The method of claim 7,
Wherein the first pose parameter includes a parameter indicating a tilting angle of the RGB-D camera and the second pose parameter includes a panning angle, a tilting angle, and a zooming of the PTZ camera, And a parameter indicative of the ratio. 8. The method of claim 7,
Wherein,
Further comprising a sensor pose parameter correcting unit for correcting sensor pose parameters of a sensor network constituted by the RGB-D camera and the PTZ camera using the RGB-D camera, wherein the sensor pose parameter is a parameter indicating a height of the sensor network And estimates the position of the user. 8. The method of claim 7,
The second position calculating unit calculates,
And estimating the depth information of the face using the depth information of the body and the second pose parameter of the PTZ camera.

Images