KR20070112597A - Method and apparatus of control of projecting an image by virtual camera - Google Patents

Abstract

A method and an apparatus for adjusting a projected image using a virtual camera are provided to allow a user to watch an image with less distortion excluding the front of the screen. An image of virtual three-dimensional space including a screen with an original image displayed is captured by a virtual camera(S01). The image captured by the virtual camera is three-dimensionally converted to create a view port(S02). A request for changing the image is received from a user(S03). The view port is changed by using view conversion and/ or projection conversion according to the image change request and the changed view port is provided(S04). An image is projected on a screen by using the changed view port(S05).

Description

Method and apparatus for adjusting projection image using virtual camera {Method and apparatus of control of projecting an image by virtual camera}

1 is a flowchart illustrating a process of adjusting a projection image according to the present invention.

2 is a block diagram of an apparatus for adjusting a projection image according to the present invention.

3 illustrates an original image to be projected on a screen.

4 illustrates a scene of projecting an image on a screen by a projector.

5 is a conceptual diagram of photographing a virtual three-dimensional space including a screen on which an original image is displayed with a virtual camera.

6A illustrates an example in which the size of an image is enlarged.

6B illustrates an example of reducing the size of an image.

6C illustrates an example of changing the position of an image.

6D shows an example in which the image is rotated on the X axis.

6E illustrates an example in which the image is rotated on the Y axis.

6F illustrates an example of rotating the image on the Z axis.

<Description of the symbols for the main parts of the drawings>

10: video projection unit

20: Original video support

30: virtual camera

40: image change unit

50: user interface unit

60: control unit

70: screen

80: original image

100: projector

The present invention relates to a method and apparatus for adjusting a projected image using a virtual camera. More particularly, the present invention relates to a virtual camera concept without moving a projector by hand or changing an internal hardware in projecting an image using a projector. The present invention relates to a method and apparatus for freely changing the size and position of a projected image by using the same, and rotating the projected image to suit a user's gaze.

In today's information society, highly complex information is produced in large quantities, and information transmission technology is rapidly developing. Accordingly, the necessity of a display capable of displaying a large amount of image information has increased, and various display studies for high resolution, large screen, and lightweight have been actively conducted.

In particular, the conventional TV, which has played an absolute role as an image information displayer, has a fixed position and orientation, so that the user has to limit the range of motion and the posture is not free to view the screen. The development of a projection display system capable of providing a large screen is possible, and the development and spread of projectors using liquid crystal display (LCD) or digital light processing (DLP) are rapidly increasing.

In addition, the development of technology for increasing the utilization of such a projection display system is being actively made. In particular, projectors that provide large screens with short throw distances using wireless, ultra-compact and ultra-lightweight projectors and mirrors are gaining popularity in the market. Meanwhile, various techniques for correcting keystone distortion of an image generated when a screen or a projector is not parallel to the ground have been developed, and are widely applied to commercially available projector products.

However, the conventional projector image projection method described above has the following problems.

First, the projector must be moved directly to adjust the size of the projected image and the position of the projected image in space. In other words, the user must find the empty space that can be projected and place the projector in front so that the user needs to move the projector to adjust the size and position of the projected image.

Second, when the projector is placed at a specific position, the position and direction of the projection image are fixed, thereby limiting the user's gaze and posture. Once you have positioned the wall or screen to project and set the projector position and orientation to match the surface, you must position yourself in front of the screen. Otherwise, as the user's position moves away from the front of the screen as shown on the side or below, the image is severely distorted.

The present invention is to solve such a conventional problem, an object of the present invention is to move the projector or change the internal hardware to freely adjust the size, position, and rotation of the projection image to suit the user's position. It is to eliminate the inconvenience.

Still another object of the present invention is to make it possible to view an image with less distortion even at a position outside the front of the screen, so as to escape the constraints of the width and shape of the projector installation location.

The present invention is to achieve the above object, the adjustment method of the projection image using a virtual camera according to an aspect of the present invention, in the adjustment method of the projection image, a virtual three-dimensional including a screen on which the original image is displayed Photographing the space with a virtual camera (a); (B) creating a viewport by three-dimensionally converting the image photographed by the virtual camera; (C) receiving an image change request from a user; (D) providing the modified viewport by changing the view transformation and / or projection transformation of the viewport according to the image change request; And (e) projecting an image on the screen by using the modified viewport.

Preferably, the image change request may be characterized in that it comprises a request for changing the size of the image, change the position, rotation.

More preferably, the size change of the image is performed by obtaining an enlargement or reduction transformation matrix having an enlargement or reduction value as an element and applying the same to the projection transformation matrix of the current projected image; Repositioning of the image is performed by obtaining a parallel shift transformation matrix having a parallel shift amount as an element and applying it to the projection transformation matrix of the current projected image; The rotation change of the image may be performed by obtaining a rotation transformation matrix having the rotation amount as an element and applying the transpose matrix of the rotation transformation matrix to the view transformation matrix of the current projected image.

According to another aspect of the present invention, there is provided an apparatus for adjusting a projection image using a virtual camera, the apparatus including: an image projection unit for projecting the image onto the screen; An original image support unit supporting the original image; The virtual camera photographing the virtual three-dimensional space; An image changer for changing the view transformation and / or projection transformation of the viewport according to the image change request to provide the changed viewport; A user interface unit for inputting a manipulation signal of the user; And a controller for controlling the image projector, the virtual camera, and the image changing unit according to the user's manipulation signal.

Hereinafter, with reference to the accompanying drawings will be described in detail with respect to preferred embodiments of the present invention.

1 is a flowchart illustrating a process of adjusting a projection image according to the present invention, and FIG. 3 illustrates an original image to be projected on a screen. 4 illustrates a scene of projecting an image on a screen by a projector, and FIG. 5 is a conceptual diagram of photographing a virtual three-dimensional space including a screen on which an original image is displayed.

In FIG. 1, the virtual camera photographing step S01 is a step of photographing a virtual three-dimensional space including the screen 70 on which the original image 80 is displayed using the virtual camera 30.

Here, the virtual camera refers to a virtual conceptual camera that does not actually exist.

When the projector 100 and the screen 70 are installed as shown in FIG. 4 and the original image 80 of FIG. 2 is projected on the screen 70, the virtual image including the screen 70 on which the original image 80 is projected is projected. A three-dimensional space is assumed and a virtual camera 30 looking at this virtual space is assumed. The virtual camera 30 photographs a virtual three-dimensional space in front of the projector 100, and variously captured images are obtained according to the position of the virtual camera 30, the properties of the lens, and the like.

As shown in FIG. 5, the virtual camera 30 is positioned at (0,0,0), which is the origin, and the original image 80 to be displayed is positioned at coordinates (0,0,1), in a three-dimensional space. Can be expressed in three-dimensional coordinates.

The three-dimensional transformation step S02 is a step of creating a viewport by performing three-dimensional transformation on the virtual three-dimensional image photographed in the virtual camera photographing step S01. In order to display image data photographed in 3D space on a 2D screen or a monitor, 3D transformation is performed. The three-dimensional transformation consists of a world transformation that determines the position of objects, a view transformation that determines the position of the camera, and a projection transformation that transforms the three-dimensional coordinate system into a two-dimensional coordinate system. Create a viewport by mapping this information to pixels.

The image change request input step S03 is a step of requesting to change and project an image as instructed by a user. The user may request to enlarge or reduce the size of the projected image or to move the position of the projected image up or down or left and right. In addition, when the user is located on the side away from the front of the screen, the user may request to rotate the projection image so as to recognize the image without distortion.

As an image change request method, a menu of a resize mode, a position adjustment mode, and a rotation mode may be provided in the manipulator, and a method of selecting or directly inputting the type of adjustment mode or the degree of adjustment may be performed. You can also choose to resize, position, rotate, or request more than one.

As a mechanism used for the image change request, a mouse, a remote controller, a controller, and the like can be widely used. When the image projected on the screen is simultaneously displayed on a computer monitor or the like, it is also possible to change the projection image by moving the mouse connected to the computer or the like by setting the position of the cursor as the position of the virtual camera. In this case, the movement of the mouse functions as a function of collectively indicating the type of adjustment mode and the degree of adjustment of the size adjustment, position adjustment, and rotation.

The mechanism used for the image change request may be provided by adding the image change request function module of the present invention to an existing home appliance controller such as a remote controller.

The view conversion projection conversion change step (S04) is a step of providing a viewport changed according to a user position by changing a view conversion and / or projection conversion mapped to a viewport created by photographing an original image with a virtual camera.

Since the virtual camera 30 looking at the three-dimensional space assumes the eyes of the user looking at the screen, in the image change request step (S03), the user adjusts the size, position, and rotation of the projection image according to his position. Specifying the mode type and the adjustment degree has the same effect as specifying the position and the attribute of the virtual camera 30 after all. In addition, changing the position and the attribute of the virtual camera 30 means that a new viewport is provided by changing the matrix value of the view transformation and / or projection transformation mapped to the viewport of the current projection image. Here, the current projection image refers to an image scene displayed on the screen at the time of the image change request. In the case of the first image change request, the screen on which the original image is displayed is taken by a virtual camera. Refers to the changed video scene reflecting the request for changing the video up to the point in time.

The method of changing the view transformation or projection transformation differs depending on the type of adjustment mode of the projected image, i.e. resizing, positioning, or rotation.

In the case of the first resizing, a zoom conversion matrix, S '(Equation 1-1) having a zoom value input by a mouse or the like as an element, is constructed and the projection conversion matrix of the current projection image (Equation 1) 1-2), multiply M and replace M with the result matrix, M '(Equation 1-3). This enlarges / reduces the size of the projected image. Where is the zoom value to be applied to the X and Y axes. M is a projection transformation matrix that combines the zoom transformation, S _m , translation transformation, T _m , rotation transformation, and R _m applied to the current scene.

(수학식 1-1)

(Equation 1-1)

(수학식 1-2)

(Equation 1-2)

(수학식 1-3)

(Equation 1-3)

는 각각 X, Y축에 대한 이동값을 의미한다.In the case of the second position adjustment, a parallel shift transformation matrix (Equation 2-1) having the amount of parallel movement input by a mouse or the like as an element is constructed and applied to the projection transformation matrix M of the current projected image, and as a result, the matrix, Replace M with M '(Equation 2-2). This moves the position of the projected image up, down, left and right. here

Denotes a shift value about the X and Y axes, respectively.

(수학식 2-1)

(Equation 2-1)

(수학식 2-2)

(Equation 2-2)

로 구성되며,

를 각각 카메라의 위치, 우향, 상향, 전방 벡터라고 하면 뷰 변환행렬 V'가 구성된다(수학식 3-1). V'의 전치행렬로 현재 장면의 뷰변환행렬 V를 대체한다(수학식 3-2). 이로써 투사영상이 사용자 시선방향에 맞게 변환된다.In the case of the third rotation, a view transformation matrix is generated using a virtual camera vector representing the current user's gaze direction input from a mouse or the like. Virtual camera vector

Consists of,

If is the position, right, up, and forward vector of the camera, respectively, the view transformation matrix V 'is constructed (Equation 3-1). The transpose matrix of V 'replaces the view transformation matrix V of the current scene (Equation 3-2). This converts the projected image according to the user's line of sight.

(수학식 3-1)

(Equation 3-1)

(수학식 3-2)

(Equation 3-2)

The modified image projection step S05 is a step of projecting the image changed by the new viewport created by changing the view transformation and / or projection transformation onto the screen 70.

6A shows an image in which an enlarged image is projected onto the screen when the user requests an enlargement of the projected image.

6B shows an image in which an enlarged image is projected onto the screen when the user requests a reduction of the projected image.

Fig. 6C shows an image in which the first image shown by the dotted line is moved to the right and the bottom in parallel when the user requests the position adjustment of the projected image.

Fig. 6D shows an image in which the first image shown by the dotted line is rotated on the X-axis and projected on the screen when the user requests the rotation of the projected image positioned below the screen.

FIG. 6D is a view in which an image of Y-axis rotation of the first image, which is indicated by a dotted line, is projected onto the screen when the user requests the rotation of the projection image located on the left side of the screen.

FIG. 6E illustrates an image in which an image of Z-axis rotation of the first image, which is indicated by a dotted line, is projected on the screen when the user is located in a clockwise oblique position and requests rotation of the projection image.

When the projection image is requested to be repositioned, the image on which the first image, which is indicated by the dotted line, is moved to the right and the lower side is projected on the screen. As shown in Fig. 6C, a reduced image than the first image is projected on the screen.

On the other hand, the present invention is the viewer after the virtual camera shooting step (S01), three-dimensional conversion step (S02), image change request input step (S03), view conversion projection conversion change step (S04), the changed image projection step (S05) When the image change request is continuously performed, the image change request input step S03, the view conversion projection conversion change step S04, and the changed image projection step S05 may be repeatedly performed.

2 is a block diagram showing an apparatus for adjusting a projection image according to the present invention.

The apparatus shown in FIG. 2 includes an image projector 10, an original image support unit 20, a virtual camera 30, an image change unit 40, a user interface unit 50, and a controller 60. The adjusting device of the projected image configured to project the image on the screen 70.

The image projector 10 receives original image information from the original image support unit 20 or receives changed image information from the image changing unit 40 to project light to display an image on the screen 70. If the image on the screen 70 is distorted, the image projection unit 10 may also perform a function of correcting image distortion according to an inclination angle with respect to the reference plane of the projector 100 and the screen 70 by performing ladder-type correction. have.

The original image support unit 20 transmits the image information received from the external device or the image information stored in itself to the image projector 10, and supports the original image 80 to the virtual camera 30 to support the virtual camera 30. Allows you to shoot virtual space.

The original image support unit 20 may be configured as a connection hardware for receiving image information from an external device and a memory device for storing the same.

The virtual camera 30 is a virtual conceptual camera that does not actually exist, and photographs a three-dimensional space in front of the project 100. In this case, the three-dimensional space in front of the project 100 refers to a virtual three-dimensional space including the screen 70 on which the original image 80 is projected.

The virtual camera 30 operates the camera as a numerical value so that it is not limited by physical conditions. According to the image change request input through the user interface unit 50, the position of the camera, the lens properties, and the like are manipulated by a numerical method.

The image changing unit 40 changes the original image 80 by combining the original image 80 and the numerical information of the virtual camera 30 operated according to the image change request of the user.

The change method differs depending on the type of adjustment mode of the projected image, i.e., size, position or rotation.

When the image change request is a request for resizing, a zoom transformation matrix having a zoom value input by a mouse or the like as an element is constructed and applied to the projection transformation matrix of the current projected image. This enlarges / reduces the size of the projected image.

When the image change request is a request for position adjustment, a parallel shift transformation matrix having a parallel shift amount input by a mouse or the like as an element is constructed and applied to the projection transformation matrix of the current projected image. This moves the position of the projected image up, down, left and right.

When the image change request is a request for rotation, the most suitable direction for the user's gaze direction can be set by rotating the projection image on three axes X, Y, and Z. A rotation transformation matrix is constructed using the rotation amount input for each axis, and the transpose matrix of the transformation matrix is applied to the view transformation matrix of the current projected image.

The user interface unit 50 serves to receive an image change request of the user. A mouse remote controller, a controller, or the like can be used, and can also be configured as a device combined with these wired or wireless. In particular, the image change request function module of the present invention may be added to an existing home appliance controller such as a remote controller.

The user interface unit 50 may be configured to present or provide a menu to the user so as to select or input the type of adjustment mode and the degree of adjustment of the size adjustment, position adjustment, and rotation. In addition, the image projected on the screen 70 may be simultaneously displayed on a computer monitor or the like and may be configured to directly determine the position of the virtual camera 30 by clicking with a mouse. At this time, the movement of the mouse actually functions to collectively instruct the type of adjustment mode, size of adjustment, position adjustment, and rotation.

If the image change request input through the user interface unit 50 does not require adjustment of the image, the controller 60 controls the image projector 10 to project the original image on the screen. When the image change request requires the adjustment of the image, the virtual camera 30 and the image changing unit 40 are controlled to change the image to suit the position of the viewer and project the image on the screen 70.

The screen 70 is the side on which the image is projected.

On the other hand, the projection image control unit 10, the original image support unit 20, the virtual camera 30, the image changing unit 40, the user interface unit 50 and the control unit 60, the projection image adjusting device is a projector (100). It can be composed of a part of).

The method and apparatus for adjusting a projection image using a virtual camera according to the present invention can be modified and applied in various forms within the scope of the technical idea of the present invention. The embodiments and the drawings are only for the purpose of describing the contents of the invention in detail, and are not intended to limit the scope of the technical idea of the invention, the scope of the present invention is not limited to the claims to be described later, the equivalent scope thereof. Should be judged.

According to the present invention, the size, position, and orientation of the projected image can be freely adjusted without moving the projector or changing the internal hardware, thereby making the use of the projector much more convenient.

In addition, since the image with less distortion can be viewed even at a position outside the front of the screen, the restrictions on the projector installation location and installation location are relaxed, and the user's position and posture are also free.

Furthermore, since the projection image is software-adjusted, the projection image can be finely adjusted than when the user directly manipulates the projector position and the degree of tilt.

Claims (6)

In the method of adjusting the projection image, Photographing a virtual three-dimensional space including a screen on which an original image is displayed by using a virtual camera (a); (B) creating a viewport by three-dimensionally converting the image photographed by the virtual camera; (C) receiving an image change request from a user; (D) providing the modified viewport by changing the view transformation and / or projection transformation of the viewport according to the image change request; And And (e) projecting an image onto the screen by using the modified viewport. The method of claim 1, And repeating step (c), step (d) and step (e) sequentially. The method according to claim 1 or 2, And the image change request includes a request for a size change, a position change, and a rotation change of the image. The method of claim 3, wherein The size change of the image is performed using an enlarged or reduced conversion matrix having an enlarged or reduced value as an element and a changed projection conversion matrix obtained by calculating a projection conversion matrix of the current projected image. The position change of the image is performed by using a parallel shift transformation matrix having a parallel shift amount as an element and the changed projection transformation matrix obtained by calculating a projection transformation matrix of the current projected image. The rotation of the image is adjusted by using a transpose matrix of the view transformation matrix obtained by using a virtual camera vector indicating the user's line of sight. The method according to claim 1 or 2, The image change request is input by using a mouse, a remote controller, a controller, etc. The adjustment method of the projection image using a virtual camera. In the adjusting device of the projection image, An image projection unit for projecting the image onto the screen; An original image support unit supporting the original image; The virtual camera photographing the virtual three-dimensional space; An image changer for changing the view transformation and / or projection transformation of the viewport according to the image change request to provide the changed viewport; A user interface unit for inputting a manipulation signal of the user; And And a controller for controlling the image projector, the virtual camera, and the image changing unit according to the user's manipulation signal.

Images