WO2018110822A1

WO2018110822A1 - Method of projecting image onto curved projection area and projection system therefor

Info

Publication number: WO2018110822A1
Application number: PCT/KR2017/011753
Authority: WO
Inventors: Kyung Yoon Jang; Hyung Jin Yoon
Original assignee: Cj Cgv Co., Ltd.
Priority date: 2016-12-16
Filing date: 2017-10-24
Publication date: 2018-06-21
Also published as: KR101847996B1; JP2020507100A; EP3539289A1; CN110089111A; IL267087A; EP3539289A4

Abstract

Disclosed are an image projection method and system in a theater equipped with a projection area of a curved surface and a plurality of projection apparatuses. Specifically, disclosed are an image projection method and system, which can define a plurality of projection apparatuses as one cluster by computing relative transform information between the projection areas of the plurality of projection apparatuses and can project a single image onto a curved projection area without distortion through the plurality of projection apparatuses by defining a relative relation between the reference projection apparatus of the cluster and the projection area of the original image.

Description

METHOD OF PROJECTING IMAGE ONTO CURVED PROJECTION AREA AND PROJECTION SYSTEM THEREFOR

The present invention relates to an image projection method and system in a theater equipped with a projection area of a curved surface and a plurality of projection apparatuses and, more particularly, to an image projection method and system, which can define a plurality of projection apparatuses as one cluster by computing relative transform information between the projection areas of the plurality of projection apparatuses and can project a single image onto a curved projection area without distortion through the plurality of projection apparatuses by defining a relative relation between the reference projection apparatus of the cluster and the projection area of the original image.

In order to provide a screening environment having a more three-dimensional effect and a high sense of reality, efforts to implement a single image using a plurality of projection apparatuses recently continue to be made. In particular, as the demand for movie appreciation through a large-sized screen increases, lots of interests in a technology for projecting a single image onto a large-sized screen using a plurality of projection apparatuses are focused.

In a conventional technology, in general, a plane screen has been used as a theater screen. In this case, when a single image is to be implemented on the large-sized screen of the plane using a plurality of projection apparatuses, an attempt to define the plurality of projection apparatuses as one cluster has been made. In this case, when clustering is performed, keystone correction is performed using homography assuming that most of projection surfaces are planes.

Recently, however, as a "curved screen" is known to be more advantageous in forming an improved three-dimensional effect and a high sense of immersion, the number of theaters in which a projection surface is formed of a curved surface instead of a plane screen is increased. In line with such an increase, an effort to implement a single image using a plurality of projection apparatuses even in a projection area of a curved surface is made.

In a conventional technology, when clustering is performed, homography is used assuming that a screen is a plane. If the homography is applied to a projection area of a curved surface, there is a problem in that complicated distortion generated due to the curved surface is not removed. Furthermore, there are problems in that the focus is off in an overlap area because a relative relation between projection apparatus is not clearly defined and that edge blending is inaccurate.

In order to solve such problems, software solutions have been suggested. However, such solutions provide only a correction controller because the form of a curved surface is limited to some forms, such as a dome and a cylinder. Accordingly, there is a problem in that it is difficult to use the solutions for a specific curved projection surface.

The present invention is related to the projection of an image onto a specific curved projection surface using a plurality of projection apparatuses as described above, and has been made to satisfy the aforementioned technological needs and also to provide additional technological factors which may not be easily invented by a person having ordinary skill in the art.

[Prior Art Document]

[Patent Document]

(Patent Document 1) Korean Patent Application Publication No. 10-2007-0061254 (June 13, 2007)

Accordingly, an object of the present invention is to facilitate control by clustering a plurality of projection apparatuses in implementing an image on a projection area of a curved surface using the plurality of projection apparatuses.

In particular, an object of the present invention is to implement a cluster of projection apparatuses, which is capable of projecting an image on a projection surface different from a typical curved surface without distortion by relative transform information computation and clustering between projection apparatuses based on subdivision and a warping scheme based on a thin plate spline (TPS) unlike in conventional projection apparatus clustering based upon the premise that an image is projected onto a plane projection surface.

Furthermore, an object of the present invention is to dispose an image to be projected onto a projection area of a curved surface so that a single image can be implemented on the curved projection surface by a cluster of projection apparatuses without distortion.

In particular, an object of the present invention is to minimize the distortion of an image attributable to a curved surface in such a manner that when the origin image to be projected and a projection area of a curved surface are matched, a plurality of virtual control points is set in the original image and the origin image is accurately matched with a shape of the curved projection surface by moving the control points.

In one aspect, a method of projecting an image onto a curved surface using a plurality of projection apparatuses includes the steps of (a) computing, by a projection management apparatus, relative transform information between a plurality of projection apparatuses based on projection areas of the projection apparatuses and generating a projection apparatus cluster by grouping the plurality of projection apparatuses based on the relative transform information and (b) matching, by the projection management apparatus, an image to be projected by the projection apparatus cluster with a projection area of a curved surface.

In another aspect, the projection area of the curved surface onto which a specific projection apparatus can project an image and a projection area of a projection apparatus adjacent to the specific projection apparatus partially overlap.

In yet another aspect, the step of generating the projection apparatus cluster includes the steps of (a-1) setting any one of the plurality of projection apparatuses as a reference projection apparatus and (a-2) computing relative transform information between the reference projection apparatus and a projection apparatus belonging to projection apparatuses adjacent to the reference projection apparatus and included in the projection apparatus cluster.

In yet another aspect, the step (a-2) includes extracting, by the projection management apparatus, a correspondence point Ci from the projection area of the reference projection apparatus within an overlap area in which the projection area of the reference projection apparatus and the projection area of a projection apparatus adjacent to the reference projection apparatus partially overlap, extracting a correspondence point Cj from the projection area of the projection apparatus adjacent to the reference projection apparatus, and computing the relative transform information by substituting the correspondence points for a relative transform equation.

In yet another aspect, the relative transform equation is

,

wherein xⁱ: a pixel location of the reference projection apparatus, Cⁱ _k is a location of each vertex of a correspondence point mesh, w_k: weight, and A(xⁱ) is Global Affine Transformation.

In yet another aspect, the method further includes the step (a-3) of computing relative transform information between the reference projection apparatus and a projection apparatus which belongs to the projection apparatuses adjacent to the reference projection apparatus and which is not included in the projection apparatus cluster.

In yet another aspect, the step of matching the image to be projected by the projection apparatus cluster with the projection area of the curved surface includes the steps of (b-1) extracting a specific number of initial points from the outermost of an image to be projected by the projection apparatus cluster; (b-2) disposing a plurality of control points within the image to be projected and subdividing an area into a plurality of sub-image areas by connecting the plurality of control points; and (b-3) matching the initial points and the control points with a shape of a projection area of the curved surface.

In yet another aspect, the step (b-3) is performed using a bilinear interpolation scheme.

In yet another aspect, the initial point is the outermost vertex of the image to be projected.

In yet another aspect, the plurality of control points is disposed at equal intervals.

In yet another aspect, a system for projecting an image onto a curved surface includes a projection management apparatus configured to compute relative transform information between a plurality of projection apparatuses included in a theater, generate a projection apparatus cluster by grouping the plurality of projection apparatuses based on the relative transform information, and match an image to be projected by the projection apparatus cluster with a projection area within the theater and the plurality of projection apparatuses configured to project images onto the projection area with the theater.

In yet another aspect, the projection management apparatus sets any one of the plurality of projection apparatuses as a reference projection apparatus and computes relative transform information between projection apparatuses adjacent to the reference projection apparatus.

In yet another aspect, the projection management apparatus extracts a correspondence point Ci from a projection area of the reference projection apparatus within an overlap area in which the projection area of the reference projection apparatus and a projection area of a projection apparatus adjacent to the reference projection apparatus partially overlap, extracts a correspondence point Cj from the projection area of the projection apparatus adjacent to the reference projection apparatus, and computes the relative transform information by substituting a relative transform equation into the correspondence points.

In yet another aspect, the projection management apparatus matches the image to be projected by the projection apparatus cluster with the projection area of a curved surface, extracts a specific number of initial points from the outermost of an image to be projected by the projection apparatus cluster, disposes a plurality of control points within the image to be projected and subdividing an area into a plurality of sub-image areas by connecting the plurality of control points, and matches the initial points and the control points with a shape of a projection area of the curved surface.

In accordance with an embodiment of the present invention, there is an effect in that a plurality of projection apparatuses can be grouped into one cluster and controlled. Specifically, in accordance with an embodiment of the present invention, a task, such as an image area arrangement or image correction, can be performed by controlling a plurality of projection apparatuses for each cluster. For example, in accordance with an embodiment of the present invention, a reference projection apparatus has been defined within a cluster of projection apparatuses. Accordingly, there is an effect in that a plurality of projection apparatuses can be easily controlled because an image area arrangement or image correction is automatically performed on the remaining projection apparatuses if a task, such as an image area arrangement or image correction, is performed based on the reference projection apparatus.

Furthermore, in accordance with an embodiment of the present invention, relative transform information between projection apparatuses can be obtained based on subdivision and a thin plate spline (TPS) operation when projection apparatus clustering is performed. Accordingly, there is an effect in that projection apparatuses can be effectively clustered with respect to an atypical curved projection surface unlike in conventional clustering based upon the premise of a plane projection surface.

Furthermore, in accordance with an embodiment of the present invention, there is an effect in that an image can be effectively matched with an atypical curved surface using a subdivision method in the step of arranging image areas after the step of clustering projection apparatuses.

FIG. 1 shows a method of projecting an image onto a curved surface according to an embodiment of the present invention according to order.

FIG. 2 is a detailed diagram showing the step of clustering a plurality of projection apparatuses.

FIG. 3 shows the state in which the projection areas of the projection apparatuses have overlapped.

FIG. 4 is a diagram helping understanding of relative transform information between the projection apparatuses.

FIG. 5 is a diagram illustrating a projection area according to the cluster of projection apparatuses.

FIG. 6 is a detailed diagram showing the step of matching an image to be projected with the projection area of a curved surface.

FIGS. 7 and 8 show the step of matching an image with the projection area of a curved surface.

FIGS. 9 and 10 show the configuration of a system for projecting an image onto a curved surface according to an embodiment of the present invention.

100: projection apparatus

151a~151e: projection areas for respective projection apparatus

200: cluster of projection apparatuses

1000: projection management apparatus

1100: computation unit 1200: storage unit

1300: projection apparatus management unit

1400: communication unit

1500: user interface unit 1600: camera unit

1700: control unit

The details of the objects and technical configurations of the present invention and acting effects thereof will be more clearly understood from the following detailed description based on the accompanying drawings. Hereinafter, embodiments of the present invention are described in detail with reference to the accompanying drawings.

The embodiments disclosed in this specification should not be construed or used as limiting the scope of the present invention. It is evident to those skilled in the art that a description including the embodiments of this specification may have various applications. Accordingly, some embodiments described in the detailed description of the present invention are illustrative for better understanding, and the scope of the present invention is not intended to be restricted by the embodiments.

Functional blocks illustrated in the drawings and described hereunder are only examples of possible implementations. In other implementations, other functional blocks may be used without departing from the spirit and scope of the detailed description. Furthermore, one or more functional blocks of the present invention are illustrated as separate blocks, but one or more of the functional blocks of the present invention may be a combination of various hardware and software elements for executing the same function.

Furthermore, it should be understood that an expression that some elements are included is an expression of an open type and the expression simply denotes that the corresponding elements are present, but does not exclude additional elements.

Furthermore, when one element is described as being connected or coupled to the other element, it should be understood that one element may be directly connected or coupled to the other element, but a third element may be interposed between the two elements.

Furthermore, it is to be understood that a method of projecting an image onto a curved surface to be described hereinafter may be implemented through the cooperative operation of various types of hardware and software. For example, the method may be implemented through a cooperative operation between a plurality of projection apparatuses and a projection management apparatus (or server) connected to the projection apparatuses in a wired or wireless manner. In addition to such a cooperative operation, the method may be implemented by the cooperative operation of various types of hardware and software.

FIG. 1 is a brief diagram showing a method of projecting an image onto a curved surface in a theater equipped with a projection area of a curved surface and a plurality of projection apparatuses.

Referring to FIG. 1, the method of projecting an image onto a curved surface according to an embodiment of the present invention includes a first step of computing, by a projection management apparatus, relative transform information between a plurality of projection apparatuses based on the projection areas of the projection apparatuses and clustering the plurality of projection apparatuses based on the relative transform information and a second step of matching, by the projection management apparatus, images to be projected by projection apparatus clusters with a projection area of a curved surface.

Hereinafter, the first step of clustering the projection apparatuses and the second step of matching images with the projection area of the curved surface are separately described.

The first step of clustering the projection apparatuses is first described below with reference to FIGS. 2 to 4.

FIG. 2 shows the subdivision of the clustering step. Referring to FIG. 2, the clustering step includes step S101 of setting any one of the plurality of projection apparatuses as a reference projection apparatus and step S103 of computing relative transform information between the reference projection apparatus and a projection apparatus adjacent to the reference projection apparatus.

Prior to the full-scale description of step S101 and step S103, first, in this step, it is based upon the premise that the projection areas of the plurality of projection apparatuses to be clustered have been arranged to partially overlap. As described above, an object of the present invention is to project images onto a single projection surface through the plurality of projection apparatuses, but to enable all of the projected images to generally implement a single completed image. Accordingly, it is based upon the premise that some projection areas of the plurality of projection apparatuses described in the present invention have been arranged to overlap as in FIG. 3.

In this case, the projection areas of adjacent projection apparatuses may overlap in the form of a square having four sides and four vertexes. In order to compute relative transform information between the projection apparatuses, at least four or more pieces of point correspondence information are necessary. The reason for this is that if some projection areas overlap in the form of a square as described above, point correspondence information can be easily extracted. Furthermore, if an overlap projection area is a square, image correction (e.g., edge blending or a black offset) for the overlap projection area can be easily performed because the overlap projection area can be specified relatively clearly. It is however to be noted that the projection area of an adjacent projection apparatus is necessarily limited to a specific geometric shape (e.g., a rectangle) and the projection areas of projection apparatuses may overlap in various forms depending on the arrangement state of the projection apparatuses.

Referring back to step S101, the projection management apparatus sets any one of the plurality of projection apparatuses as a reference projection apparatus. In this case, the reference projection apparatus is not limited by a special condition, and any one of the plurality of projection apparatuses may be freely set as the reference projection apparatus.

In relation to a reason why the reference projection apparatus is set, in the case of clustering in a plane projection surface (assuming that the projection surface of FIG. 3 is a plane), a relative homography is computed by performing matching the correspondence points of adjacent P1 and P3, and P1 and P2, and a relative relation between P2 and P3 can be automatically obtained by combining the two computed homographies. In the case of clustering in a projection area of a curved surface, correspondence points for all of adjacent projection apparatuses must be input because accurate geometric information about the curved surface cannot be aware.

As a result, unlike in the clustering in the plane, in the case of the clustering in the curved surface, a circulating correspondence relation (e.g., P1↔P2↔P3) is generated. If only relative transform information in any one direction is computed, for example, if only relative transform information in the direction of P1↔P2 is computed, the results of computation in the direction of P2↔P3 are influenced. Accordingly, there is a problem in that independent computation is difficult as in the plane. In an embodiment of the present invention, in order to solve the problem, after a reference projection apparatus is selected, pieces of relative transform information are sequentially computed with respect to all of the projection apparatuses.

In step S103, relative transform information between the reference projection apparatus and an adjacent projection apparatus is computed. In an embodiment of the present invention, the relative transform information is computed using a thin plate spline (TPS). The TPS is one of schemes used in computer graphics and a computer vision for scattered data interpolation. The TPS can efficiently process correspondence points (e.g., the matching of four correspondence points) input by a user and irregularly generated in a projector overlap area because the TPS is expressed in the form of a radial basis function (RBF). Furthermore, the TPS has an advantage in that the remaining projection areas outside an overlap area can also be extrapolated because Global Affine Transformation (GAT) is also computed.

GAT is one of models for representing a motion of an object as a function for representing point correspondence in an n-dimensional space. GAT in a two-dimension space is a 2*3 matrix and defined as follows.

If a two-dimension point p=(x,y) is moved to p'=(x',y') using GAT applied to a two-dimensional point, p' is as follows.

That is, the two-dimension point p=(x,y) is moved to p'=(x',y'), that is, x'= a00*x + a01*y + a02 and y'= a10*x + a11*y + a12. If all of pixels are moved using GAT as in the aforementioned method, an image projected by a projection apparatus is deformed. As a result, the remaining projection areas outside an overlap area can be computed by extrapolating, that is, estimating them.

A process of computing, by the projection management apparatus, actual relative transform information is described below with reference to FIG. 4.

C_i and C_j within an overlap area may be present as correspondence points between the reference projection apparatus of n projection apparatuses and a projection apparatus adjacent to the reference projection apparatus. The number of correspondence points is identically "m" between the reference projection apparatus and the adjacent projection apparatus. In relation to the number of correspondence points, a larger number of correspondence points may be necessary for the clustering of a curved surface compared to four correspondence points used for the clustering of a plane in a conventional technology. That is, the accuracy of a curved surface geometric approximation through the TPS using only the existing four correspondence points may be slightly low. In order to solve this problem, additional correspondence points through a subdivision may be generated in addition to the four correspondence points. In this case, the coordinates of the additional correspondence points may simply have an average value of surrounding points, so the division (mesh) of a projection area can be smoothly maintained.

In this state, the computation of relative transform information from the reference projection apparatus to an adjacent projection apparatus and the computation of relative transform information from the adjacent projection apparatus to the reference projection apparatus are performed according to the following equation.

[Equation 1]

In Equation 1, xⁱ indicates the location of pixels of a projection apparatus pⁱ, and Cⁱ _k indicates the location of a correspondence point and means a center value that defines a TPS function. Furthermore, w_k indicates weight of each center value. The TPS uses r^2*log r as an RBF kernel function

(r).

A(xⁱ) indicates GAT. w_k and A are calculated using the least square solver after a simultaneous linear equation is configured according to C_j=TPS_i _→j(C_i). The least square solver is a method of calculating a value most close to the most accurate value of a plurality of measured values and is a method of determining the sum of squares of an error so that the sum is the least.

A correspondence point mesh having four vertexes is as follows.

w_kof Equation 1 obtained using the least square solver after the simultaneous linear equation is configured according to Cj=TPSi→j(Ci) is as follows.

A of Equation 1 obtained using the least square solver after simultaneous linear equation is configured according to Cj=TPSi→j(Ci) is as follows.

If xⁱ of the projection apparatus i is (1043.01, 640.131), x^j, that is, relative transform information, is computed as (2160.57, 611.839) through Equation 1.

After the relative transform information between the reference projection apparatus and the adjacent projection apparatus is computed in step S103, the correspondence points are updated with the coordinates of the reference projection apparatus using TPS_i _→ref, that is, the computation result value (S105). That is, the correspondence points are applied to the reference projection apparatus using the relative transform information so that the projection area of the adjacent projection apparatus can be controlled by only control of the reference projection apparatus.

Step S101 to step S105 are repeatedly performed between the reference projection apparatus and a projection apparatus adjacent to the reference projection apparatus. That is, step S103 and step S105 are repeatedly performed on a projection apparatus adjacent to the reference projection apparatus.

FIG. 5 illustrates projection areas according to the plurality of projection apparatuses after the clustering step. From FIG. 5, it may be seen that the leftmost projection apparatus has a projection area of a relatively plane, projection apparatuses have curved surfaces toward the right side, and the rightmost projection apparatus has a projection area including a chiefly curved surface. It may also be seen that projection apparatuses that smoothly connect projection areas from the leftmost projection apparatus to the rightmost projection apparatus through curved surfaces are present between the leftmost projection apparatus and the rightmost projection apparatus. As described above, in an embodiment of the present invention, relative transform information is computed based on the projection areas of a plurality of projection apparatuses with respect to an atypical curved surface, and the plurality of projection apparatuses may be defined as a single cluster based on the relative transform information.

The step of clustering projection apparatuses has been described above with reference to FIGS. 2 to 5.

The step of matching images with a projection area of a curved surface, that is, the second step, is described below with reference to FIGS. 6 to 8.

Referring to FIG. 6, the second step includes step S201 of extracting a specific number of initial points from the outermost side of an image to be projected by a cluster of projection apparatuses, step S203 of setting a plurality of control points within the image to be projected and subdividing (so-called subdivision) an area into a plurality of sub-image areas by connecting the plurality of control points, and step S205 of adjusting a position so that the initial points and the control points are matched with a shape of a projection area of a curved surface.

FIG. 7 shows a comparison between a conventional image area arrangement method based on a projection area of a plane and an image area arrangement method in a projection area of a curved surface according to an embodiment of the present invention. Referring to FIG. 7(a), when image areas are arranged on the plane, the projection areas and the images can be matched using only four points. However, in order to designate an image area in a curved surface, it is insufficient to adjust the four points only as in the plane. The reason for this is that the estimation of a homography through the four points does not approximate a specific curved surface geometric. In order to solve this problem, more points, that is, control points, need to be used as in FIG. 7(b).

In step S201, the projection management apparatus first checks initial points in projection areas of a corresponding cluster with respect to clusters of projection apparatuses that have been clustered.

Referring to FIG. 7(b), the projection management apparatus first selects initial points in order to check the boundary of image areas. In this case, the initial point may be selected from a line that forms the outermost of an image. More specifically, the vertexes of an image area may be selected as respective initial points. The selected initial points are points, that is, a reference for matching an image area with projection areas according to a cluster. FIG. 8(a) shows the projection areas of a cluster formed of a curved surface and the original image to be matched with the projection areas. As shown in FIG. 8(b), initial points at the outermost of the image are adjusted so that they are matched with the projection areas.

In step S203, the projection management apparatus disposes a plurality of control points in an area onto which an image is to be projected and subdividing the area into a plurality of sub-image areas by connecting the plurality of control points. This is so-called a subdivision and corresponds to the step of subdividing a specific image area into sub-image areas.

FIG. 7(b) shows an embodiment in which virtual lines are disposed horizontally and vertically within an image and a plurality of control points is disposed in the image. In this case, the control points may be disposed at equal intervals. FIG. 8(b) shows the state in which the locations of the plurality of control points have been adjusted to be matched with the projection areas of the cluster, that is, the projection areas of the curved surface. As described above, the locations of initial points and control points in an image can be adjusted so that the initial points and control points are matched with the projection areas of a cluster. As the number of initial points and control points increases, more accurate matching is possible in line with a shape of an atypical curved surface.

The projection management apparatus adjusts the locations of the initial points or control points and simultaneously determines the coordinates of internal pixels included in each sub-image (or cell). In this case, in order to determine the coordinates, a bilinear interpolation scheme may be used. The reason for this is that the most stable results can be obtained compared to other interpolation schemes if each sub-image cell unit is assumed to be linear in the state in which an accurate geometric form of a curved surface is unknown.

The bilinear interpolation scheme is a scheme extended from one-dimensional linear interpolation to two-dimension linear interpolation.

The second step of matching the image area with the projection areas of a curved surface has been described above with reference to FIGS. 6 to 8.

The projection management apparatus according to an embodiment of the present invention may perform an additional function after the clustering and image area matching of the first step and second step. For example, the projection management apparatus may perform fine adjustment on each projection apparatus.

First, in relation to the fine adjustment function for each projection apparatus, the projection management apparatus may perform a fine adjustment step on each projection apparatus in order to correct an error of a correspondence point after clustering and image area matching and a fine calibration error attributable to the distortion of a projection apparatus lens.

In the fine adjustment, there is a slight difference between a plane fine adjustment method and a curved surface fine adjustment method.

In the case of the plane fine adjustment, four initial points or control points are given as initial values, and keystone adjustment using an additional homography can be performed by adjusting the four initial points or control points. Lens distortion or local correction that is difficult to correct using the keystone adjustment is corrected by increasing the number of control points through a subdivision. In particular, non-linear correction, such as lens distortion, is performed according to Equation 1 described in the aforementioned curved surface clustering.

A homography is a matrix capable of representing a change in the rotation and size of a three-dimensional plane, and is used to correct the distortion of an image generated when a projection apparatus obliquely projects an image. The homography is a matrix of 3*3 and defined as follows.

If p=(x,y) is moved to p'=(x',y') by applying a homography to the two-dimension point p=(x,y), x' and y' are as follows.

p'= (x',y') is a point deformed by the homography. If points are moved by applying the above method to all of pixels, an image projected by a projection apparatus may be deformed.

In the case of the curved surface fine adjustment, the projection management apparatus automatically generates control points for correction in an image area that has already been matched and disposed. When the control points for correction in the image area are connected, subdivision areas are generated to form grid meshes. The grid meshes are disposed based on the aforementioned "reference projection apparatus." Accordingly, the locations of the control points for correction in the grid meshes may be computed as in Equation 2.

[Equation 2]

TPS_ref _→i (i.e., a mapping function from the reference projection apparatus to a projection apparatus Pi) is the results of the computation of the relative transform information described in the first step. The control points for correction of a sub-image area (or cell) through a subdivision within each projection area of each projection apparatus may be determined according to Equation 2. As a result, some of control points (or grid meshes) disposed in the projection areas of a cluster with respect to each projection apparatus are used as control points for correction (or grid meshes for correction).

The process of projecting, by the projection management apparatus, an image onto a curved projection surface has been described above. A system for projecting an image onto a curved surface according to an embodiment of the present invention is described below with reference to FIGS. 9 and 10.

The system for projecting an image onto a curved surface is an exemplary system for implementing the aforementioned "method of projecting an image onto a curved surface". Accordingly, although the category is different, the characteristics described in relation to the "method of projecting an image onto a curved surface" may also be naturally analogized and applied to the system for projecting an image onto a curved surface.

Referring to FIG. 9, the system for projecting an image onto a curved surface according to an embodiment of the present invention may include two or more projection apparatus clusters and the projection management apparatus configured to control and manage the operations of the two or more projection apparatus clusters.

The two or more projection apparatus clusters are elements in which a plurality of projection apparatuses disposed in a theater has been grouped. In this case, each of the projection apparatus clusters may include a plurality of projection apparatuses 100.

A process of grouping the plurality of projection apparatuses disposed in a theater may be performed in various manners. The plurality of projection apparatuses may be grouped based on projection surface information formed in the theater. For example, if a theater is a multi-projection theater including a plurality of projection surfaces and a "front projection surface, a left projection surface, a right projection surface, a ceil projection surface and a bottom projection surface" are formed within the theater, a plurality of projection apparatuses disposed in the theater may be grouped into a "cluster for projecting an image onto the front projection surface, a cluster for projecting an image onto the left projection surface, a cluster for projecting an image onto the right projection surface, a cluster for projecting an image onto the ceil projection surface, and a cluster for projecting an image onto the bottom projection surface." That is, a plurality of projection apparatuses for projecting images onto a specific projection surface together may be grouped into a single cluster.

FIG. 9 shows an example in which a plurality of projection apparatuses disposed in a theater has been grouped into three clusters, that is, a cluster A, a cluster B and a cluster C. In this case, the three clusters may be formed in various ways, but may be formed based on projection surface information formed in the theater as described above. For example, the cluster A may be formed by grouping projection apparatuses for projecting images onto the left projection surface. The cluster B may be formed by grouping projection apparatuses for projecting images onto the front projection surface. The cluster C may be formed by grouping projection apparatuses for projecting images onto the right projection surface.

The projection management apparatus controls and manages the two or more projection apparatus clusters. Specifically, the projection management apparatus is an element capable of controlling a plurality of projection apparatuses disposed in a theater for each cluster or individually controlling the projection apparatuses.

The projection management apparatus may be implemented in the form of various electronic devices and may be implemented as a single electronic device or in a form in which some electronic devices are interconnected. For example, the projection management apparatus may be implemented in a form including one server apparatus or may be implemented in a form in which two or more servers are interconnected. Alternatively, the projection management apparatus may be implemented in a form in which a server and other electronic devices are connected or may be implemented by other electronic devices other than a server. Alternatively, the projection management apparatus may be implemented in a form including input and output devices for a user interface.

An example of detailed elements which may be included in the projection management apparatus is described below with reference to FIG. 10.

Referring to FIG. 10, the projection management apparatus may include a computation unit 1100, a storage unit 1200, a projection apparatus management unit 1300, a communication unit 1400, a user interface unit 1500, a camera unit 1600 and a control unit 1700. In addition to the elements, the projection management apparatus may further include various elements for managing projection apparatus clusters.

The computation unit 1100 computes and manages relative transform information of projection apparatuses for each cluster with respect to the two or more projection apparatus clusters.

The computation unit 1100 may compute relative transform information between a reference projection apparatus and the remaining projection apparatuses included in each cluster in the state in which projection areas of a plurality of projection apparatuses included in each cluster have overlapped.

The process of computing relative transform information between projection apparatuses has been described above, and thus a detailed description thereof is omitted hereinafter.

The storage unit 1200 stores various pieces of information related to the system for projecting an image onto a curved surface according to an embodiment of the present invention. In particular, the storage unit 1200 may store relative transform information generated by the computation unit 1100, and may database the relative transform information. In this case, the storage unit 1200 may assign an identifier to each cluster and may database relative transform information. Furthermore, the storage unit 1200 may assign an identifier to each of projection apparatuses included in each cluster and may database transform information.

Furthermore, the storage unit 1200 may store the aforementioned various data temporarily or permanently, and may be configured in a form including various types of memory devices.

The projection apparatus management unit 1300 controls the operation of a plurality of projection apparatuses included in the two or more projection apparatus clusters. Specifically, the projection apparatus management unit 1300 may control the lens, body, etc. of each of projection apparatuses disposed in a theater, and may control the projection direction of each projection apparatus through such control.

As described above, in order for the computation unit 1100 to compute relative transform information for each cluster, the projection areas of projection apparatuses included in each cluster need to be arranged to overlap (i.e., the projection areas of adjacent projection apparatuses need to be arranged to overlap in a square form). Such an arrangement state may be controlled through the operation of the projection apparatus management unit 1300.

The projection apparatus management unit 1300 may control projection apparatuses included in each cluster based on various types of information. For example, the projection apparatus management unit 1300 may control the operation of the projection apparatuses based on information received through the user interface unit 1500, information generated by the camera unit 1600, and information received through the communication unit 240. Furthermore, the projection apparatus management unit 1300 may control the projection direction of the projection apparatus by complexly considering such pieces of information, and can further improve the accuracy of control through such an operation.

The communication unit 1400 transmits and receives various types of information related to the operation of the system. The projection management apparatus may be connected to a plurality of projection apparatuses, a plurality of screening apparatuses, a user terminal and an external server apparatus disposed in a theater in a wired or wireless manner through the communication unit 1400, and may transmit and receive various pieces of information necessary for the operation of the system.

The communication unit 1400 may include various types of wired or wireless transmission/reception modules (transceivers), and may transmit and receive data over a wired communication network or a wireless communication network of various communication standards.

The user interface unit 1500 implements an environment that enables an interface with a user. The user interface unit 1500 may include various input devices, display devices and sound output devices, and may receive information, that is, a basis for control of the system, from a user or provide various types of information related to the system.

If the projection apparatus management unit 1300 controls the operation of the projection apparatus based on information received through the user interface unit, the user interface unit 1500 may implement various visual functions capable of helping to identify an overlap projection area. For example, the user interface unit 1500 may implement a function of displaying only the projection areas of two adjacent projection apparatuses forming an overlap projection area and not displaying the projection areas of the remaining projection apparatuses, a function of displaying the projection areas of adjacent projection apparatuses in different colors (e.g., blue and red) and a function of visually displaying the correspondence points of an overlap projection area.

The camera unit 1600 is disposed within a theater, and senses various types of visual information within the theater or visualizes the sensed visual information. The camera unit 1600 may include various camera modules.

In particular, the camera unit 1600 may recognize a projection area onto which each projection apparatus disposed in a theater projects an image, and may visually recognize an overlap projection area formed by two or more adjacent projection apparatuses. Furthermore, the camera unit 1600 may transfer such information to the projection apparatus management unit 1300 so that the information is used as basic information for control of the operation of the projection apparatus.

The control unit 1700 controls the operations of various elements of the projection management apparatus, including the computation unit 1100, the storage unit 1200, the projection apparatus management unit 1300, the communication unit 1400, the user interface unit 1500 and the camera unit 1600, individually or complexly.

The control unit 1700 may include at least one operation means. In this case, the operation means may be a general-purpose central processing unit (CPU), but may be a programmable device (CPLD, FPGA), application-specific integrated circuit (ASIC) or microcontroller chip implemented for a specific purpose.

The invention has been described in detail with reference to preferred embodiments thereof. However, it will be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims

A method for a projection management apparatus to project an image onto a curved surface using a plurality of projection apparatuses, the method comprising steps of:

(a) computing relative transform information between a plurality of projection apparatuses based on projection areas of the projection apparatuses and generating a projection apparatus cluster by grouping the plurality of projection apparatuses based on the relative transform information; and

(b) matching an image to be projected by the projection apparatus cluster with a projection area of a curved surface.
The method of claim 1, wherein a projection area of the curved surface onto which a specific projection apparatus of the plurality of projection apparatuses is capable of projecting an image and a projection area of a projection apparatus adjacent to the specific projection apparatus partially overlap.
The method of claim 2, wherein the step (a) comprises steps of:

(a-1) setting any one of the plurality of projection apparatuses as a reference projection apparatus; and

(a-2) computing relative transform information between the reference projection apparatus and a projection apparatus belonging to projection apparatuses adjacent to the reference projection apparatus and included in the projection apparatus cluster.
The method of claim 3, wherein the step (a-2) comprises:

extracting a correspondence point Ci from a projection area of the reference projection apparatus within an overlap area in which the projection area of the reference projection apparatus and a projection area of a projection apparatus adjacent to the reference projection apparatus partially overlap,

extracting a correspondence point Cj from the projection area of the projection apparatus adjacent to the reference projection apparatus, and

computing the relative transform information by substituting the correspondence points for a relative transform equation.
The method of claim 4, wherein the relative transform equation is

wherein xⁱ: a pixel location of the reference projection apparatus, Cⁱ _k is a location of each vertex of a correspondence point mesh, w_k: weight, and A(xⁱ) is Global Affine Transformation.
The method of claim 3, further comprising a step (a-3) of computing relative transform information between the reference projection apparatus and a projection apparatus which belongs to the projection apparatuses adjacent to the reference projection apparatus and which is not included in the projection apparatus cluster.
The method of claim 1, wherein the step (b) comprises steps of:

(b-1) extracting a specific number of initial points from an outermost of an image to be projected by the projection apparatus cluster;

(b-2) disposing a plurality of control points within the image to be projected and subdividing an area into a plurality of sub-image areas by connecting the plurality of control points; and

(b-3) matching the initial points and the control points with a shape of a projection area of the curved surface.
The method of claim 7, wherein the step (b-3) is performed using a bilinear interpolation scheme.
The method of claim 7, wherein the initial point is an outermost vertex of the image to be projected.
The method of claim 7, wherein the plurality of control points is disposed at equal intervals.
A system for projecting an image onto a curved surface, comprising:

a projection management apparatus configured to compute relative transform information between a plurality of projection apparatuses included in a theater, generate a projection apparatus cluster by grouping the plurality of projection apparatuses based on the relative transform information, and match an image to be projected by the projection apparatus cluster with a projection area within the theater; and

the plurality of projection apparatuses configured to project images onto the projection area with the theater.
The system of claim 11, wherein the projection management apparatus sets any one of the plurality of projection apparatuses as a reference projection apparatus and computes relative transform information between projection apparatuses adjacent to the reference projection apparatus.
The system of claim 11, wherein the projection management apparatus extracts a correspondence point Ci from a projection area of the reference projection apparatus within an overlap area in which the projection area of the reference projection apparatus and a projection area of a projection apparatus adjacent to the reference projection apparatus partially overlap, extracts a correspondence point Cj from the projection area of the projection apparatus adjacent to the reference projection apparatus, and computes the relative transform information by substituting a relative transform equation into the correspondence points.
The system of claim 13, wherein the relative transform equation is

wherein xⁱ: a pixel location of the reference projection apparatus, Cⁱ _k is a location of each vertex of a correspondence point mesh, w_k: weight, and A(xⁱ) is Global Affine Transformation.
The system of claim 11, wherein the projection management apparatus matches the image to be projected by the projection apparatus cluster with the projection area of a curved surface, extracts a specific number of initial points from an outermost of an image to be projected by the projection apparatus cluster, disposes a plurality of control points within the image to be projected and subdividing an area into a plurality of sub-image areas by connecting the plurality of control points, and matches the initial points and the control points with a shape of a projection area of the curved surface.