JP2015100084A - Space projection device, space projection method, and space projection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a space projection device that can geometrically and optically correct a predetermined image and project the corrected image accurately on a subject, in the general situation where one pixel of a projector screen does not necessarily correspond to one pixel of a camera screen.SOLUTION: A space projection device includes: an LT acquisition unit that projects, on a subject, a pixel pattern in which a rectangular area having m pixels in the horizontal direction and n pixels in the vertical direction is white, and a pixel area other than the rectangular area is black, using as an image for a point light source, to acquire light transport data; an LT adjustment unit that calculates a brightness distribution ratio for adjusting the brightness of projector pixels on the basis of the light transport data; and a projection image creation unit that geometrically and optically corrects a projection object image by using geometrical correspondence information on the projector pixels and camera pixels, the brightness distribution ratio, and brightness compensation parameters calculated from a reference color sample image, so as to create an image projecting the projection object image on the subject.

Description

  The present invention relates to a spatial projection apparatus, a spatial projection method, and a spatial projection program that project an image according to spatial information using a video projection apparatus.

  The projector, which is a video projection device, is an extremely flexible image output device that provides free-size video display for personal spaces, conference halls, huge public viewing, building walls, etc. as long as it is within the projection specifications. be able to. When projecting onto an arbitrary structure (hereinafter referred to as a subject) that is not a flat screen using a projector, the shape of the subject is measured in advance, and a predetermined image is projected according to the shape so that there is no spatial distortion. The technique is known as projection mapping. Taking advantage of the arbitrary scale display, projection mapping is used for entertainment-oriented events and advertisement display by illumination.

  In projection mapping, a projector-camera system is used in order to accurately project a predetermined image onto a subject. In the projector / camera system, if an image output from the projector is controlled to be geometrically transformed in accordance with an image input from the camera, a predetermined image can be projected at a designated position. In general, when projecting and mapping to a flat screen, the plane-homography obtained in the previous calibration work is used, and the points on the projector screen and the points on the camera screen are passed through the screen surface. Associate. Using this correspondence, the predetermined image output from the projector is geometrically transformed, and the predetermined image is projected onto a predetermined place on the screen surface.

  On the other hand, when the spatial structure is uneven or bent, Non-Patent Document 1 uses laser measurement to obtain the spatial structure or uneven shape of the subject. The predetermined image is geometrically deformed in accordance with the shape, and the predetermined image is projected to the designated place in the same state as the flat screen. As a similar approach, a method of projecting a geometric pattern called structured light from a projector and measuring the concavo-convex shape or depth of the spatial structure from a distortion image observed by the camera can be used. As described above, according to the conventional technology, the spatial structure of the outside world or the subject is grasped by using a laser measurement or a projector / camera system, and a predetermined image is projected to a designated location according to the uneven shape or depth.

On the other hand, Non-Patent Document 2 discloses projection mapping using light transport without measuring the spatial structure of the outside world. Light transport is data representing the correspondence between all projector pixels and all camera pixels. FIG. 7 is a diagram illustrating measurement of a light transport matrix using an uncalibrated projector / camera system. As shown in FIG. 7, each projector pixel is turned on one by one in order, and the light transport is obtained by detecting a camera pixel in which light from the pixel is connected by optical reflection or refraction. When the number of pixels on the projector screen is p × q pixels and the number of pixels on the camera screen is u × v pixels, a matrix T in which the image C _j of the camera response of each point light source is converted into a column vector is arranged as a light transport. It is called a matrix and is a huge and sparse matrix of uv × pq pixels.

Non-Patent Document 2, by using the transposed ^{T T} of the matrix, a method of generating a virtual image (dual photography) artificially that would be observed from the point of view of the projector is described.

  In general, three-dimensional shape measurement using a laser or structured light is premised on receiving only primary reflected light. For this reason, the optical multiple reflection and refraction caused by the structure of the subject cannot be handled. On the other hand, since the light transport can handle primary reflected light, multiple reflected light, and refracted light without distinction, a highly accurate projection based on the correspondence between all projector pixels and all camera pixels. Enable mapping.

  Note that there is a technique described in Non-Patent Document 3 as a brightness compensation technique for a projector / camera system.

J. Shimamura and K. Arakawa: "Location-Aware Projection with Robust 3-D Viewing Point Detection and Fast Image Deformation", Proc. ACM International Conference on Multimedia, pp.296.299, 2004. P. Sen, B. Chen, G. Garg, SR Marschner, M. Horowitz, M. Levoy, and HPA Lensch: "Dual Photography", ACM Transactions on Graphics, vol.23, no.3, pp.745.755, 2005 . K. Fujii, MD Grossberg, and SK Nayar: "A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments", Proc. Of IEEE Computer Vision and Pattern Recognition (CVPR), vol.1, pp.814.821, 2005 .

  In order to project and map a predetermined image to an arbitrary shape without relying on three-dimensional measurement with laser light or structured light, a pixel output from a projector (this is called a projector pixel) and a pixel observed by a camera (this is called It is necessary to associate all of the camera pixels. Here, this operation is referred to as acquisition of the light transport. FIG. 7 shows an outline of acquiring the light transport, and most simply, each pixel is used as a point light source. When repeatedly illuminating one by one on the projector screen of p × q pixels, pq images are observed with the camera, and the camera response pixel is detected in each image to acquire the light transport. To do.

  However, depending on the relationship between the position and orientation of the projector and camera, it may be insufficient to use one pixel on the projector screen as a point light source. FIG. 8 is a diagram illustrating an example when the projector is far from the subject. When the illumination light emitted from one pixel on the projector screen advances and reflects off the subject, and when the reflected light is observed with several pixels on the camera screen, the illumination from one pixel on the projector screen is too weak to be observed with the camera There is. This situation depends on the optical sensitivity characteristics and function settings of the illuminating projector and the optical sensitivity characteristics and function settings of the observing camera. It is not common to use one point as a point light source.

  Further, in order to accurately project a predetermined image onto a subject, the pixel value of each pixel when observed with a camera needs to be the pixel value of the predetermined image. According to the method of Non-Patent Document 3, the output luminance from the projector can be optically compensated so that the predetermined luminance is observed with a camera. The method deals with luminance compensation on the surface of an object other than a planar object, but is limited to a case where projector coordinates and camera coordinates are one-to-one geometrically associated.

  Further, the light transport for an arbitrary shape includes secondary or higher-order reflected light and refracted light in addition to directly reflected light. Therefore, in order to geometrically and optically project and map a predetermined image on a subject, the image is output from the projector in consideration of the M to N spatial correspondence between the projector pixel and the camera pixel in the light transport. There is a problem that it is necessary to automatically adjust the brightness of the color.

  The present invention has been made in view of such circumstances, and in projection mapping utilizing the operation of the projector / camera system, one pixel on the projector screen is not always associated with one pixel on the camera screen. An object of the present invention is to provide a spatial projection device, a spatial projection method, and a spatial projection program that can correct a predetermined image geometrically and optically and accurately project it on a subject.

  In the present invention, a pixel pattern in which a rectangular area of m pixels in the horizontal direction, n pixels in the vertical direction is white, and the other pixel areas are black is used as a point light source image and projected onto a subject. An LT acquisition unit that acquires transport data, an LT adjustment unit that calculates a luminance distribution ratio for adjusting the luminance of the projector pixel based on the light transport data, and a geometric correspondence between the projector pixel and the camera pixel Using the information, the luminance distribution ratio, and the luminance compensation parameter calculated from the reference color sample image, the projection target image is geometrically and optically corrected, and the projection target image is projected onto the subject. And a projection image generation unit that generates an image.

  In the present invention, the projection image generation unit acquires an observation image of an image obtained by projecting the projection target image onto the subject, and obtains a luminance error between the observation image and the generated image that projects the projection target image. When the luminance error is larger than a predetermined threshold, the luminance to be fed back is calculated, and the luminance of the projector pixel is updated based on the calculated luminance.

  The present invention is a spatial projection method performed by a spatial projection apparatus including an LT acquisition unit, an LT adjustment unit, and a projection image generation unit, wherein the LT acquisition unit includes m pixels in the horizontal direction and n pixels in the vertical direction. An LT acquisition step for acquiring light transport data by projecting a subject on a subject using a pixel pattern in which the rectangular area of the pixel is white and the other pixel area is black as an image for a point light source; An LT adjustment step of calculating a luminance distribution ratio for adjusting the luminance of the projector pixel based on the light transport data, and the projection image generation unit includes geometric correspondence information between the projector pixel and the camera pixel; The image to be projected is corrected geometrically and optically using the luminance distribution ratio and a luminance compensation parameter calculated from a reference color sample image to And having a projection image generation step of generating an image to be projected a projection target image.

  The present invention is a spatial projection program for causing a computer to function as the spatial projection device.

  According to the present invention, the geometrical correspondence between the projector pixel and the camera pixel based on the light transport of the projector / camera system is projected onto the screen plane as to the spatial structure of the unevenness or the bent shape. A predetermined image can be projected as if it were. Furthermore, an effect that it becomes possible to realize augmented reality and video communication using the video presentation is obtained.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a flowchart for the processing operation of the LT acquisition unit 31 shown in FIG. It is a figure which shows the setting of a projector coordinate, and a pixel pattern. It is a figure which shows an example of write transport data. 3 is a flowchart showing a processing operation of an LT adjustment unit 32 shown in FIG. It is a flowchart which shows the processing operation of the projection image generation part 33 shown in FIG. It is a figure which shows the measurement of the light transport matrix using an uncalibrated projector camera system. It is a figure which shows the example when a projector is far from a to-be-photographed object.

  Hereinafter, a spatial projection apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a system configuration of the embodiment. In the embodiment shown in FIG. 1, a projector / camera system of one projector 1 and one camera 2 is used. This system includes a spatial projection device 3 and a video database (DB) 4. In FIG. 1, reference numeral 31 denotes an LT acquisition unit that acquires a light transport that is a geometric correspondence between a projector pixel and a camera pixel. In this specification, the write transport is abbreviated as “LT”. Reference numeral 32 denotes an LT adjustment for storing the light transport obtained in the LT acquisition unit 31 in a simple data format suitable for reading and calculating a luminance distribution ratio for adjusting the luminance of the projector pixel based on the data. Part. Reference numeral 33 designates a geometrical and optical correction of a predetermined image using a geometric correspondence between the projector pixel and the camera pixel, a luminance distribution ratio of the projector pixel, and a luminance compensation parameter calculated from a certain color sample image. A projection image generation unit that projects a predetermined image onto the subject 5. When the point light source image is input from the LT acquisition unit 31, the projector 1 projects the image onto the subject 5. The camera 2 transfers the observation image to the projection image generation unit 33 in order to acquire a predetermined image by feedback.

  In this configuration, the projector 1 and the camera 2 do not necessarily have to be connected as components, and it is only necessary to acquire data necessary for processing, and the LT acquisition unit 31, the LT adjustment unit 32, and the projection image generation unit 33 respectively. The data flow to the arrow may be in the form of using a recording medium such as a hard disk, a RAID device, a CD-ROM, or using a remote data resource via a network.

  Next, the processing operation of the LT acquisition unit 31 shown in FIG. 1 will be described. FIG. 2 is a flowchart showing the processing operation of the LT acquisition unit 31 shown in FIG. In this processing operation, the pixel pattern within the range of the designated projector pixel is blinked, the point light source is illuminated from the projector 1, the reflected light is measured by the camera 2, and the light transport is obtained. It is. Although other means for acquiring the write transport are conceivable, in the present embodiment, the write transport is acquired by the simplest true-force scanning (all-point scanning method within a predetermined range).

First, when processing is started, the LT acquisition unit 31 sets coordinate values (x _j , y _j ) as projector coordinates for lighting on the projector screen (step S1). Next, the LT acquisition unit 31 determines that the projector coordinates are within a predetermined range (determination of “lit pixel OK?” Shown in FIG. 2) (step S2). Since this determination does not use an excessive memory for acquiring LT, it does not target all projector screens, but also plays a role to target projector coordinates within a predetermined range. If the lighting pixel is OK, the LT acquisition unit 31 uses the projector coordinates (x _j , y _j ) as the coordinates of the lower left corner with respect to the projector screen as shown in FIG. 3, and m in the horizontal direction (m is a natural number). An image for a point light source is generated by setting a pixel pattern in which an area of m × m pixels of a rectangular shape of m pixels in the vertical direction is white and the other pixel areas are black (step S3). FIG. 3 is a diagram illustrating projector coordinate settings and pixel patterns. In the present embodiment, the area is not limited to m × m pixels, and may generally be an area of m × n pixels. The size of the pixel pattern is sufficient if the reflected light from each illumination from the projector 1 can normally observe the camera response, and the user can freely set the size of the pixel pattern.

  Next, after setting the pixel pattern, the LT acquisition unit 31 outputs the image (point light source image) to the projector 1 and projects the point light source image onto the subject 5 (step S4). Then, the LT acquisition unit 31 stands by until the image observation is OK (step S5), and after determining “image observation OK?”, The camera 2 captures the subject projected by the projector. If the image observation is OK, the LT acquisition unit 31 acquires an observation image through the camera 2 (step S6). On the other hand, before the point light source image is projected, the LT acquisition unit 31 projects a black image in which no pixels are lit from the projector 1 and acquires a background image.

Next, the LT acquisition unit 31 holds the background image for work, extracts the background image (step S7), and processes the background difference between the observation image and the background image (step S8). The LT acquisition unit 31 sets a predetermined threshold (step S9), and detects a pixel value exceeding the threshold in the background image (step S10). At this time, since the number of response pixels is not necessarily one point, the LT acquisition unit 31 records all the camera coordinates of pixels that exceed the threshold (step S11). The plurality of camera coordinates (u _k , v _k ) obtained in this way are the pixel pattern and light transport of the projector given as the point light source image. However, since the projector pixels are given in a rectangular m × m pixel pattern in the pixel pattern setting, it is not necessary to record all the pixels. Therefore, the representative value is used to represent the light transport of the projector pixels and the camera coordinates. It is sufficient to record the coordinates (xj, yj) of the projector pixel as the starting point. Therefore, in writing the projector coordinates, only the representative coordinate values (x _j , y _j ) of the projector pixels given by the point light source image are recorded (step S12).

Next, when the writing of the projector coordinates is completed, the LT acquisition unit 31 returns to the setting of the projector coordinates (step S1), shifts the pixels by m pixels in the horizontal direction at the next projector coordinates,
(X _j , y _j ) ← (x _j + m, y _j ) (1)
And set projector coordinates. Here, ← represents substitution.

Alternatively, if the horizontal direction exceeds the specified range, shift the pixel in the vertical direction by m pixels in the projector pixel,
(X _j , y _j ) ← (x _j , y _j + m) (2)
And set projector coordinates.

  The above process is repeated to process the coordinates of all projector pixels. Subsequently, after obtaining all the light transports of the projector pixels within the designated range, that is, after determining that there are no lighting pixels (NO in step S2), the LT acquisition unit 31 performs camera pixel in the vertical direction. The data obtained by arranging the coordinates of the projector pixels connected in the horizontal direction in the horizontal direction is temporarily secured as the light transport data (step S13). An example of the write transport data secured here is shown in FIG. FIG. 4 is a diagram illustrating an example of write transport data. Then, the LT acquisition unit 31 stops the process when the secured write transport data is transferred to the LT adjustment unit 32 (step S14), and ends this process.

  Next, the processing operation of the LT adjustment unit 32 shown in FIG. 1 will be described. FIG. 5 is a flowchart showing the processing operation of the LT adjustment unit 32 shown in FIG. This processing operation calculates a luminance distribution ratio for adjusting the luminance of the projector pixel value for projecting a predetermined image on the subject 5 based on the light transport data transferred from the LT acquisition unit 31. is there.

  First, when the process is started, the LT adjustment unit 32 sets all the voting tables of projector coordinates related to the designated range processed by the LT acquisition unit 31 to 0 (step S21). As an example of the voting table, projector coordinates related to the designated range processed by the LT acquisition unit 31 are secured in a matrix format.

Next, after initialization, the LT adjustment unit 32 loads the write transport data (step S22). Since the write transport data has the data format shown in FIG. 4 in the vertical direction, the LT adjustment unit 32 reads out the leftmost camera coordinates (u _k , v _k ) in the vertical direction (step S23). ) All projector coordinates (x _j , y _j ) recorded in the horizontal direction are taken out (step S24). At this time, since the projector coordinates are representative coordinates of m × m pixels, the projector coordinates (x _j , y _j ) to the projector coordinates (x _j + m−1, y _j + m−1) in the m × m pixel area. One vote is cast on the voting table of projector coordinates in the range of projector coordinates (x _j + Δx, y _j + Δy), 0 ≦ Δx ≦ m−1, 0 ≦ Δy ≦ m−1 (step S25). Voting is performed for all projector coordinates associated with the camera coordinates in the light transport data targeted for this (step S26).

When the vote result of each projector coordinate (x _j , y _j ) is N _xy , the projection image generation unit 33 sets the luminance distribution ratio that the projector coordinate contributes to image projection to 1 / N _xy (step S27). This luminance distribution ratio is used in the projection image generation unit. For example, when the luminance distribution ratio is 1/3, it means that the pixel value of the coordinate is distributed to three camera coordinates. If the vote result is N _xy = 0, it is determined that the projector coordinates do not contribute to the image projection, and the projection image generation unit does not process it. After the luminance distribution ratio is determined, the light / transport data and the voting table are transferred to the projection image generation unit 33, and this processing is stopped.

  Next, the processing operation of the projection image generation unit 33 shown in FIG. 1 will be described. FIG. 6 is a flowchart showing the processing operation of the projection image generation unit 33 shown in FIG. This processing operation is to project each image extracted from the video DB 4 onto the subject 5 using the light transport data obtained by the LT acquisition unit 31 and the luminance distribution ratio obtained by the LT adjustment unit 32.

In the present embodiment, the brightness correction method of the projector / camera system disclosed in Non-Patent Document 3 is used in the process of compensating the pixel value of each pixel when observed by the camera 2 to the pixel value of the predetermined image. To do. In the conventional luminance compensation method, an ambient light vector including a color mixing matrix of each camera pixel and a black offset light from the projector is used. Black offset light refers to weak projection light emitted when a black image is projected from a projector. Also in this embodiment, the color mixing matrix of each camera coordinate (u, v) is obtained from four sample images (images used in Non-Patent Document 3) captured in advance by means known in Non-Patent Document 3. V _uv and ambient light vector F _uv are calculated in _advance . For example, as a setup in advance, a plane-shaped subject is placed in front of the camera 2, a sample image is projected onto the subject, and a color mixing matrix and an ambient light vector are calculated by a method known in Non-Patent Document 3. To do. If a sample image is projected onto an arbitrary subject, an abnormal pixel value may be obtained, so that it is possible to prevent occurrence of a pixel for which a color mixing matrix and an ambient light vector cannot be obtained.

When the projector outputs the luminance linearly with respect to the input luminance, the camera luminance C _uv and the projector luminance P _xy are
C _uv = V _uv P _xy + F _uv (3)
(See Non-Patent Document 3). If the subject is a plane and the projector coordinates and the camera coordinates are geometrically associated with each other on a one-to-one basis, the brightness value P _xy of the projector coordinates associated with the camera coordinates and the light transport is expressed by the equation Using the relational expression (3), P _xy = V _uv ⁻¹ (C _uv −F _uv ) (4)
Can compensate the luminance.

_Assuming that the color mixing matrix V _uv and the ambient light vector F _uv for each camera coordinate (u, v) are obtained in advance according to Non-Patent Document 3, when the processing is started, the projection image generation unit 33 The color mixing matrix V _uv and the ambient light vector F _uv (compensation parameter) are read out (step S31). Next, the projection image generation unit 33 reads a predetermined image from the video DB 4 (step S32), and sequentially processes the following for each coordinate of the image.

により算出する（ステップＳ３７）。 First, camera coordinates (u, v) are set (step S33), the light transport data is referred to (step S34), and they are geometrically linked by the relationship between the camera coordinates (u, v) and the light transport. M _uv projector coordinates (x, y) are extracted (step S35). In parallel with this, the luminance distribution ratio 1 / N _xy of the projector coordinates (x, y) obtained by the LT adjustment unit 32 is set (step S36). When the pixel value of the camera coordinates (u, v) is C _k , the projector pixel P _xy is

(Step S37).

If the voting result of the projector coordinates (x, y) is N _xy = 0, P _xy = 0 is set in Equation (5). According to Expression (5), it is possible to compensate the luminance of each projector on the assumption that a plurality of projector coordinates and a plurality of camera coordinates are geometrically associated by the light transport. When the above processing is completed for all camera coordinates, the compensated image is projected from the projector 1 onto the subject 5 (step S38).

Next, the projected image is observed by the camera 2 (step S39), and the luminance error ΔCuv between the predetermined image and the observed image is determined as ΔC _uv (t) = C _uv −C _uv (t) (( 6)
(Step S40). In Expression (6), time t is used as an index representing a change in state, and C _uv (t) is a camera observation image at time t. Along with this notation, each projector brightness calculated by Expression (5) is assumed to be an initial state P _xy (0) at time t = 0. At this time, all luminance errors are
ΔC _uv ^T (t) ΔC _uv (t) <ε (7)
Is satisfied (step S41), and if it is within the predetermined allowable value ε, this process is stopped. ε is a parameter that can be freely set according to the user's request according to the system environment.

により算出する（ステップＳ４２）。続いて、現時点（時刻ｔ）でのプロジェクタ画素値がＰ_ｘｙ（ｔ）のとき、次の時刻ｔ＋１のプロジェクタ輝度Ｐ_ｘｙ（ｔ＋１）を
Ｐ_ｘｙ（ｔ＋１）＝Ｐ_ｘｙ（ｔ）＋ＧΔＰ_ｘｙ（ｔ） ・・・（９）
により更新する（ステップＳ４３）。式（９）では、ゲイン行列Ｇ：

を使う。κは０＜κ＜１の任意の値であり、本実施例ではκ＝０．５と設定する。この反復的な輝度補償を全ての輝度誤差が式（７）を満たすまで繰り返す。 If the luminance error ΔC _uv (t) is greater than or equal to the predetermined allowable value ε, the luminance amount to be fed back using the observation image at time t is used in order to adapt the luminance compensation to the surface of the subject having an arbitrary shape.

(Step S42). Subsequently, when the projector pixel value at the present time (time t) is P _xy (t), the projector brightness P _xy (t + 1) at the next time t + 1 is set to P _xy (t + 1) = P _xy (t) + GΔP _xy (t (9)
(Step S43). In equation (9), the gain matrix G:

use. κ is an arbitrary value of 0 <κ <1, and in this embodiment, κ = 0.5 is set. This iterative luminance compensation is repeated until all luminance errors satisfy Equation (7).

  By performing the above-described brightness compensation, the brightness can be flexibly adjusted according to each projector pixel projected on the surface of the arbitrary subject. Note that the processing of the projection image generation unit may be terminated without detecting the luminance error.

  By performing the processing described above, it is possible to project a high-quality image in which the projection luminance is compensated without the geometric distortion of the subject projection as in the case of image projection onto a flat screen.

  You may make it implement | achieve the space projector 3 in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  In the projection mapping using the operation of the projector / camera system, in a general situation where one pixel of the projector screen does not always correspond to one pixel of the camera screen, a predetermined image is corrected geometrically and optically, It can be applied to applications where it is essential to accurately project onto a subject.

  DESCRIPTION OF SYMBOLS 1 ... Projector, 2 ... Camera, 3 ... Spatial projection apparatus, 31 ... LT acquisition part, 32 ... LT adjustment part, 33 ... Projection space production | generation part, 4 ... Image | video Database (DB), 5 ... Subject

Claims (4)

Light and transport data is projected onto a subject using a pixel pattern in which a rectangular area of m pixels in the horizontal direction, n pixels in the vertical direction is white, and the other pixel areas are black as a point light source image. An LT acquisition unit to acquire;
An LT adjustment unit for calculating a luminance distribution ratio for adjusting the luminance of the projector pixel based on the light transport data;
Using the geometric correspondence information between the projector pixel and the camera pixel, the luminance distribution ratio, and the luminance compensation parameter calculated from the reference color sample image, geometrically and optically correct the projection target image, A spatial projection apparatus comprising: a projection image generation unit configured to generate an image for projecting the projection target image onto the subject.   The projection image generation unit acquires an observation image of an image obtained by projecting the projection target image onto the subject, obtains a luminance error between the observation image and an image on which the generated projection target image is projected, and the luminance error 2. The spatial projection device according to claim 1, wherein when the value is larger than a predetermined threshold, the brightness to be fed back is calculated, and the brightness of the projector pixel is updated based on the calculated brightness. A spatial projection method performed by a spatial projection device including an LT acquisition unit, an LT adjustment unit, and a projection image generation unit,
The LT acquisition unit projects a pixel pattern in which a rectangular area of m pixels in the horizontal direction and n pixels in the vertical direction is white and the other pixel area is black as a point light source image and projected onto the subject. LT acquisition step for acquiring write transport data;
An LT adjustment step in which the LT adjustment unit calculates a luminance distribution ratio for adjusting the luminance of the projector pixel based on the light transport data;
The projection image generation unit uses the geometric correspondence information between the projector pixel and the camera pixel, the luminance distribution ratio, and the luminance compensation parameter calculated from the reference color sample image to convert the projection target image geometrically and And a projection image generation step of generating an image for optically correcting and projecting the projection target image onto the subject.   A space projection program for causing a computer to function as the space projection device according to claim 1.

Images