JP2023172183A - Projection video control apparatus, method, and program - Google Patents

Abstract

To provide a projection video control apparatus, method and program which enable high-precision alignment without depending on size of an opening of an invisible light marker or an optical sensor.SOLUTION: A projection object 10 includes: a plurality of illuminance sensors 102 for sensing illuminance of irradiation light; an optical mask 101 for blocking a part of the irradiation light incident on a light receiving unit of each illuminance sensor 102; and a sensor read unit 103 which reads sensor signals output from the illuminance sensors 102 and converts them into illuminance values. A video projection control unit 20 includes a first position identifying unit 201, a second position identifying unit 202, and a control unit 203, specifies, in stages, positions of the illuminance sensors 102 in a pixel coordinate system controlling a projection video, based on the illuminance values output by the sensor read unit 103, and perform various controls for keystone correction or dimming correction of a specific region on a video projected by a video projection unit 30 on the basis of the positions of the illuminance sensors 102. The video projection unit 30 projects corrected video data on a video screen for visualization.SELECTED DRAWING: Figure 1

Description

The present invention relates to a projection image control device, method, and program for projecting an image from a projector onto a predetermined region of a projection target with high precision.

In recent years, projectors have become smaller and more sophisticated, and their usage has expanded. In an environment where the projector faces the projection target (screen) and is fixedly installed, there is little need to automatically control the projected image. However, when at least one of the projector or the projection target moves, when you want to control the image according to the movement of a moving object within the projection area, or when you want to accurately project an image onto a predetermined area within the projection area. requires a system to control the projected images.

Patent Document 1 discloses that a light transmission hole and a light sensor are installed in the housing of the projector, and the angle of the optical axis with respect to the screen is estimated by detecting the reflected light from the screen with the light sensor, and keystone correction is automatically performed. A technique for performing this has been disclosed.

Patent Document 2 describes a technology that automatically detects the presenter in front of the screen by installing optical sensors at multiple locations on the screen, and protects the presenter's eyes by reducing the amount of light projected near the presenter's eyes. Disclosed.

Patent Document 3 discloses a technique for detecting a flat area suitable for a screen in a three-dimensional space using a three-dimensional sensor and automatically controlling a projection area.

Patent Document 4 discloses a technique in which a plurality of optical sensors are installed on a screen, and a projector projects an alignment pattern onto the optical sensors to accurately align a projected image at a predetermined position.

Patent Document 5 discloses a technique in which a plurality of reflective materials are installed on a screen, and the movement of the screen is detected by capturing an image of the reflected light with a camera, thereby causing a projected image to follow.

Patent Document 6 discloses a technique for installing a plurality of invisible light markers on a moving object, detecting the invisible light markers with a sensor to detect the position of the moving object, and causing a projected image to follow.

Patent Document 7 discloses a technology that accurately detects the position of an optical sensor installed to automatically adjust the display brightness of a projected image, and allows an evaluation image to enter the optical sensor with high precision. There is.

Japanese Patent Application Publication No. 2005-159426 Japanese Patent Application Publication No. 2013-25014 Japanese Patent Application Publication No. 2017-73705 Japanese Patent Application Publication No. 03-259188 Japanese Patent Application Publication No. 2019-102931 Japanese Patent Application Publication No. 2011-254411 Japanese Patent Application Publication No. 2009-175355

In any of the above-mentioned conventional techniques, it is difficult to project an image from a projector onto a predetermined region (projection target region) of a projection target object with high precision.

For example, the projection image control systems disclosed in Patent Documents 1 to 3 can automatically control the projected image (keystone correction, dimming of light in a specific area, control of projection onto a flat area), but they cannot control the projection image with high precision on the projection target area. It was not a technology that could be applied to image projection purposes.

The projection image control systems disclosed in Patent Documents 4 to 7 are all aimed at highly accurate image projection onto a projection target area or an optical sensor for evaluation, but the accuracy of projection control is not sufficient.

For example, in Patent Document 4, an optical sensor is used to align the projection target area on the screen, so the accuracy of alignment depends on the size of the aperture of the optical sensor. In this method, the accuracy of alignment is improved by making the aperture as small as possible within the range in which the optical pattern for alignment can be detected. However, as the aperture becomes smaller, the amount of transmitted light decreases, making it more difficult to detect the light pattern, so there is a limit to the reduction of the aperture (in Patent Document 4, the aperture size is assumed to be 5 mm square). The accuracy of alignment could not be made sufficiently high.

The same applies to the projection image control technologies disclosed in Patent Documents 5 and 6, in which reflective materials or invisible light markers are installed on the screen, and the reflected light is captured by a sensor such as a camera to align the projection target area. . However, the accuracy of alignment depends on the size of the reflective material and the invisible light marker, and the smaller the size, the more difficult it becomes to detect the reflective material or the invisible light marker itself, so the accuracy of alignment cannot be made sufficiently high.

The same applies to the projection image control technology disclosed in Patent Document 7, and since the position detection accuracy of the optical sensor depends on the size of the aperture of the optical sensor, the accuracy of alignment cannot be made sufficiently high.

An object of the present invention is to provide a projection image control device, method, and program that solve the above technical problems and enable highly accurate positioning without depending on the size of the invisible light marker or the aperture of the optical sensor. There is a particular thing.

In order to achieve the above object, the present invention provides a projection target with a plurality of illuminance sensors whose relative positions with respect to the projection target area are known, and the position of each illuminance sensor specified in the pixel coordinate system of the projected image. A projection image control device that controls a projection image based on the following features is characterized in that it has the following configuration.

(1) An optical mask that blocks the incident light to each illuminance sensor with a predetermined mask pattern, and the output of each illuminance sensor when a mask pattern image having the same pattern structure as the mask pattern is scanned on the projection target. and means for specifying the position of each illuminance sensor in the pixel coordinate system based on.

(2) The amount of light transmitted through the mask pattern increases as the amount of deviation from the projected mask pattern image decreases, and the rate of change in the amount of light transmitted relative to the change in the relative position of each pattern is The pattern structure is such that the amount of deviation is larger in a relatively small range than in a large range.

(3) The means for specifying the position of each illuminance sensor includes a first position specifying means for specifying the rough position of each illuminance sensor in the pixel coordinate system, and a first position specifying means for determining the rough position of each illuminance sensor in the projection target object. means for calculating a transformation matrix for converting to the position of each illuminance sensor specified in the above; means for transforming the mask pattern image using the transformation matrix; and second position specifying means for specifying the position of each illuminance sensor based on a change in the output of the illuminance sensor.

Note that the present invention can be realized not only as a projection image control device having such a characteristic configuration, but also as a projection image control method using such a characteristic configuration as a procedure, and furthermore, can be realized as a projection image control method having such a characteristic configuration as a procedure, and furthermore, can be realized as a projection image control method having such a characteristic configuration as a procedure. It can also be realized as a projection image control program.

According to the present invention, the following effects are achieved.

(1) Since the mask pattern image having the same pattern structure as the mask pattern of the optical mask that blocks the incident light to the illuminance sensor is scanned on the projection target, it depends on the size of the invisible light marker and the aperture of the optical sensor. The sensor output value of the illuminance sensor can be made to differ significantly between the scan position where each mask pattern matches and the scan position where the mask patterns do not match. Therefore, the position of the illuminance sensor in the pixel coordinate system can be accurately specified.

(2) The amount of light transmitted from the mask pattern of the optical mask increases as the amount of deviation from the projected mask pattern image decreases, and the rate of change in the amount of light transmitted relative to the change in the relative position of each pattern is Since each pattern has a pattern structure in which the amount of deviation is larger in a relatively small range than in a large range, the sensor output value of the illuminance sensor is specifically increased between the scanning position where each mask pattern matches and the scanning position where it does not match. can be made different. Therefore, the position of the illuminance sensor in the pixel coordinate system can be specified more accurately.

(3) A first position specifying means and a second position specifying means are provided, and the first position specifying means performs a general position search within a wide range, while the second position specifying means performs a general position search specified by the first position specifying means. Since a precise position search is performed only in the vicinity of the position, the position of each illuminance sensor can be precisely specified in a short time.
Moreover, in this embodiment, the second position specifying means projects a mask pattern image transformed by homography transformation so as to cancel out the distortion, so that the peak of the illuminance value indicates that the center coordinates of the optical mask match the true value. can be detected as .

FIG. 1 is a functional block diagram of a projection image control device according to an embodiment of the present invention. FIG. 3 is a diagram showing a relative positional relationship between a projection target region R1, a plurality of illuminance sensors, and their optical masks. It is a figure showing an example of a light mask. FIG. 3 is a diagram showing the relationship between the amount of shift of a projected mask pattern image with respect to the mask pattern of an optical mask and the amount of light transmitted. FIG. 3 is a functional block diagram of a sensor reading section. FIG. 2 is a diagram showing the relationship between a pixel coordinate system that controls a projected image and a projection target. 5 is a flowchart showing the operation of the video projection control section. FIG. 3 is a diagram showing an example of an image projected to roughly specify the position of an illuminance sensor. FIG. 7 is a diagram showing an example of a pattern structure image in which mask pattern images are arranged at positions corresponding to each illuminance sensor, which is projected in order to accurately specify the position of the illuminance sensor. FIG. 6 is a diagram illustrating a method for calculating a homography matrix that coordinates transforms the position of each optical mask on a projection target into each rough position specified in a pixel coordinate system.

Embodiments of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of the main parts of a projection image control device according to an embodiment of the present invention, and the main components include a projection target 10, an image projection control section 20, and an image projection section 30. do.

The projection target 10 includes a plurality of illuminance sensors 102 that detect the illuminance of irradiated light, a light mask 101 that blocks part of the irradiated light that enters the light receiving section of each illuminance sensor 102, and a sensor signal output by each illuminance sensor 102. It is equipped with a sensor reading unit 103 that reads and converts it into an illuminance value. Each illuminance sensor 102 is configured integrally with or separately from a video screen or the like that visualizes the projected image.

The image projection control unit 20 includes a first position specifying unit 201, a second position specifying unit 202, and a control unit 203, and controls the projected image based on the illuminance value output by the sensor reading unit 103. The position of each illuminance sensor 102 is identified step by step, and based on the position of each illuminance sensor 102, the image projected by the image projection unit 30 on the projection target 10 is subjected to geometric transformation such as keystone correction and dimming correction of a specific area. Performs various controls such as

Such an image projection control unit 20 is configured by implementing an application (program) that realizes each function described in detail below on a general-purpose computer or server equipped with a CPU, ROM, RAM, bus, interface, etc. can. Alternatively, it can be configured as a dedicated machine or single-function machine in which part of the application is converted into hardware or software.

The video projection unit 30 projects the corrected video data onto a video screen to visualize it. As the image projection unit 30, any general-purpose projector device such as DLP (Digital Light Processing), liquid crystal, laser, etc. can be used.

FIG. 2 is a diagram showing the relative positional relationship between the projection target region R1 on the projection target object 10 onto which the video for viewing is irradiated and the plurality of illuminance sensors 102. As shown in FIG. Note that in this embodiment, the optical mask 101 is provided at the light receiving section of the illuminance sensor 102, and the position of each optical mask 101 viewed from the image projection section 30 is the position of the corresponding illuminance sensor 102. In some cases, the position of the mask 101 is considered to be the position of each illuminance sensor 102 in the explanation.

In this embodiment, four illuminance sensors 102 and their optical masks 101 are placed in a gap area outside the projection target area R1 within the physical projectable area on the video screen, relative to the projection target area R1. The relationship is fixed in a known position.

Each illuminance sensor 102 detects the light transmitted through the optical mask 101 and outputs a sensor signal such as a current value according to the intensity to the sensor reading unit 103. Each illuminance sensor 102 is generally implemented using an arbitrary device called a photodiode or phototransistor that converts light into a sensor signal and outputs it, or a device such as a photoresistor whose electrical resistance decreases depending on the intensity of light. Is possible.

In this embodiment, when a mask pattern image L having the same pattern structure as that of the mask pattern is projected onto the position of the optical mask 101, the amount of light transmitted specifically increases as the positional deviation of each pattern decreases. The pattern structure of each mask pattern is designed.

FIG. 3 is a diagram showing an example of a mask pattern of the optical mask 101 in this embodiment, where black portions represent mask regions that do not transmit light, and white portions represent non-masked regions that transmit light. In a region of each optical mask 101 corresponding to the light receiving surface of each illumination sensor 102, a plurality of mask regions and a plurality of non-mask regions are arranged in a predetermined pattern structure.

In this embodiment, the position of each illuminance sensor 102 in the pixel coordinate system is determined based on the output of each illuminance sensor 102 when a mask pattern image having the same pattern structure as the mask pattern of the optical mask 101 is scanned on the projection target. In order to accurately specify the mask pattern of the optical mask 101, the mask pattern of the optical mask 101 has a pattern structure in which the amount of light transmitted is sensitive to positional deviation from the mask pattern image.

More specifically, as will be described later in detail with reference to FIG. 4, the mask pattern of the optical mask 101 increases the amount of light transmitted as the amount of deviation from the projected mask pattern image decreases, and The pattern structure has a pattern structure in which the ratio of change in the amount of light transmission to a change in the relative position of each pattern is larger in a range where the amount of shift of each pattern is relatively small than in a range where it is large.

In FIG. 4A, a plurality of rectangular mask areas and non-mask areas are arranged in a houndstooth pattern. In other words, a plurality of non-masked regions are arranged discontinuously in the masked region corresponding to the light-receiving surface of the illuminance sensor 102.

In FIG. 4B, a plurality of rectangular mask areas and non-mask areas are irregularly arranged. In other words, the mask area corresponding to the light receiving surface of the illuminance sensor 102 is arranged so as to allow a plurality of non-mask areas to be interconnected and integrated, resulting in a plurality of complex non-mask areas. has been done.

In FIG. 4(c), a plurality of annular mask regions and non-mask regions of different sizes are alternately arranged concentrically. Note that the shape of each non-masked region is not limited to a rectangle or an annular shape, but can be any shape.

FIGS. 4(a) to 4(c) are diagrams in which the relationship between the amount of deviation of the mask pattern image and the amount of light transmitted for each mask pattern of each optical mask 101 is calculated and made into a three-dimensional graph, and FIG. 4(d) shows, for comparison, the relationship in the conventional optical mask employed in Patent Document 4, that is, a simple pattern structure in which one non-mask area is surrounded by one mask area.

If an optical mask 101 with a complex mask pattern and a pattern structure sensitive to positional deviation is used as in this embodiment, a pattern structure with a simple mask pattern and insensitive to positional deviation as shown in FIG. Compared to the case of using an optical mask with I understand. Ignoring the effects of environmental light, measurement noise, mismatch in angle between the projected image and the optical mask, etc., it is expected that the illuminance value will ideally be sensed as a two-dimensional graph with a similar shape.

Note that with a regular mask pattern as shown in FIG. 2(a), a plurality of local maximum values occur, and the peak position may be erroneously detected due to the influence of environmental light and measurement noise. On the other hand, if multiple masked areas and non-masked areas are arranged irregularly as shown in Figure (b), it is possible to reduce the illuminance value when the pattern image is projected at the wrong position. , enables highly accurate and robust position detection.

Such an irregular mask pattern is created by dividing the area corresponding to the light receiving part of the illuminance sensor 102 into two-dimensional grids, generating pseudo-random numbers such as M-sequence codes corresponding to the grid numbers, and generating pseudo-random numbers with a code of 1. It can be generated by making the grid a non-masked area.

Here, for example, an M-sequence code having the length of the number of columns of a two-dimensional grid may be generated and assigned to the first row, and codes obtained by shifting the code by several bits may be assigned to the second and subsequent grids. At this time, the rows and columns may be reversed. This makes it possible to provide M-sequence code properties not only in the row (column) direction but also in the column (row) direction.

In general, the autocorrelation of M-sequence codes reaches a peak value when the amount of deviation is 0, and is low at other times. By reducing the illuminance value when projected, more accurate and robust position detection becomes possible.

In addition, if a concentric mask structure is adopted as shown in (c) of the same figure, it is possible to reduce the illuminance value when the position is misaligned, as in the case of the irregular pattern shown in (b) of the same figure. Since the illuminance value is less affected by the angle mismatch between the projected image and the optical mask 101, position detection that is robust to the angle mismatch between the projected image and the optical mask 101 becomes possible.

Note that the size of the area in which the non-masked area is arranged in the optical mask 101 is preferably about the same size as the light receiving section of the illuminance sensor 102 (for example, 5 mm square), or smaller than that. By doing so, the illuminance sensor 102 can detect all of the light that has passed through the optical mask 101, making it possible to detect the peak of the illuminance value when the most amount of light has passed through the optical mask.

The sensor reading unit 103 reads the sensor signals output by each illuminance sensor 102, converts it into an illuminance value, and outputs it to the image projection control unit 20. FIG. 5 is a functional block diagram of the sensor reading unit 103, and here, the case where the illuminance sensor 102 is a photodiode will be explained as an example.

The sensor reading unit 103 converts analog sensor signals such as current values at time t output by each illuminance sensor 102 into digital signals, converts them to corresponding illuminance values, performs error correction, etc., and then reads each illuminance. The illuminance values of the sensor 102 are sequentially outputted to the image projection control unit 20 as L _1,t , L _2,t . . . L _K,t .

The image projection control section 20 mainly includes the first position specifying section 201, the second position specifying section 202, and the control section 203, and has the illuminance value L ₁ of each illuminance sensor 102 at time t acquired from the sensor reading section 103. _,t , L _2,t ...L _K,t, the position of each optical mask 101 is identified step by step, and the position represents the position of each illumination sensor 102. Furthermore, the projection target area R1 is specified based on the position of each specified illuminance sensor 102, and the image projected by the subsequent image projection unit 30 and the projection target area R1 are controlled.

As shown in FIG. 6, the projection target area R1 of the projection target object 10 is a rectangular area, and K pieces (K≧3: 4 pieces in this embodiment) of illumination sensors 102 are arranged near the outer periphery of the rectangular area, and the optical mask 101 are fixedly arranged on the same plane as the projection target region R1, define the center coordinates P _M,1 , P _M,2 , P _M,3 , P _M,4 of each optical mask 101 and the projection target region R1. The coordinates P _T,1 , P _T,2 , P _T,3 , and P _T,4 of the four corners can be defined in the pixel coordinate system (u, v) of the image projected by the image projection unit 30.

Then, by using a homography matrix H _MT that coordinates transforms the center coordinates P _M,1 to P _M,4 of each optical mask 101 to the coordinates P _T,1 to P _T,4 of the four corners of the projection target area R1. , it becomes possible to specify the projection target region R1 from the center coordinates of each optical mask 101. Note that since the optical mask 101 and the projection target region R1 have a known positional relationship, it is possible to use a homography matrix _HMT calculated in advance. That is, by specifying the center coordinates P _M,1 , P _M,2 , P _M,3 , P _M,4 of each optical mask 101, the projection target region R1 can be specified.

Therefore, in this embodiment, the position of each light mask 101 is assumed to represent the position of each illuminance sensor 102, the position of each light mask 101 in the pixel coordinate system is specified, and the projection is performed based on the position of each light mask 101. Identify the position of the target region R1 in the pixel coordinate system.

Next, the operation of the video projection control section 20 will be explained in detail with reference to the flowchart in FIG. In step S1, the first position specifying unit 201 _, as _shown in an example in FIG. An image is projected by scanning the projection target 10 with a spot light in which only a circular area with a radius of is white (maximum brightness). Then, the coordinates of each scanning position of the spot light at which the illuminance values L _1,t to L _4,t of each illuminance sensor 102 are respectively the maximum values are determined as the rough position P ^' in the pixel coordinate system of each optical mask 101. Specify as _M,1 ~P ^' _M,4 .

Alternatively, as a method that can be expected to be executed in a shorter time, a method may be adopted in which a white straight line with a predetermined line width is scanned in the u direction and the v direction, instead of a circular area. Other methods include dividing the image projection area into grids, assigning an ID to each grid, and identifying the position of each light mask by switching between black and white image projection in a specific pattern according to the ID, and distance measurement in general. It is also possible to adopt a method of projecting a plurality of phase-shifted sine wave patterns using a technique called a phase shift method, which is used in E.g., and estimating the phase of the sine wave with sub-pixel accuracy from the illuminance value.

The position specifying process of the optical mask 101 by the first specifying unit 201 can specify the position of the illuminance sensor with a certain degree of accuracy even when performed alone. However, in both methods, the change in illuminance value when the projected image is slightly shifted from the true value at the center of the mask is similarly small, and it is difficult to accurately identify the position of the optical mask robustly to measurement noise. Have difficulty. Therefore, in this embodiment, the second position specifying unit 202 performs more precise position specifying based on the rough position P ^' of each optical mask 101.

As an example is shown in FIG. 9, the second position specifying unit 202 provides a mask having the same pattern structure as each optical mask 101 at a position corresponding to the optical mask 101 provided in the light receiving part of each illuminance sensor 102 on the projection target 10. A pattern structure image is held in which the pattern image CP is placed and the entire remaining area is black. In this embodiment, a 3×3 houndstooth pattern is assumed as the optical mask 101, so each mask pattern image CP also has a 3×3 houndstooth pattern. The relative position of each mask pattern image CP1 to CP4 in the pixel coordinate system is equal to the relative position of each optical mask 101 on the projection target 10.

FIG. 10 calculates a homography matrix that coordinates transforms the position of each optical mask 101 on the projection object 10, represented by the positions of each mask pattern image CP1 to CP4, to each rough position P ^' specified in the pixel coordinate system. FIG.

In step S2, as shown in FIG. 10, the center coordinates p _M,1 , p _M,2 , p _M,3 , p _M,4 of each mask pattern image CP in the pattern structure image and the The center of each mask pattern image CP is determined based on the point correspondence with the rough positions ^P'M _,1 , ^P'M _,2 , ^P'M _,3 , ^P'M _,4 of each optical mask 101 identified by 201. A homography matrix H pM → _{P M} that transforms the coordinates p _M,1 to p _M,4 to the rough positions P ^′ _M,1 to P ^′ _M,4 of each corresponding optical mask 101 is calculated.

In step S3, the control unit 203 applies the transformation matrix H _pM →P _M to the pattern structure image to generate a transformed pattern structure image subjected to homography transformation (planar projection transformation). In the converted pattern structure image, the pattern structure of each mask pattern image CP has also been transformed into a shape corresponding to its position by the homography transformation H _pM → _PM .

In step S4, the second position specifying unit 202 determines the area of each mask pattern image CP in the converted pattern structure image with the corresponding rough positions P ^′ _M,1 to P ^′ _M,4 as centers. The illuminance value of each illuminance sensor 102 is acquired while shifting (scanning) within a predetermined rectangular or circular area. Then, the coordinates where the illuminance values L _1,t to L _4,t of each illuminance sensor reach their maximum values are specified as precise positions P _M,1 to P _M,4 in the pixel coordinate system of each light mask 101. .

The second position specifying unit 202 can determine the positions P _M,1 to P _M,2 of each optical mask 101 with sub-pixel precision by shifting the mask pattern image CP with sub-pixel precision. Alternatively, the illuminance value is up-sampled by applying parabolic fitting, bicubic method, etc. using the illuminance value around the maximum value of the illuminance value, and the positions P _M,1 to P _M,2 of each light mask 101 are sub-pixel. You can also find it by accuracy.

In addition, as a method for calculating the homography matrix from point correspondences, the initial values are estimated using the DLT (Direct Linear Transform) method using at least four sets of point correspondences, and then optimized using the Gauss-Newton method, Levenberg-Marquardt method, etc. There are generally known methods for doing this. When using this method, it is necessary to use at least four optical masks 101 and detect their center coordinates.

Another method for calculating the homography matrix from point correspondences is to estimate the external parameters of the projector using the PnP (Perspective-n-Point) method using three or four sets of point correspondences, and perform calibration in advance. There is also a known method of calculating a homography matrix by multiplying it by the acquired internal parameters of the projector. When using this method, it is necessary to use at least three optical masks and detect their central coordinates. In the present invention, the means for calculating the homography matrix is not particularly limited, and any method can be used.

In step S5, the control unit 203 uses the homography matrix HMT to set the specific positions P _M,1 to P _M,4 of each optical mask 101 in the pixel coordinate system to the coordinates P _T,1 of the four corners of the projection target area R1. The projection target region R1 of the projection target object 10 is specified by coordinate transformation to ~P _T,4 . If the projection target region R1 can be specified, for example, a homography matrix H _PC →P that transforms the vertex coordinates P _C,1 to P _C,4 of arbitrary four corners of the projection target object 10 to P _T,1 to P _T,4 By calculating and applying _M , it becomes possible to control and project an image to be projected onto the entire projection area so that it fits within the projection target area R1.

Note that the transformation matrix homography matrix H _pM →P _M is recalculated by regarding each specific position P _M,1 to P _M,4 as a rough position, and the recalculated transformation matrix is used to transform the pattern. The second position specifying unit 202 repeatedly deforms the structural image and specifies the position of each optical mask 101 anew using the newly deformed mask pattern image CP until a predetermined convergence condition is satisfied. Also good.

The control unit 203 controls the projected image using each of the specific positions P _M,1 to P _M,4 at the time when the convergence condition is satisfied. The image projection section 30 projects the image controlled by the image projection control section 20 onto the projection target 10.

According to this embodiment, since a mask pattern image having the same pattern structure as the mask pattern of the optical mask 101 that blocks the incident light to the illuminance sensor 102 is scanned on the projection target, the scanning where each mask pattern coincides with the mask pattern image is performed. The sensor output value of the illuminance sensor can be made to differ significantly between the scanning positions and the scanning positions that do not match the scanning position. Therefore, the position of the illuminance sensor 102 in the pixel coordinate system can be accurately specified.

Furthermore, according to the present embodiment, the mask pattern of the optical mask 101 has a structure in which the amount of light transmitted increases as the amount of deviation from the projected mask pattern image decreases, and the amount of light transmitted increases as the amount of deviation from the projected mask pattern image decreases. Since the pattern structure has a pattern structure in which the rate of change in amount is larger in a range where the amount of deviation of each pattern is relatively small than in a large range, the illuminance sensor sensor Output values can be made to vary significantly. Therefore, the position of each optical mask 101 in the pixel coordinate system can be specified more accurately.

Further, according to the present embodiment, a first position specifying unit 201 and a second position specifying unit 202 are provided, and while the first position specifying unit 201 performs a rough position search within a wide range, the second position specifying unit 202 performs a rough position search within a wide range. Since a precise position search is performed only in the vicinity of the rough position specified by the first position specifying unit 201, the position of each optical mask 101 in the pixel coordinate system can be precisely specified in a short time.

Furthermore, simply by projecting a mask pattern image having the same pattern structure as the light mask 101, the peak of the illuminance value cannot be accurately calculated due to the angle mismatch between the projected image and the light mask 101. Since a mask pattern image transformed by homography transformation is projected so as to cancel the distortion, it becomes possible to detect the coincidence of the center coordinates of the optical mask 101 with the true value as the peak of the illuminance value.

In the above embodiment, the position of each illuminance sensor 102 is estimated by homographically converting the mask pattern image and projecting it, but the present invention is not limited to this. Instead, by physically controlling (moving) the position, direction, and angle at which the image is projected, the mask pattern image can be deformed to match the optical mask 101, or controlled so that it fits within the projection target area R1. Also good.

Furthermore, in this embodiment, one optical mask 101 covers the light receiving part of one illuminance sensor 102, but the present invention is not limited to this, and multiple illuminance sensors 102 may be arranged close together. , all the light receiving parts of the plurality of illuminance sensors 102 may be covered with one large optical mask 101, and the light transmitted through the optical mask 101 may be detected without omission by the light receiving parts of the plurality of illuminance sensors 102. In this case, by summing up the illuminance values of the plurality of illuminance sensors 102 corresponding to one optical mask 101, the sensor reading unit 103 can treat it as if one large illuminance sensor 102 were used.

Furthermore, in the above embodiment, the mask patterns of all the optical masks 101 are the same, the first position specifying unit 201 projects an image scanning a spot light without a pattern structure, and only the second position specifying unit 201 projects an image scanning a spot light without a pattern structure. The description has been made assuming that an image having the same pattern structure as a mask pattern is scanned.

However, the present invention is not limited to this, and if a large number (for example, 8) of illuminance sensors 102 can be provided, four of them are provided with optical masks 101a that are insensitive to pattern displacement. Alternatively, a light mask 101b sensitive to pattern displacement may be applied to the remaining four.

Generally, the mask area ratio is higher than the non-mask area ratio, and the mask becomes more sensitive to pattern misalignment, so in the optical mask 101a, the mask area ratio is lower than the non-mask area ratio, and the optical mask In 101b, the ratio of masked areas is made higher than the ratio of non-masked areas.

The first position specifying unit 201 targets the illuminance sensor 102 masked with the optical mask 101a, and performs rough position identification that prioritizes speed over accuracy by scanning a mask pattern image having the same pattern structure as the optical mask 101a. I do. The second position specifying unit 202 estimates the rough position of the light mask 101b based on the rough position of the light mask 101a, and targets the illuminance sensor 102 masked with the light mask 101b. By scanning a mask pattern image with a pattern structure, precise positioning is performed with priority given to accuracy.

According to the embodiment described above, it is possible to project images with high viewing quality, so Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations: It will be possible to contribute to "promoting industrialization."

10... Projection object, 20... Image projection control section, 30... Image projection section, 101... Optical mask, 102... Illuminance sensor, 103... Sensor reading section, 201... First position specifying section, 202... Second position specifying section , 203...control unit

Claims (17)

In a projection image control device that provides a projection object with a plurality of illuminance sensors whose relative positions with respect to a projection target area are known, and controls a projected image based on the position of each illuminance sensor specified in a pixel coordinate system of the projection image,
a light mask that blocks incident light to each illuminance sensor with a predetermined mask pattern;
and means for specifying the position of each illuminance sensor in a pixel coordinate system based on the output of each illuminance sensor when a mask pattern image having the same pattern structure as the mask pattern is scanned on the projection target object. Characteristic projection image control device. The amount of light transmitted through the mask pattern increases as the amount of deviation from the projected mask pattern image decreases, and the ratio of the change in the amount of light transmitted to the change in the relative position of each pattern is determined by the amount of deviation of each pattern. 2. The projection image control device according to claim 1, wherein the projection image control device has a pattern structure in which the area is larger in a relatively small range than in a large area. The means for specifying the position of each illuminance sensor includes:
a first position specifying means for specifying the rough position of each illuminance sensor in a pixel coordinate system;
means for calculating a conversion matrix for converting the relative position of each illuminance sensor on the projection object into the roughly specified position of each illuminance sensor;
means for transforming the mask pattern image with the transformation matrix;
2. A second position specifying means for specifying the position of each illuminance sensor based on a change in the output of each illuminance sensor when the deformed mask pattern image is scanned on the projection target object. 2. The projection image control device according to 2. recalculating the transformation matrix by regarding the position of each illuminance sensor identified by the second position specifying means as a rough position;
Transforming the mask pattern image again using the newly calculated transformation matrix,
The second position specifying means specifies the position of each illuminance sensor anew based on the output change of each illuminance sensor when scanning the newly transformed mask pattern image on the projection target, if a predetermined condition is met. Repeat until satisfied,
4. The projected image control device according to claim 3, wherein the means for controlling the projected image controls the projected image based on the newly specified position of each illuminance sensor. 3. The first position specifying means specifies the approximate position of each illuminance sensor based on a change in the output of each illuminance sensor when an image in which a spot light is scanned on the projection target object is projected. The projection image control device described in . The means for transforming the mask pattern image is configured to transform a pattern structure image in which the same number of mask pattern images as illuminance sensors are arranged at positions corresponding to each illuminance sensor, and the area other than the mask pattern image is black, using the transformation matrix. death,
4. The projected image according to claim 3, wherein the second position specifying means scans each mask pattern image portion of the transformed pattern structure image in the vicinity of the rough position of the corresponding illuminance sensor. Control device. The projection image control according to claim 1 or 2, wherein the mask pattern has a pattern structure in which a plurality of mutually separated non-mask areas are arranged in a mask area corresponding to a light receiving surface of the illuminance sensor. Device. The mask pattern has a pattern structure in which a plurality of non-mask regions are arranged in a mask region corresponding to a light-receiving surface of the illuminance sensor, allowing each non-mask region to be interconnected and integrated. The projection image control device according to claim 1 or 2. The pattern structure of the mask pattern includes a first pattern structure in which a plurality of mask regions and non-mask regions are arranged in a staggered pattern, a second pattern structure in which a plurality of mask regions and non-mask regions are arranged irregularly, and 3. The projection image control device according to claim 1, wherein the projection image control device has a third pattern structure in which a plurality of annular mask regions and non-mask regions of different sizes are alternately arranged concentrically. The second pattern structure is characterized in that the area for masking light is divided into two-dimensional grids, M-sequence codes corresponding to each grid number are generated, and non-masked areas are provided according to each code. The projection image control device according to claim 9. In the second pattern structure, an M-sequence code having the length of the number of columns or rows of the two-dimensional grid is generated and assigned to the first row or column, and is assigned to the second row or grid after the second row. 11. The projection image control apparatus according to claim 10, wherein a code obtained by shifting the code by a predetermined bit is assigned. The projection target object is provided with a plurality of illuminance sensors whose relative positions to the projection target area are known and a light mask that blocks incident light to the illuminance sensors, and each illuminance sensor is identified by the computer in the pixel coordinate system of the projected image. In a projected image control method for controlling a projected image based on the position of
The position of each illuminance sensor in the pixel coordinate system is specified based on the output of each illuminance sensor when a mask pattern image having the same pattern structure as the mask pattern of the optical mask is scanned on the projection target. Projection image control method. The amount of light transmitted through the mask pattern increases as the amount of deviation from the projected mask pattern image decreases, and the ratio of the change in the amount of light transmitted to the change in the relative position of each pattern is determined by the amount of deviation of each pattern. 13. The projection image control method according to claim 12, wherein the pattern structure is such that the pattern is larger in a relatively small range than in a large range. When specifying the position of each illuminance sensor in the pixel coordinate system,
Identify the rough position of each illuminance sensor in the pixel coordinate system,
Calculating a conversion matrix that converts the relative position of each illuminance sensor on the projection target into the roughly specified position of each illuminance sensor,
transforming the mask pattern image with the transformation matrix;
The projection image control method according to claim 12 or 13, characterized in that the position of each illuminance sensor is specified based on a change in the output of each illuminance sensor when the deformed mask pattern image is scanned on the projection target object. . A plurality of illuminance sensors whose relative positions with respect to the projection target area are known and a light mask that blocks incident light to the illuminance sensors are provided on the projection target object, and each illumination sensor is positioned at the position specified by the pixel coordinate system of the projected image. In a projection image control program that controls a projection image based on
The position of each illuminance sensor in the pixel coordinate system is specified based on the output of each illuminance sensor when a mask pattern image having a mask pattern having the same pattern structure as the mask pattern of the optical mask is scanned on the projection target. A projection image control program characterized by causing a computer to perform certain functions. The amount of light transmitted through the mask pattern increases as the amount of deviation from the projected mask pattern image decreases, and the ratio of the change in the amount of light transmitted to the change in the relative position of each pattern is determined by the amount of deviation of each pattern. 16. The projection image control program according to claim 15, wherein the projection image control program has a pattern structure that is larger in a relatively small range than in a large range. When specifying the position of each illuminance sensor in the pixel coordinate system,
Identify the rough position of each illuminance sensor in the pixel coordinate system,
Calculating a conversion matrix that converts the relative position of each illuminance sensor on the projection target into the roughly specified position of each illuminance sensor,
transforming the mask pattern image with the transformation matrix;
The projection video control program according to claim 15 or 16, characterized in that the position of each illuminance sensor is specified based on a change in the output of each illuminance sensor when the deformed mask pattern image is scanned on the projection target object. .

Images