JP6304135B2 - Pointer control device, projector and program - Google Patents

Description

  The present invention relates to a pointer control device, a projector, and a program.

2. Description of the Related Art A projection / operation system is known that operates a PC or the like by operating a pointer by designating a screen portion projected on a screen with a pointer device.

In such a projection / operation system, the user's operation may be hindered by the movement of the pointer that is not intended by hand movement. Patent Document 1 discloses a technique for assisting a user's operation of the projection / operation system by correcting camera shake.

In the technique of Patent Document 1, a correction parameter is determined by executing a predetermined calibration operation prior to executing camera shake correction. The correction parameter is applied to the entire projection image.

JP 2008-65522 A

In the technique described in Patent Document 1, in order to perform camera shake correction, it is necessary to execute calibration in advance to acquire parameters, and there is a problem that the operation is complicated.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a pointer control device, a projector, and a program capable of executing camera shake correction without requiring a complicated operation.

In order to achieve the above object, a pointer control device according to the present invention provides:
An instruction coordinate acquisition unit that acquires instruction coordinates indicating a position indicated by the pointer device on the projection surface or the projection image;
A comparison unit that compares the shape of the projection image displayed on the projection plane with the reference shape of the projection image;
A discriminant criterion for determining a discriminant criterion for discriminating whether or not the change in the indicated coordinate acquired by the indicated coordinate acquisition unit is due to camera shake based on the reference position on the projection surface or the projection image and the comparison result by the comparison unit A definition part;
It is characterized by providing.

According to the present invention, a pointer control device capable of performing camera shake correction without requiring a complicated operation,
A projector and a program can be provided.

It is a block diagram which shows the structure of the image projection system which concerns on Embodiment 1 of this invention. It is a figure which shows the example projected by the image projection system which concerns on Embodiment 1, and the example of pointer control, (a) when the angle of view of a projection image is large, (b) when the angle of view of a projection image is small, (C) is a figure which respectively shows the example of pointer control in case a projection image is distorted. It is a block diagram which shows the structure of the projector and pointer control apparatus which concern on Embodiment 1. FIG. It is a block diagram which shows the physical structure of the pointer control apparatus which concerns on Embodiment 1. FIG. It is a figure which shows the image which the projector which concerns on Embodiment 1 projects on 1 frame in time series. It is a figure for demonstrating the numerical parameter used for the camera-shake correction range determination process which concerns on Embodiment 1, (a) is a reference | standard correction range, (b) is a distortion parameter of a horizontal direction, (c) is a vertical direction. (D) is a figure for demonstrating the position of a pointer, and the resolution of a projection image. It is a flowchart of the pointer control process which the pointer control apparatus which concerns on Embodiment 1 performs. 6 is a flowchart of camera shake correction range determination processing executed by the pointer control device according to the first embodiment. It is a block diagram which shows the structure of the pointer which concerns on Embodiment 2 of this invention, and the pointer control apparatus contained in a projector. It is a figure which shows the image which the projector which concerns on Embodiment 2 projects on 1 frame in time series. 10 is a flowchart of pointer control processing executed by the pointer control device according to the second embodiment. 12 is a flowchart of camera shake correction range determination processing executed by the pointer control device according to the second embodiment.

Hereinafter, an image projection system according to an embodiment for carrying out the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

(Embodiment 1)
An information processing system 1 according to Embodiment 1 of the present application will be described with reference to FIG. The image projection system 1 includes a personal computer (PC 10), a projector 20 including a pointer control device 30, a screen 40, and a pointer device 50.

The PC 10 displays an image to be projected on the screen 40 by the projector 20.
This is a computer that outputs 0. In the first embodiment, the PC 10 is an analog RGB terminal or HDMI.
A notebook computer having a function of outputting an image externally using an image output terminal such as a (registered trademark) terminal.

The projector 20 is an image projection device that projects an image output from the PC 10 onto the screen 40. The projector 20 receives from the pointer device 50 position information indicating a position at which the pointer device 50 points on the projection image, and information (operation information) about an operation performed on the pointer device 50 by the user. The projector 20 transmits to the PC 10 the position of the pointer determined by the internal pointer control device 30 (after correcting camera shake) based on the position information and the size / shape of the projection image together with the operation information.

The screen 40 is a curtain that functions as a projection surface for receiving a light beam projected by the projector 20 and forming an image.

The pointer device 50 designates a part on the projection image on the screen 40 and acquires position information of the designated part. The position information is information indicating designated coordinates (x, y) on the image to be projected (projection image) corresponding to the position on the screen 40 designated by the pointer device 50. The pointer device 50 includes operation receiving means such as buttons, and transmits information of the received operation and position information to the projector 20. As an example of the operation received by the operation receiving unit, there is an operation corresponding to, for example, a right click or left click of the mouse.

The pointer device 50 acquires the designated coordinates by any known coordinate acquisition method on the screen, such as the pointer device 50 irradiating an infrared laser on the screen 40 and acquiring the position of the infrared laser with a camera. Here, it is assumed that the pointer device 50 includes an infrared camera, the projector 20 captures an infrared gradation image projected at a predetermined interval, and acquires coordinates on the image from the intensity.

An image projected by the projector 20 onto the screen 40 (projection image, image PI1 to image P
I3) on how the pointer control device 30 controls the pointer in FIG.
Will be described with reference to FIG.

FIG. 2A shows an example of camera shake correction when the projector 20 projects the image PI1 large on the screen 40 without distortion.
In FIG. 2A, the user uses the pointer device 50 to display the image PI on the screen 40.
1 part is indicated, and a pointer (black thick arrow) is displayed on the part. The user's hand holding the pointer device 50 shakes with a certain width.

At this time, when the pointer device 50 shakes due to hand shake, the part (instructed position) indicated by the pointer swings within a certain range. The pointer control device 30 determines that the movement in the predetermined range is a camera shake, and does not reflect the change in the designated position in the range in the position of the pointer.
Since PI1 is largely projected on the screen 40, the range in which the designated position is shaken by camera shake is smaller than the image PI1 on the screen 40. For this reason, the influence of camera shake on an operation indicating an icon, an operation part, or the like on the image PI1 is relatively small. there,
The pointer control device 30 sets a range (correction range, camera shake correction range) to be determined as camera shake to a relatively small range (for example, inside the part A1) of the image PI1.

FIG. 2B shows an example of camera shake correction when the projector 20 projects the image PI2 on the screen 40 small without distortion.
Suppose that the user points to a part on PI2 from the same position as in FIG. 2A together with the same amount of hand movement (FIG. 2B). At this time, the range in which the shake of the designated position due to camera shake is in the range of the image P
Since the ratio to I2 is larger than that in FIG. 2A, the influence of camera shake on the operation is large.
Therefore, the pointer control device 30 sets the correction range to a relatively large range on the image PI2 (
For example, it is set in the part A2. That is, when the image PI1 and the image PI2 have the same resolution, the area A2 has a larger area (number of pixels) than the area A1.

An example of camera shake correction when the image PI3 is distorted on the screen 40 as a result of the projector 20 projecting the image PI3 obliquely onto the screen 40 is shown in FIG.
From the same position as in FIG.
It is assumed that the part on the upper image PI3 is pointed. At this time, the range in which the shake of the indicated position due to the camera shake (the effect of the camera shake on the operation) varies depending on the position on the image PI3.

In the example of FIG. 2C, for example, in the vicinity of the part A3, the image is greatly extended due to distortion. Therefore, the range of movement of the indicated position due to camera shake is relatively small, and the camera shake affects the user operation. The impact is small.
On the other hand, in the vicinity of the part A4, the image is reduced to a small size due to the distortion, so that the influence of the shake of the designated position due to the camera shake on the user operation becomes larger than the vicinity of the part A3.

Therefore, the pointer control device 30 sets the correction range to a relatively small range (for example, the part A3) in the part where the image PI3 is greatly extended due to distortion, such as the part A3.
3), and in the portion reduced by distortion such as the part A4, the image PI
3 is set to a relatively large range (for example, inside the part A4).

Next, the configuration of the projector 20 will be described with reference to FIG. The projector 20 includes an input unit 200, an A / D converter 210, a display control unit 220, a light emission control unit 230,
The lamp unit 240, the projection device 250, the optical lens unit 260, the communication unit 270, the imaging unit 280, the output unit 290, and the pointer control device 30 are configured.

The input unit 200 includes image signal input terminals such as an analog RGB terminal and an HDMI (registered trademark) terminal, and receives an image signal output from the PC 10.
The input unit 200 transmits the received image signal to the A / D converter 210.

The A / D converter 210 converts the image signal transmitted from the input unit 200 into digital image information and transmits the digital image information to the display control unit 220.

The display control unit 220 includes a control CPU (Central Processing Uni).
t), RAM (Random Access Memory) used as a work area
), EPROM (Erasable P) for storing control programs, control parameters, etc.
and the light emission control unit 230 and the projection device 250 are controlled.
The display control unit 220 performs correction processing such as trapezoid correction on the image transmitted from the A / D converter 210, and controls the light emission control unit 230 and the projection device 250 to project the image onto the projection screen. .

The light emission control unit 230 is a control circuit that receives control from the display control unit 220 and controls the lamp unit 240.
The light emission control unit 230 causes the lamp unit 240 to supply a beam (light beam) necessary for the image projection by the projection device 250.

The lamp unit 240 emits light under the control of the light emission control unit 230, and the projection device 2.
50 is a light emitting device for supplying a beam to 50.

The projection device 250 includes a color wheel, DMD (Digital Micromir).
a device that is responsible for the projection function of a projector composed of
Based on the control of the display control unit 220, the image transmitted from the display control unit 220 is converted into a light beam and transmitted to the optical lens unit 260.
The projection device 250 is not limited to the above configuration, and may be any known image projection device such as an image projection device that projects an image using a liquid crystal panel.

The optical lens unit 260 is an optical device including a plurality of lenses and a motor that moves the lenses to a predetermined position. The optical lens unit 260 forms an image of the light beam transmitted from the imaging device 250 on the screen 40.

The communication unit 270 includes Bluetooth (registered trademark), wireless LAN (Local Ari).
a Network), and the like. The communication unit 270 communicates with the pointer device 50 and sequentially receives information on the designated coordinates indicating the position indicated by the pointer device 50 and operation information. The communication unit 270 sequentially transmits the received information to the communication control unit 340 of the pointer control device 30. A method for the communication unit 270 to communicate with the pointer device 50 is not limited to wireless communication, and a configuration in which communication is performed using a wired communication device such as a USB (Universal Serial Bus) or a parallel port is also possible.

The imaging unit 280 includes a lens, a CCD (Charge Coupled Device Im).
This is an imaging device configured to shoot a projected image projected on the screen 40.
The imaging unit 280 transmits the captured image obtained by capturing the projection image to the imaging control unit 310 of the pointer control device 30.

The output unit 290 is a communication device compliant with standards such as LAN, USB, and parallel port. The output unit 290 outputs information indicating a part on the projection image indicated by the pointer device 50 transmitted by the pointer device 50 and operation information to the PC 10.

As shown in FIG. 4, the pointer control device 30 physically includes an information processing unit 301, a data storage unit 302, a program storage unit 303, an input / output unit 304, and an internal bus 307. .

The information processing unit 301 includes a CPU (Central Processing Unit),
A process executed by the pointer control device 30 to be described later is executed according to a control program 308 that is configured by a DSP (Digital Signal Processing) and the like and stored in the program storage unit 303.

The data storage unit 302 includes a RAM (Random Access Memory) and the like, and is used as a work area of the information processing unit 301.

The program storage unit 303 includes a nonvolatile memory such as a flash memory or a hard disk, and stores a control program 308 that controls the operation of the information processing unit 301.
Note that the information processing unit 301, the data storage unit 302, the program storage unit 303, and the input / output unit 304 are respectively connected by an internal bus 307 and can transmit information.

The input / output unit 304 is an I / O unit that controls input / output of information with an external device.
The input / output unit 304 acquires data input from the communication unit 270 and the imaging unit 280 of the projector 20 and transmits the data to the information processing unit 301. The input / output unit 304 is a projector 2
0 output unit 290 communicates with an external device and outputs the calculation result of the information processing unit 301.

As shown in FIG. 3, the pointer control device 30 has an imaging control unit 310, an image processing unit 380 including a square field angle correction amount calculation unit 320 and a distortion amount calculation unit 330, and a communication control unit 340, as shown in FIG. 3. A correction range determination unit 350, a pointer position determination unit 360, and a storage unit 370.
And function as.

The imaging control unit 310 controls the imaging unit 280 and captures an image projected by the projection device 250 onto the screen 40 through the optical lens unit 260. The imaging control unit 310 transmits the captured image to the image processing unit 380.

The communication control unit 340 controls the communication unit 270 and acquires position information (coordinates of the designated position) transmitted by the pointer device 50 and operation information. The communication control unit 340 transmits the acquired coordinates of the designated position and the operation information to the pointer position determination unit 360.

Here, a projection image projected on the screen 40 by the projector 20 in one frame period is
A description will be given in time series with reference to FIG.
In one frame period T (for example, 1/60 seconds), the projector 20 first, during the image display period Ta, the red component (image IR) and the green component (image IG) of the image acquired from the PC 10.
) And the blue component (image IB) are sequentially projected. By projecting the image IR, the image IG, and the image IB in the image display period Ta, the projector 20 projects the projection image acquired from the PC 10 onto the screen 40.

Next, the projector 20 projects a distance measurement image (image IDx and image IDy) for measuring the shape and size of the projection image (distance measurement period Tb). In the distance measurement period Tb, first, an image IDx for measuring the distance in the horizontal direction is projected, and an image IDy for measuring the distance in the vertical direction is projected.
The image IDx is a vertical stripe pattern with a constant interval and a horizontal stripe pattern with a constant interval of the image IDy. Image IDx
The image IDy is projected on the screen 40 as a stripe pattern corresponding to the size and shape of the projection area on the screen 40. When the projection area is distorted in a trapezoidal shape on the screen 40, the image IDx and the image IDy are also distorted and projected as shown in FIGS. 6B and 6C. Image I
Dx and image IDy may be striped images of any color, but here they are infrared images.

Next, the projector 20 projects an image (an image IPx and an image IPy) for the pointer device 50 to acquire designated coordinates on the projection image (coordinate period Tc). In the image IPx, the light intensity has a horizontal gradation on the projection image, and the light intensity of a certain part corresponds to the x coordinate of the part. In the image IPx, the light intensity is a vertical gradation on the projection image, and the light intensity of a certain part corresponds to the y coordinate of the part.

The pointer device 50 uses the camera to acquire the intensity of the image IPx and the intensity of the image IPy, and to which part of the projected image the position pointed to is the x-coordinate and y-coordinate (indicated coordinates). get.

The imaging control unit 310 captures the image IDx and the image IDy using the imaging unit 280, and transmits the captured image to the image processing unit 380.

The communication control unit 340 includes information on the designated coordinates acquired by the pointer device 50 from the luminosities of the images IPx and IPy, and operation information indicating an operation (right click, left click, etc.) performed on the pointer device 50 by the user. Are acquired using the communication unit 270.

Returning to FIG. 3, the description will be continued. The image processing unit 380 extracts captured images of the image IDx and the image IDy from the captured image transmitted from the imaging control unit 310. Then, the square field angle correction amount calculation unit 320 and the distortion amount calculation unit 330 calculate a parameter (correction parameter) that defines a correction range for camera shake correction from the extracted images IDx and IDy. The content of the correction parameter and the process for calculating the correction parameter will be described later.
The image processing unit 380 transmits the calculated correction parameter to the correction range determination unit 350.

The correction range determination unit 350 determines a camera shake correction range from the correction parameters transmitted from the image processing unit 380 and the coordinates of the reference position stored in the history storage unit 3710 of the storage unit 370.

A process in which each unit of the image processing unit 380 calculates the correction parameter and the correction range determination unit 350 determines the camera shake correction range will be described with reference to FIG. FIG. 6A is a reference projection image BI projected on the screen 40 without distortion from a predetermined distance based on a predetermined setting. The setting for projecting the reference projection image BI is referred to as a reference projection setting. On the reference projection image BI, a reference distortion correction range SA (a rectangle having a size of vertical Cbv pixels and horizontal Cbh pixels on the projection image) is displayed as shown in FIG. The reference distortion correction range may be a square.

When the projector 20 projects an image with the reference projection setting, the vertical stripes of the image IDx are projected on the screen 40 as stripes having the reference width Hb. Further, the vertical stripes of the image IDx are projected on the screen 40 as stripes having a reference width Vb.

The vertical size Cbv, the horizontal size Cbh, the reference width Hb, and the reference width Vb of the reference distortion correction range SA are stored in advance as constants in the setting storage unit 3720 of the storage unit 370 described later.

FIGS. 6B to 6D show projection screens when projection is performed with distortion other than the reference projection setting.
). At this time, the vertical stripes of the image IDx appear on the screen 40 from the maximum width H1 to the minimum width H1.
It is projected as a stripe having s (FIG. 6B). Further, the horizontal stripe of the image IDy is the screen 4
It is projected as a stripe having a maximum width Vl to a minimum width Vs on 0 (FIG. 6C). From the image IDy, the image processing unit 380 obtains the width of the stripe on the origin O side (here, the minimum width Vs) and the width of the stripe farthest from the origin to the + side on the y axis (here, the maximum width Vl). Further, from the image IDx, the width of the stripe on the origin O side (here, the minimum width Hs) and the width of the stripe farthest from the origin to the + side on the x axis (here, the maximum width Hl) are obtained. The maximum width Hl and the minimum width Hs described above are (
Acquired in a strip-shaped region located in the center (in the horizontal direction). The width of this strip-shaped area is
It is assumed that the effect of strain in the region is small enough to be ignored. Maximum width Vl and minimum width Vs are
It shall be acquired in the same way by changing the direction.

The square field angle correction amount calculation unit 320 calculates a horizontal field angle ratio Zh from the acquired minimum width Hs and reference width Hb. The horizontal angle of view ratio Zh is a numerical value indicating how much the entire image is stretched in the horizontal direction.
The horizontal field angle ratio Zh is calculated using, for example, the calculation formula (1).
Zh = Hs / Hb (1)

The square field angle correction amount calculation unit 320 calculates a vertical field angle ratio Zv from the acquired minimum width Vs and reference width Vb. The vertical angle of view ratio Zh is a numerical value indicating how much the entire image is stretched in the vertical direction.
The vertical field angle ratio Zv is calculated using, for example, the calculation formula (2).
Zv = Vs / Vb (2)

The method for obtaining the horizontal field angle ratio Zh and the vertical field angle ratio Zv is not limited to the equations (1) and (2). For example, Hs (Vs) is divided by the square of Hb (Vb). ) And Hb (Vb
) And an argument may be used, or may be determined by using an arbitrary method such as dividing Hl (Vl) by Hb (Vb).

Further, the square field angle correction amount calculation unit 320 calculates the calculated horizontal field angle ratio Zh and the standard correction amount C.
A horizontal square correction amount C0h is calculated from bh.
The horizontal square correction amount C0h is one of the parameters for determining the horizontal size of the camera shake correction range, and increases as the ratio of the entire image stretched in the horizontal direction increases.
The horizontal square correction amount C0h can be calculated using, for example, Expression (3).
C0h = Cbh · Zh (3)
Similarly, from the calculated vertical angle of view ratio Zv and the standard correction amount Cbv, a vertical square correction amount C
0v can be calculated using equation (4).
C0v = Cbv · Zv (4)

The method for obtaining the horizontal square correction amount C0h and the vertical square correction amount C0v is expressed by the following equations (3), (
4), for example, a multiplication formula of squares of Cbh (Cbv) and Zh (Zv), or Cbh (
You may obtain | require using arbitrary methods, such as determining using the predetermined | prescribed reference table which uses Cbv) and Zh (Zv) as an argument.

The distortion amount calculation unit 330 calculates a horizontal distortion ratio Kh from the maximum width HI and the minimum width Hs. The horizontal distortion ratio Kh is a coefficient indicating how much the image is distorted in the horizontal direction and the degree of the distortion. Here, it is equal to 1 when the projected image is projected without distortion, and the portion at the origin O (here, the left side) becomes larger than 1 when the image is stretched more than the right side, and vice versa. The coefficient is smaller than 1.
The horizontal distortion ratio Kh can be calculated using, for example, Expression (5).
Kh = Hl / Hs (5)
Further, the vertical distortion ratio Kv is calculated from the maximum width Vl and the minimum width Vs using Expression (6). The vertical distortion ratio Kv is a coefficient that is larger than 1 when the image is stretched from the upper side at the origin O (lower side in this case), and smaller than 1 in the opposite case.
Kv = Vl / Vs (6)

Note that the method for obtaining the horizontal distortion ratio Kh and the vertical distortion ratio Kv is not limited to Expressions (5) and (6). For example, Hl (Vl) is divided by the square of Hs (Vs), or Hl (Vl ) And Hs (
Vs) may be obtained by using an arbitrary method such as using a predetermined reference table with an argument.

The image processing unit 380 generates a horizontal square correction amount C0h calculated by the square field angle correction amount calculation unit 320.
And the vertical square correction amount C0v, the horizontal distortion ratio Kh calculated by the distortion calculation unit 330, and the vertical distortion ratio Kv are transmitted to the correction range determination unit 350.

When the correction range determination unit 350 receives the horizontal square correction amount C0h, the vertical square correction amount C0v, the horizontal distortion rate Kh, and the vertical distortion rate Kv from the image processing unit 380, the correction range determination unit 350 further stores the storage unit The coordinates (Px, Py) of the reference position P stored in the history storage unit 3710 of 370 are acquired. Here, the reference position P is the current pointer position.
FIG. 6D shows that the reference position P stored in the history storage unit 3710 of the storage unit 370 has coordinates (P
The example is located at x, Py). Here, the resolution of the image PI4 is Gh pixels in the horizontal direction and Gv pixels in the vertical direction. Note that Px is a position where the reference position P (t) is shifted in the direction of the X axis by Px pixels from the origin O, and Py is a position where the reference position P is Py from the origin O.
Each pixel is located at a position shifted in the direction along the y-axis.

The correction range determination unit 350 calculates the horizontal correction amount Ch using Equation (7) and the vertical correction amount Cv using Equation (8).
Ch = C0h · Kh · (Px + Gh) / Gh (7)
Cv = C0v · Kv · (Py + Gv) / Gv (8)

Here, the horizontal correction amount Ch and the vertical correction amount Cv are expressed by the above equations (1) to (8).
However, the method of calculating Ch and Cv is not limited to this, and the correction amount increases as one pixel (or a predetermined number of pixels) of the projection image is greatly extended at the reference position on the projection screen. Using other calculation methods that reduce Ch and the correction amount Cv, the correction amount C
v and the correction amount Ch may be obtained.

The correction range determination unit 350 determines a region of horizontal Ch pixels and vertical Cv pixels centered on the reference position P on the projection image as a camera shake correction range (correction range), and sends information on the region to the pointer position determination unit 360. introduce.

Returning to FIG. 3, the description will be continued. The pointer position determination unit 360 determines a new point position based on the current designated coordinates and operation information transmitted from the communication control unit 340 and the camera shake correction range transmitted from the correction range determination unit 350.
Then, using the updated coordinates as a new reference position P, the history storage unit 371 of the storage unit 370.
Store to zero.

The pointer position determination unit 360 outputs the coordinates of the new pointer position and the operation information to the output unit 2.
90 is transmitted to the PC 10.

The storage unit 370 is a storage device that stores programs and information used by each unit other than the storage unit 370 of the pointer control device 30 for pointer control processing. The storage unit 370 supplies information to be stored in response to commands from each unit other than the storage unit 370 of the pointer control device 30.
The storage unit 370 stores the processing results of the units other than the storage unit 370 of the pointer control device 30 including the pointer position determination unit 360.
Storage unit 370 includes a history storage unit 3710 and a setting storage unit 3720.

The history storage unit 3710 stores a history of pointer coordinates including the coordinates of the reference position P.
The setting storage unit 3720 includes a vertical size Cbv and a horizontal size Cbh of the reference distortion correction range SA.
, Reference width Hb, reference width Vb, and the like are stored in various setting information necessary for pointer control processing.

Next, pointer control processing 1 executed by the pointer control device 30 will be described with reference to FIG.
In the pointer control device 30, the projector 20 projects a projection image, and the pointer device 5
When the designated coordinates are transmitted from 0, the pointer control processing 1 in FIG. 7 is started.

In the pointer control process 1, first, the imaging control unit 310 captures the image IDx and the image IDy using the imaging unit 280. Then, the image processing unit 380 extracts a fringe pattern (step S101).

The image processing unit 380 calculates the maximum width H1 and minimum width H of the vertical stripes of the image IDx from the acquired pattern.
s, and the maximum width Vl and the minimum width Vs of the image IDy are acquired (step S102).

Next, the square field angle correction amount calculation unit 320 calculates the horizontal field angle ratio Zh and the vertical field angle ratio Zv using the equations (1) and (2) (step S103).

Then, the distortion amount calculation unit 330 uses the expressions (5) and (6) to calculate the horizontal distortion ratio Kh.
And the vertical distortion ratio Kv are calculated (step S104).

Next, camera shake correction range determination processing 1 is executed using the horizontal field angle ratio Zh, the vertical field angle ratio Zv, the horizontal distortion ratio Kh, and the vertical distortion ratio Kv (step S105).

The camera shake correction range determination process 1 executed in step S105 will be described with reference to FIG.
In the camera shake correction range determination process 1, first, the square field angle correction amount calculation unit 320 calculates a horizontal square correction amount C0h using Equation (3) and a vertical square correction amount C0v using Equation (4). (Step S201).

Next, the correction range determination unit 350 acquires the coordinates (Px, Py) of the reference position P stored in the history storage unit 3710 of the storage unit 370 (step S202).

When the coordinates of the reference position P are acquired in step S202, the correction range determination unit 350 calculates the equation (7
) Is used to calculate the horizontal correction amount Ch and the vertical correction amount Cv using equation (8) (step S203).

The correction range determination unit 350 determines the region of horizontal Ch pixels and vertical Cv pixels centered on the reference position P as the region of the camera shake correction range (step S204), and ends the camera shake correction range determination process 1.

Returning to FIG. 7, in step S106, the communication control unit 340 obtains the current designated coordinates (step S106).

Then, the pointer position determination unit 360 determines whether or not the current designated position is included in the camera shake correction range (step S107).
If it is determined that the designated position is within the range of the camera shake correction range (step S107; YES)
Since it can be estimated that the movement in the range is a camera shake, the pointer position is not updated, the reference position is transmitted to the PC 10 as the pointer position, and the process returns to step S101.

On the other hand, if it is determined that the designated position is outside the range of the camera shake correction range (step S107; N
O) Based on the assumption that the operation of the pointer device 50 is not a camera shake, the new designated position is set as the pointer position (step S108).
In step S108, the updated pointer position is stored in the history storage unit 3710 of the storage unit 370 as a new reference position P. Also, a new pointer position is transmitted to the PC 10.

  Then, the process is repeated from step S101 using the new reference position.

According to the pointer control device 30 of the present embodiment, the movement of the designated position within a predetermined range from the reference position, which is the previous pointer position, can be determined as camera shake by the above processing, and the pointer position can be corrected. In addition, since the range to be determined as the camera shake correction can be automatically determined according to how much the projection image is extended at the reference position, the user's based on the size of the actual projection image Camera shake correction that meets your needs can be performed. Therefore, when the user operates the projected image using the pointer device 50, it is easy to use. Further, camera shake correction can be executed without the user performing complicated operations such as setting correction parameters by themselves.

In addition, according to the pointer control device 30 of the present embodiment, a square image that is calculated by using a fringe pattern conventionally used for trapezoidal correction and an imaging unit that captures the range to be determined as camera shake correction. It can be determined automatically using the angle and the distortion rate. Therefore, the user can provide a highly convenient camera shake correction function that does not require complicated operations such as special camera shake setting processing. Further, it is not necessary to execute special image processing for camera shake correction.
In addition, in the camera shake correction according to the conventional technique, since the parameters acquired by calibration are applied to the entire projection image, the camera shake correction is performed using the same correction parameters even when the screen is distorted. Flexible camera shake correction such as changing parameters could not be realized. Therefore, even if the parameter is determined based on the user operation, there is a problem that the camera shake correction parameter is different from the user's needs depending on the part. According to the pointer control device 30 of the present embodiment, camera shake correction can be performed using parameters calculated in consideration of how much the image is stretched at the current reference position. Camera shake correction can be realized.

In the first embodiment, the square angle of view and the distortion rate are acquired for each frame, and the correction parameters are calculated one by one. As a modification of the first embodiment, it is possible to store the correction parameter once acquired in the storage unit 370 and execute the camera shake correction process using the correction parameter stored for each frame.
At this time, the correction parameter may be calculated again in response to an event such as updating of the keystone correction setting or pressing of the reset button.

In the first embodiment, the distortion amount calculation unit 330 calculates the distortion rate Kh in the horizontal direction and the distortion rate Kv in the vertical direction as distortion amounts, and transmits them to the correction range determination unit 350. As a modification of the first embodiment, the distortion amount calculation unit 330 uses the width Hll of the stripe on the origin side (leftmost stripe) of the image IDx and the stripe farthest from the origin in the + direction of the x axis (the stripe on the rightmost side). ) Width Hlr may be transmitted to the correction range determination unit 350 as a horizontal distortion amount. Similarly, the combination of the stripe farthest from the origin in the + direction of the y-axis (uppermost stripe) Vlh and the width of the stripe on the origin side (lowermost stripe) Vsl is the amount of distortion in the vertical direction. To the correction range determination unit 350.

At this time, the correction range determination unit 350 uses the following equation (9), for example, to correct the horizontal correction amount C.
Calculate h.

(Gh−Px) / Gh in Equation (9) is a coefficient that increases as the designated coordinate Px is closer to the origin O side (left side) of the image, and Hll / (Hll + Hlr) increases as the left side of the image is stretched. It is a coefficient. Px / Gh indicates that the designated coordinate Px is on the + side (right side) from the origin of the image.
The coefficient, Hlr / (Hll + Hlr), which increases as the value increases, is a coefficient that increases as the right side of the image is stretched.

(Gh−Px) / Gh can be replaced with an arbitrary coefficient representing how much the designated coordinates Px are on the origin side in the projection image. For example, (Gh-Px) 2 / Gh2 can be substituted.
The same applies to Px / Gh.

Further, Hll / (Hll + Hlr) can be replaced with an arbitrary coefficient representing the degree to which the origin side of the projection image is stretched. For example, the coefficient Hll2 / (Hll + Hlr) 2 can be substituted.
The same applies to Hlr / (Hll + Hlr).

In addition, the correction range determination unit 350 calculates Cv in the vertical direction using, for example, the following equation (10).

In equation (9), (Gv−Py) / Gv is a coefficient that increases as the designated coordinate Py is closer to the origin O side (lower side) of the image, and Hll / (Hll + Hlr) is larger as the lower side of the image is stretched. Is a coefficient. Px / Gh indicates that the designated coordinate Px is on the + side (right side) from the origin of the image.
The coefficient, Hlr / (Hll + Hlr), which increases as the value increases, is a coefficient that increases as the right side of the image is stretched.
In the equation (10), the coefficient can be replaced as in the equation (9).

(Embodiment 2)
Next, Embodiment 2 of the present invention will be described.
In the image projection system 2 according to the second embodiment, the projector 21 projects a reference graphic centered on the current pointer position (reference position) onto the screen 40, and the pointer is based on the size of the reference graphic on the screen. The control device 31 determines the camera shake correction range.

Compared with the image projection system 1 according to the first embodiment, the image projection system 2 according to the second embodiment projects a predetermined figure centered on the pointer position for a predetermined period and includes a pointer control device 31. The projector 21 is different. Other configurations are the same as those of the image projection system 1 according to the first embodiment.

The configurations of the projector 21 and the pointer control device 31 according to the second embodiment will be described with reference to FIG.
The projector 21 includes a function of a display control unit (display control unit 221) and a pointer control device (
It has the same configuration as the projector 20 (FIG. 2) except that the function of the pointer control device 31) is different.

The display control unit 221 causes the projection device 250 to project a predetermined image (reference graphic) centered on the pointer position on the screen 40 for a predetermined period based on an instruction from the pointer position determination unit 361 of the pointer control device 31. Here, the reference graphic is a square of 20 pixels × 20 pixels on the projection image.

Here, a projection image projected on the screen 40 by the projector 21 in one frame period is
A description will be given in time series with reference to FIG.
The projector 21 projects a distance measurement image ID including a square SB of 20 pixels × 20 pixels centered on the current pointer position during the distance measurement period Tb. Other projected images are the same as the images projected by the projector 20 (FIG. 5).

Next, pointer control processing 2 executed by the pointer control device 31 will be described with reference to FIG.
In the pointer control device 31, the projector 21 projects a projection image, and the pointer device 5
When the designated position is transmitted from 0, the pointer control processing 2 in FIG. 11 is started.

In the pointer control processing 2, first, the reference graphic SB is centered on the current pointer pointing position.
And its arrangement are transmitted to the display control unit 221. The display control unit 221 displays an image ID including the reference graphic SB in the distance measurement period Tb (step S301).

Then, the imaging control unit 310 captures an image ID using the imaging unit 280. Then, the image processing unit 381 extracts (acquires) the reference graphic SB from the image ID (step S302).

The image processing unit 381 acquires the height H and width W of the extracted reference graphic SB (step S).
303).

Then, the image processing unit 381 executes the camera shake correction range determination process 2 using the height H and the width W of the reference graphic SB (step S304).

The camera shake correction range determination process 2 executed in step S304 will be described with reference to FIG. In the camera shake correction range determination process 2, first, step S2 of the camera shake correction range determination process 1 in FIG.
Similarly to 02, the coordinates of the reference position are acquired (step S401).
Next, the correction amount calculation unit 321 uses the equation (11) to calculate the horizontal correction amount Ch using the equation (12).
) Is used to calculate the vertical correction amount Cv (step S402).
Ch = CSw · W / SW (11)
Cv = CSh · H / SH (12)
However, SW is the width of the reference figure when projected with the reference projection setting, and SH is the height of the reference figure when projected with the reference projection setting.
CSw is the width (unit: pixel) of the image stabilization range when projected with the standard projection setting, C
Sh is the height (unit: pixel) of the camera shake correction range when projected with the standard projection setting. The numerical values of CSw, CSh, SW, and SH are stored in the setting storage unit 3720 of the storage unit 370.

When the correction amount is calculated in step S402, the camera shake correction range determination process 1 according to the first embodiment is performed.
As in step S204 (FIG. 8), the camera shake correction range is determined (step S403).

Returning to FIG. 11, when step S304 is completed, processing for determining a new pointer position is executed in the same manner as in step S106 to step S108 (FIG. 7) of pointer control processing 1 in the first embodiment (step S305 to step S307). .

In the pointer control process 2, a new pointer position is determined in step S 307, the reference position is updated, a reference graphic is arranged around the new reference position in the image ID, and the image ID is transmitted to the display control unit 221. (Step S308). In the next frame, a camera shake correction range is determined based on the new image ID.

According to the pointer control device 31 described in the second embodiment, it is necessary to determine the correction amount without performing the process of obtaining the vertical stripes and the horizontal stripes and determining the correction amount, thereby realizing the camera shake correction. Less processing is required.

(Modification)
As mentioned above, although the Example of this invention was described, embodiment of this invention is the said Embodiment 1-2.
However, the following modifications are possible.

For example, in the first and second embodiments, the camera shake correction area is a rectangle, but the camera shake correction range may be a circle, an ellipse, a triangle, or the like.
In the case of a circular shape, the radius of the camera shake correction range may be an average of the horizontal correction amount Ch and the vertical correction amount Cv.

In the first and second embodiments, the pointer is not moved when it is within the correction range. However, when it is within the correction range, a configuration in which the moving speed is reduced is also possible. At this time,
The reference position may be updated with a new pointer position, or the reference position may not be updated during a period within the correction range.

In addition, the information processing unit 301, the data storage unit 302, the program storage unit 303, and the input / output unit 3
The central part for performing processing for the pointer control device 30 (pointer control device 31) composed of 04, etc. can be realized by using a normal computer system, not by a dedicated system. For example, a computer program for executing the above operation is stored in a computer-readable recording medium (flexible disk, CD-ROM, DVD-
An information terminal that executes the above-described processing may be configured by storing and distributing in a ROM or the like and installing the computer program in a computer. Alternatively, the computer program may be stored in a storage device included in a server device on a communication network such as the Internet, and the information processing device may be configured by being downloaded by a normal computer system.

In addition, when the function of the pointer control device 30 (pointer control device 31) is realized by sharing the OS (operating system) and the application program, or in cooperation with the OS and the application program, only the application program portion is recorded. You may store in a medium or a memory | storage device.

It is also possible to superimpose a computer program on a carrier wave and distribute it via a communication network. For example, a bulletin board on a communication network (BBS: Bulletin Board Syste
The computer program may be posted in m) and distributed via a network. The computer program may be started and executed in the same manner as other application programs under the control of the OS, so that the above-described processing may be executed.

In addition, a part of the processing executed by the pointer control device 30 (pointer control device 31) is
You may implement | achieve using the computer independent of the pointer control apparatus 30 (pointer control apparatus 31).

As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to the specific embodiment which concerns, This invention includes the invention described in the claim, and its equivalent range It is. Hereinafter, the invention described in the scope of claims of the present application will be appended.

(Appendix 1)
An instruction coordinate acquisition unit that sequentially acquires instruction coordinates indicating a position indicated by the pointer device on the projection image that is an image to be projected onto the projection surface;
A reference position definition unit that determines a reference position on the projection image;
A projection mode acquisition unit that acquires a projection mode that is a mode in which the projection image is projected onto the projection plane;
A reference position determined by the reference position definition unit, a projection mode acquired by the projection mode acquisition unit, and
A determination criterion defining unit for determining a determination criterion for determining whether or not the change in the instruction coordinates acquired by the instruction coordinate acquisition unit is due to camera shake based on
When the designated coordinates acquired by the designated coordinate acquisition unit change, it is determined whether or not the change is due to camera shake using a determination criterion determined by the determination criterion definition unit, and the determination result and the reference position definition unit A pointer position control unit that determines the position of the pointer based on at least one of the determined reference position and the designated coordinates after the change;
A pointer control device comprising:

(Appendix 2)
The discrimination criterion defining unit defines a camera shake region including the reference position based on the projection mode acquired by the projection mode acquiring unit, and when the changed coordinate is included in the camera shake region, the change of the designated coordinate Determining the discrimination criteria as being due to camera shake,
The pointer control device according to Supplementary Note 1, wherein

(Appendix 3)
The projection mode acquisition unit includes at least one of a vertical size and a horizontal size of a projection image obtained by projecting the projection image on the projection plane, a horizontal distortion rate of the projection image, and a vertical direction. And at least one of the distortion rate of
The discrimination criterion defining unit includes at least one of a vertical size and a horizontal size acquired by the projection mode acquisition unit, and at least a horizontal distortion rate and a vertical distortion rate of the projection image. Deriving how much the projection image is extended on the projection plane at the reference position based on any one of them, and the greater the extent that the camera shake region is extended, the projection image The smaller on the top, the smaller the larger on the projection image,
The pointer control device according to Supplementary Note 2, wherein

(Appendix 4)
The projection aspect acquisition unit acquires the shape and size of the aspect measurement graphic on the projection image corresponding to the predetermined aspect measurement graphic, and based on the shape and size of the aspect measurement graphic on the projection image The projection mode is derived by deriving at least one of the vertical size and the horizontal size and at least one of the horizontal distortion rate and the vertical distortion rate of the projection image. To get the
The pointer control device according to Supplementary Note 3, wherein

(Appendix 5)
The projection mode acquisition unit obtains at least one of a height and a width of an area on the projection image corresponding to a reference graphic on the projection image including the reference position defined by the reference position definition unit. As a projection mode,
The pointer control device according to appendix 1 or 2, characterized by the above.

(Appendix 6)
The reference position defining unit defines the reference position based on a history of pointer positions determined by the pointer position control unit;
The pointer control device according to any one of appendices 1 to 5, characterized in that:

(Appendix 7)
If the pointer position control unit determines that the change in the designated coordinate is due to camera shake, the pointer position control unit sets the reference position as a new pointer position, and determines that the change is not due to camera shake, determines the designated position after the change. A new pointer position,
The pointer control device according to any one of appendices 1 to 6, characterized in that:

(Appendix 8)
A projection unit that projects a projection image that is an image to be projected onto the projection plane;
An instruction coordinate acquisition unit that sequentially acquires instruction coordinates indicating positions indicated by the pointer device on the projection image;
A reference position definition unit that determines a reference position on the projection image;
A projection mode acquisition unit that acquires a projection mode that is a mode in which the projection image is projected onto the projection plane;
A reference position determined by the reference position definition unit, a projection mode acquired by the projection mode acquisition unit, and
A determination criterion defining unit for determining a determination criterion for determining whether or not the change in the instruction coordinates acquired by the instruction coordinate acquisition unit is due to camera shake based on
When the designated coordinates acquired by the designated coordinate acquisition unit change, it is determined whether or not the change is due to camera shake using a determination criterion determined by the determination criterion definition unit, and the determination result and the reference position definition unit A pointer position control unit that determines the position of the pointer based on at least one of the determined reference position and the designated coordinates after the change;
A projector comprising:

(Appendix 9)
On the computer,
A process of sequentially obtaining designated coordinates indicating a position designated by a pointer device on a projection image, which is an image to be projected on a projection plane;
Processing for determining a reference position on the projection image;
A process of acquiring a projection mode in which the projection image is projected onto the projection plane;
Processing for determining a discrimination criterion for discriminating whether or not the change in the designated coordinates is due to camera shake based on the reference position and the projection mode;
When the indicated coordinates change, it is determined whether or not the change is due to camera shake using the determination criterion, the determination result, at least one of the reference position and the changed indicated coordinates,
Processing to determine the position of the pointer based on
A program characterized by having executed.

DESCRIPTION OF SYMBOLS 1 ... Image projection system, 10 ... PC, 20 ... Projector, 21 ... Projector, 30
DESCRIPTION OF SYMBOLS ... Pointer control apparatus, 31 ... Pointer control apparatus, 40 ... Screen, 50 ... Pointer device, 200 ... Input part, 210 ... A / D converter, 220 ... Display control part, 221 ... Display control part, 230 ... Light emission control part, 240 ... lamp unit, 250 ... projection device, 260
... Optical lens unit, 270 ... Communication section, 280 ... Imaging section, 290 ... Output section, 301 ... Information processing section, 302 ... Data storage section, 303 ... Program storage section, 304 ... Input / output section, 30
DESCRIPTION OF SYMBOLS 7 ... Internal bus 308 ... Control program, 310 ... Imaging control part, 320 ... Square view angle correction amount calculation part, 321 ... Correction amount calculation part, 330 ... Distortion amount calculation part, 340 ... Communication control part, 350 ... Correction range determination 360, pointer position determination unit, 370 ... storage unit, 380 ... image processing unit, 3
710 ... History storage unit, 3720 ... Setting storage unit, PI1 to PI4 ... Image, A1 to A4 ... Correction range, T ... One frame period, IB, IG, IR ... Image, IDx, IDy, ID ... (for distance measurement) Image, IPx, IPy ... image for coordinate measurement, SA ... reference camera shake correction range, SB ... reference figure, P ... reference position

Claims (11)

An instruction coordinate acquisition unit that acquires instruction coordinates indicating a position indicated by the pointer device on the projection surface or the projection image;
A comparison unit that compares the shape of the projection image displayed on the projection plane with the reference shape of the projection image;
A discriminant criterion for determining a discriminant criterion for discriminating whether or not the change in the indicated coordinate acquired by the indicated coordinate acquisition unit is due to camera shake based on the reference position on the projection surface or the projection image and the comparison result by the comparison unit A definition part;
A pointer control device comprising: A reference position definition unit for determining a reference position on the projection plane or the projection image;
When the designated coordinate acquired by the designated coordinate acquisition unit is changed, the discrimination criterion defining unit determines whether the change is due to camera shake using a discrimination criterion defined by the discrimination criterion defining unit. The pointer control device according to claim 1. A pointer position control unit that determines the position of the pointer based on the determination result of the discrimination reference definition unit, at least one of the reference position determined by the reference position definition unit and the indicated coordinates after the change,
The pointer control device according to claim 2 , further comprising: If the pointer position control unit determines that the change in the designated coordinate is due to camera shake, the pointer position control unit sets the reference position as a new pointer position, and determines that the change is not due to camera shake, determines the designated position after the change. A new pointer position,
The pointer control device according to claim 3. The determination criterion defining unit defines a determination region of a range smaller than the entire projection image as the determination criterion on the projection image displayed on the projection plane;
The pointer control device according to claim 1, wherein the pointer control device is a pointer control device. A recognition unit for recognizing a projection image displayed on the projection plane;
The discrimination criterion defining unit defines the discrimination region in a range based on a comparison result between a size of the projection image acquired by the recognition unit and a predetermined reference size;
The pointer control device according to claim 5. A detection means for detecting a horizontal or vertical distortion amount of the projection image displayed on the projection plane;
The discrimination criterion defining unit determines the discrimination region based on a distortion amount detected by the detection unit;
The pointer control device according to claim 5.   A projector comprising the pointer control device according to claim 1. On the computer,
Instruction coordinate acquisition processing for acquiring instruction coordinates indicating the position indicated by the pointer device on the projection surface or the projection image;
A comparison process for comparing the shape of the projection image displayed on the projection surface with the reference shape of the projection image;
A discriminant criterion for determining a discriminant criterion for discriminating whether or not the change of the indicated coordinate acquired in the indicated coordinate acquisition process is due to a camera shake based on the reference position on the projection plane or the projection image and the comparison result by the comparison process Definition processing,
A program characterized by causing a function to be executed. An instruction coordinate acquisition unit that acquires instruction coordinates indicating a position indicated by the pointer device on the projection surface or the projection image;
A recognition unit for recognizing a projection image displayed on the projection plane;
As a discrimination criterion for discriminating whether or not the change in the designated coordinates acquired by the designated coordinate acquisition unit is due to camera shake, the reference position on the projection plane or the projection image, and the size of the projection image acquired by the recognition unit And a discrimination criterion defining unit that determines a discrimination region of a range based on a comparison result with a predetermined reference size,
A pointer control device comprising: An instruction coordinate acquisition unit that acquires instruction coordinates indicating a position indicated by the pointer device on the projection surface or the projection image;
A detection unit for detecting a horizontal or vertical distortion amount of the projection image displayed on the projection plane;
A discriminant criterion for determining a discriminant criterion for discriminating whether or not the change in the indicated coordinate acquired by the indicated coordinate acquisition unit is due to camera shake based on the reference position on the projection surface or the projected image and the detection result by the detector A definition part;
A pointer control device comprising:

Images