JP2012221348A - Pointing system, image generation device, and image generation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pointing system, an image generation device, and an image generation program, capable of reducing a difference between a position indicated by a pointing device and a position where a pointer is displayed more easily than before.SOLUTION: A pointing system 1 includes an image generation device 300 for generating a composite image by compositing a pointer image indicating a pointer and a prescribed image, and an image display unit 600 for displaying the composite image. Also, the pointing system 1 includes a displacement amount detection unit for detecting a displacement amount of the position of a pointing device indicating the displayed composite image, and an operation detection unit for detecting an operation performed to the pointing device. The image generation device 300 turns a composition position for compositing the pointer image with the prescribed image to an initial position when a prescribed operation is detected, and then moves the composition position of the pointer image by a moving amount corresponding to the detected displacement amount.

Description

  The present invention relates to a pointing system, an image generation device, and an image generation program.

  2. Description of the Related Art Conventionally, a personal computer that changes the position of a pointer displayed on a large display based on displacement amounts of the pointing device detected by two angular velocity sensors with different axial directions mounted on the pointing device is known ( For example, Patent Document 1).

JP 2009-238004 A

  Here, since the position of the large display pointed to by the pointing device cannot be acquired with the personal computer of Patent Document 1, it is difficult to substantially match (that is, calibrate) the position and the position of the pointer displayed on the large display. There was a problem of being.

  The present invention has been made in view of such a point, and the object of the present invention is a pointing system that can easily reduce the difference between the position indicated by the pointing device and the position where the pointer is displayed, as compared with the prior art, An object is to provide an image generation apparatus and an image generation program.

In order to achieve the above object, a pointing system according to the first aspect of the present invention provides:
Image generating means for generating a composite image by combining a pointer image representing a pointer and a predetermined image;
Image display means for displaying the generated composite image;
A displacement amount detecting means for detecting a displacement amount of the position of the pointing device indicating the displayed composite image;
An operation detection means for detecting an operation performed on the pointing device,
When a predetermined operation is detected, the image generation unit sets a combination position for combining the pointer image with the predetermined image as a predetermined initial position, and then according to the detected displacement amount. Move the composite position of the pointer image by the amount of movement;
It is characterized by that.

  According to the pointing system, the image generation apparatus, and the image generation program of the present invention, the difference between the position indicated by the pointing device and the position where the pointer is displayed can be reduced more easily than in the past.

It is a system configuration figure showing an example of a pointing system concerning Embodiment 1 of the present invention. It is a figure showing an example of the coordinate system on the basis of a pointing device, and an example of a world coordinate system. It is a hardware block diagram showing the example of 1 structure of a pointing device. It is a functional block diagram showing an example of the function which a pointing device has. It is a flowchart showing an example of the process at the time of the 1st operation which a pointing device performs. 3 is a flowchart illustrating an example of timer interrupt processing executed by the pointing device according to the first embodiment. It is a hardware block diagram showing the example of 1 structure of an image generation apparatus. 3 is a part of a flowchart illustrating an example of image generation processing executed by the image generation apparatus according to the first embodiment. It is the other part of the flowchart showing an example of the image generation process which an image generation apparatus performs. It is a functional block diagram showing an example of the function which an image generation apparatus has. It is a figure showing an example of the coordinate system on the basis of the designated image. It is a system configuration figure showing an example of a pointing system concerning a modification of Embodiment 1. FIG. 10 is a diagram for explaining a calibration method in the second embodiment. 6 is a functional block diagram illustrating an example of functions of a pointing device according to Embodiment 2. FIG. 10 is a flowchart illustrating an example of a timer interrupt process executed by the pointing device according to the second embodiment. (A) is a figure for demonstrating the example of one relationship between the distance from a screen to a pointing device, and the movement range of a pointing device. (B) is a figure for demonstrating the other example of a relationship between the distance from a screen to a pointing device, and the movement range of a pointing device. 10 is a part of a flowchart illustrating an example of image generation processing executed by the image generation apparatus according to the second embodiment. It is a figure showing an example of the movement range which moves a pointing device in order to point the image projected on the screen.

(Embodiment 1)
Hereinafter, a pointing system 1 according to Embodiment 1 of the present invention will be described with reference to the accompanying drawings.

  A pointing system 1 according to the first embodiment of the present invention includes a pointing device 100, an access point 200, an image generation device 300, an image projection device 400, and a screen 500 as shown in FIG.

  The pointing device 100 includes a handy terminal as shown in FIG. The pointing device 100 may be configured by a PDA (Personal Digital Assistant), a mobile phone, or a PHS (Personal Handy-phone System).

  The pointing device 100 includes an LCD 100i and a keyboard 100j as shown in FIG. The pointing device 100 has a positive direction of the Xp axis defined in the right direction toward the LCD 100i, and a positive direction of the Yp axis defined in the display direction of the LCD 100i (that is, the direction perpendicular to the screen of the LCD 100i). The positive direction of the Zp axis is defined in the direction (that is, the upward direction of the image displayed on the LCD 100i). In addition, the Xw axis, the Yw axis, and the Zw axis of the world coordinate system are parallel to the Xp axis, the Yp axis, and the Zp axis, respectively, and the origin Ow is a position in the three-dimensional space where the power is supplied to the pointing device 100. Is stipulated.

  The user moves the pointing device 100 so that the position pointed to using the pointing device 100 is in the + Zp direction of the pointing device 100. When the pointing device 100 is moved by the user, it detects the displacement amount (Δx, Δy) of the position in the Xw axis direction and the Yw axis direction of the world coordinate system. The pointing device 100 detects an operation performed on the keyboard 100j by the user. The pointing device 100 transmits the operation information representing these detected operations and the displacement amount information representing the detected displacement amounts (Δx, Δy) to the image generation apparatus 300 via the access point 200. The pointing device 100 does not point at the position pointed to by the user with the laser beam.

  The access point 200 mediates information communicated between the pointing device 100 and the image generation apparatus 300.

  The image generation apparatus 300 generates a composite image by combining a predetermined image and a pointer image representing a position pointed to by the user with the pointing device 100. The image generation apparatus 300 changes a position (hereinafter referred to as a combining position) where the pointer image is combined with a predetermined image based on the operation and the displacement amount represented by the information received from the pointing device 100. Thereafter, the image generation apparatus 300 synthesizes the pointer image at the changed position of the predetermined image, and transmits information representing the generated synthesized image to the image projection apparatus 400.

  The image projection device 400 is an image display unit that is configured by a liquid crystal projector and displays a composite image on the screen 500 by projecting a composite image represented by information received from the image generation device 300 onto the screen 500.

  The pointing device 100 in FIG. 1 includes a CPU (Central Processing Unit) 100a, a ROM (Read Only Memory) 100b, a RAM (Random Access Memory) 100c, a hard disk 100d, a media controller 100e, a wireless LAN card (Local Area network) 100f, video card 100h, LCD (Liquid Crystal Display) 100i, keyboard 100j, speaker 100k, acceleration sensor 100l, and timer 100m.

  The CPU 100a performs overall control of the pointing device 100 by executing software processing according to a program stored in the ROM 100b or the hard disk 100d. The RAM 100c temporarily stores information (that is, data) to be processed when the CPU 100a executes the program.

  The hard disk 100d stores various types of information (that is, data). The pointing device 100 may include a flash memory instead of the hard disk 100d.

  The media controller 100e reads various data and programs from a recording medium including a flash memory, a CD (Compact Disc), a DVD (Digital Versatile Disc), and a Blu-ray Disc (registered trademark).

  The wireless LAN card 100f performs radio wave communication of various types of information with the image generation apparatus 300 via an access point 200 connected via a communication network 10 (not shown). The keyboard 100j inputs a signal or information corresponding to a user operation.

  The video card 100h draws (that is, renders) an image based on the digital signal output from the CPU 100a, and outputs an image signal representing the drawn image. The LCD 100i displays an image according to the image signal output from the video card 100h. The pointing device 100 may include a PDP (Plasma Display Panel) or an EL (Electroluminescence) display instead of the LCD 100i. The speaker 100k outputs sound based on the signal output from the CPU 100a.

  The acceleration sensor 100l includes an acceleration sensor (not shown) whose axial direction is the Xp axis in FIG. 2 and an acceleration sensor (not shown) whose axial direction is the Yp axis, and the acceleration applied to the pointing device 100 along the Xp axis. , And the acceleration applied to the Yp axis, respectively.

  When the operation of the timer 100m is started by the CPU 100a, an interrupt signal is input to the CPU 100a at a cycle set in the CPU 100a, and when the CPU 100a is stopped, the output of the interrupt signal is stopped.

  The CPU 100a in FIG. 3 executes an image designation process (not shown) that designates an image to be projected on the screen 500 in FIG. By executing this image designation process, the CPU 100a functions as the operation detection unit 101 as shown in FIG. 4 by cooperating with the keyboard 100j of FIG. 3, and by cooperating with the wireless LAN card 100f in FIG. It functions as the information transmission unit 102.

  When the image designation process is started, the operation detection unit 101 detects an operation of the keyboard 100j by the user. The operation detection unit 101 determines whether or not the detected operation is an operation for designating an image presented on the screen 500 (hereinafter referred to as a designation operation). At this time, if the operation detection unit 101 determines that the detected operation is not a designation operation, the operation designation unit 101 ends the execution of the image designation process. On the other hand, when the operation detecting unit 101 determines that the detected operation is a designated operation, the information transmitting unit 102 transmits operation information representing the detected designated operation via the access point 200 in FIG. After transmission to the image generation device 300, execution of the image designation process is terminated.

  When receiving the operation information transmitted from the pointing device 100, the image generating apparatus 300 in FIG. 1 generates information (hereinafter referred to as image information) representing an image designated by the designated operation represented by the operation information. Extracted from the document file stored in the apparatus 300. Next, the image generation apparatus 300 outputs the extracted image information to the image projection apparatus 400 in FIG. 1 and transmits the image information to the pointing device 100. The image projection device 400 projects an image represented by the output image information onto the screen 500 of FIG.

  The CPU 100a of the pointing device 100 executes an image display process (not shown) that displays an image represented by information transmitted from the image generation apparatus 300. By executing this image display processing, the CPU 100a further functions as the image display unit 105 in FIG. 4 by cooperating with the video card 100h and the LCD 100i, and as the information receiving unit 104 by cooperating with the wireless LAN card 100f. Function.

  When the image display process is started, the information reception unit 104 in FIG. 4 receives image information representing an image from the image generation apparatus 300, and the image display unit 105 in FIG. 4 displays an image represented by the received information. Is displayed, the execution of the image display process is terminated.

  The user operates the pointing device 100 while viewing the image displayed on the image display unit 105 in FIG. 4 or the image projected (that is, displayed) on the screen 500 in FIG. The operations performed by the user on the pointing device 100 include an operation for starting projection of a pointer image on the screen 500 in FIG. 1 (hereinafter referred to as a first operation) and an operation for ending projection of the pointer image (hereinafter referred to as “first operation”). The second operation). In the present embodiment, it is assumed that the first operation is an operation of depressing the “1” key of the keyboard 100j in FIG. 3 (hereinafter referred to as a depressing operation), and the second operation is an operation of depressing the “1” key. Although explained, it is not necessarily limited to this.

  When the CPU 100a of the pointing device 100 performs the first operation on the “1” key of the keyboard 100j in FIG. 3 and an event corresponding to the operation (hereinafter referred to as a pressing event) occurs, the CPU 100a The execution of the first operation process in FIG. 5 is started.

  First, the CPU 100a sets a variable first to store a value indicating whether or not information indicating a start request to start projection of a pointer image (hereinafter referred to as start request information) is transmitted after the occurrence of a pressing event. Then, the value “1” indicating that the data has not been transmitted yet is stored (step S11). Next, the CPU 100a starts the operation of the timer 100m in FIG. 3 (step S12).

  Each time the timer 100m that started the operation inputs an interrupt signal, the CPU 100a in FIG. 3 executes a timer interrupt process as shown in FIG. Thus, the CPU 100a functions as the displacement amount detection unit 103 in FIG. 4 by cooperating with the acceleration sensor 100l in FIG.

  When the timer interrupt process in FIG. 6 is started, the operation detection unit 101 in FIG. 4 determines whether the “1” key is pressed (that is, is being pressed) based on a signal output from the keyboard 100j in FIG. It is determined whether or not (step S21). At this time, if it is determined that the “1” key is depressed (step S21; Yes), the change amount detection unit 103 in FIG. 4 reads the detection value of the acceleration sensor 100l in FIG. (Step S22). Thereafter, the change amount detection unit 103 calculates the XwYwZw coordinates of the pointing device 100 in the world coordinate system XwYwZw of FIG. 2 based on the read value. Thereafter, the change amount detection unit 103 stores the Xw coordinate (that is, the x component) of the calculated coordinate in the variable x, and stores the Yw coordinate (that is, the y component) in the variable y (step S23).

  Next, the information transmission unit 102 in FIG. 4 determines whether or not the variable first is a value “1” (that is, whether or not start request information is transmitted) (step S24). At this time, when determining that the variable first is the value “1” (step S24; Yes), the information transmission unit 102 stores the value of the variable x stored in step S23 in the variable x0, and in step S23. The value of the variable y that stores the value is stored in the variable y0 (step S25).

  Thereafter, the information transmitting unit 102 transmits start request information for requesting to start projection of the pointer image to the image generating apparatus 300 in FIG. 1 (step S26), and then the value of the variable first is already set. After updating to the value “0” indicating transmission (step S27), the above processing is repeated from step S21.

  In step S24, when the information transmission unit 102 determines that the variable first is the value “0” (step S24; No), in step S23, from the value of the variable x storing the Xw coordinate value calculated this time, By subtracting the value of the variable x0 storing the calculated Xw coordinate value, the displacement amount Δx of the pointing device 100 in the Xw axis direction is calculated. Similarly, the information transmission unit 102 subtracts the variable y0 that stores the previously calculated Yw coordinate value from the variable y that stores the previously calculated Yw coordinate value, so that the Yw axis direction of the pointing device 100 can be reduced. A displacement amount Δy is calculated (step S28).

  Thereafter, the information transmission unit 102 transmits displacement amount information representing the calculated displacement amounts Δx and Δy to the image generation device 300 (step S29). Then, after the process similar to step S25 is performed (step S30), the said process is repeated from step S21.

  If it is determined in step S21 that the “1” key has not been depressed (step S21; No), the operation detection unit 101 determines that the “1” key has been depressed, and the operation detection unit 101 performs the second operation. Detect operations. Next, the information transmitting unit 102 transmits information representing a stop request for stopping the projection of the pointer image (hereinafter referred to as stop request information) to the image generating apparatus 300 in FIG. 1 (step S31). Then, after stopping the operation of the timer started in step S12 of FIG. 5 (step S32), the execution of the timer interrupt process is terminated.

  1 includes a CPU 300a, ROM 300b, RAM 300c, hard disk 300d, media controller 300e, wireless LAN card 300f, wired LAN card 300g, video card 300h, LCD 300i, keyboard 300j, and speaker 300k as shown in FIG. Consists of.

  The CPU 300a, ROM 300b, RAM 300c, hard disk 300d, media controller 300e, video card 300h, LCD 300i, keyboard 300j, and speaker 300k are the CPU 100a, ROM 100b, RAM 100c, hard disk 100d, media controller 100e, video card 100h, LCD 100i, keyboard of FIG. Since it is the same as 100j and the speaker 100k, description is abbreviate | omitted.

  The wireless LAN card 300f performs radio wave communication of various types of information with the pointing device 100 via the access point 200. The wired LAN card 300g performs wired communication of various types of information with the image projection device 400.

  The CPU 300a executes image generation processing as shown in FIGS. 8 and 9 using the hardware of FIG. Accordingly, the CPU 300a functions as the information search unit 302 and the image generation unit 304 as illustrated in FIG. 10, and functions as the information storage unit 303 in FIG. 10 by cooperating with the hard disk 300d and cooperates with the wireless LAN card 300f. 10 functions as the information receiving unit 301 and the information transmitting unit 308 in FIG. 10, and functions as the information output unit 307 in FIG. 10 in cooperation with the wired LAN card 300g.

  The information storage unit 303 in FIG. 10 stores an electronic file in which information representing a predetermined image is recorded. In the present embodiment, it is assumed that the predetermined image is a document image representing a document, and the information storage unit 303 stores an electronic file in which information representing the document image is recorded (that is, referred to as a document file). However, it is not limited to this. The document file is described as an electronic file created using, for example, presentation software. However, the document file is not limited to this.

  When the execution of the image generation process in FIG. 8 is started, the information receiving unit 301 in FIG. 10 determines whether information is received from the pointing device 100 via the access point 200 in FIG. 1 (step S41). . At this time, if the information receiving unit 301 determines that the information is not received (step S41; No), the process of step S41 is repeated again after a predetermined time has elapsed.

  In step S41, when the information receiving unit 301 determines that the information has been received (step S41; Yes), the information receiving unit 301 determines whether the received information is information indicating a specified operation (hereinafter referred to as specified operation information). (Step S42). At this time, if it is determined that the received information is designated operation information (step S42; Yes), the information search unit 302 in FIG. 10 represents an image designated by the designated operation (hereinafter referred to as a designated image). Information (hereinafter referred to as designated image information) is searched from the document file stored in the information storage unit 303 (step S43). Next, the information search unit 302 searches for information representing the pointer image stored in the information storage unit 303 (hereinafter referred to as pointer image information) (step S44). Thereafter, a value indicating that the pointer image is not projected (shown as pointer OFF) is stored in a pointer variable indicating whether or not the pointer image is projected on the screen 500 (step S45).

  If it is determined in step S42 that the received information is not designated operation information (step S42; No), the information receiving unit 301 determines whether the received information is start request information (step S46). At this time, if it is determined that the received information is start request information (step S46; Yes), the image generation unit 304 sets the movement amount from the previous synthesis position to the current synthesis position to “0”. . Specifically, the amount of movement of the composite position is an XfYf coordinate system as shown in FIG. 11, and the center of the designated image F represented by the information retrieved in step S43 in FIG. This is expressed using an image coordinate system in which the scanning direction of the image F is the Xf axis and the sub-scanning direction of the designated image F is the Yf axis. For this reason, the image generation unit 304 initializes the value of the movement amount variable x representing the movement amount of the composite position in the Xf-axis direction with “0”, and sets the value of the movement amount variable y representing the movement amount in the Yf-axis direction to “ Initialize with “0”.

  Further, the image generation unit 304 initializes the value of the variable lastX that stores the Xf coordinate value of the previous combined position and the value of the variable lastY that stores the Yf coordinate value of the previous combined position as the value “0” ( Step S47).

  Next, the image generation unit 304 sets the current synthesis position as a predetermined initial position represented by information stored in the information storage unit 303 of FIG. In the present embodiment, the description will be made assuming that the initial position is the origin Of of the image coordinate system. For this reason, the image generation unit 304 sets the Xf coordinate and Yf coordinate of the combined position of the pointer image (that is, the pointer position) to the value “0” (step S48). However, the present invention is not limited to this, and the initial position of the composite position may be, for example, the point Of ′ having the smallest Xf coordinate value and Yf coordinate value among the points included in the designated image F. good. After step S48, the image generation unit 304 updates the value of the pointer variable with a value indicating that the pointer image is projected (pointer ON) (step S49).

  In step S46, when the information receiving unit 301 determines that the received information is not start request information (step S46; No), the information receiving unit 301 determines whether the received information is stop request information (step S50). At this time, if it is determined that the received information is not information indicating a stop request (step S50; No), the information receiving unit 301 receives information indicating that the received information indicates the amount of displacement of the pointing device 100 (that is, displacement). Amount information).

  Next, the image generation unit 304 moves the combining position where the pointer image is combined with the designated image F based on the displacement amount represented by the received information. Specifically, the image generation unit 304 reads the displacement amount represented by the received information (shown as data read), and reads the displacement amount in the Xp-axis direction with respect to the movement amount variable x ( That is, Δx) is substituted, and the displacement amount in the Yp-axis direction (that is, Δy) is substituted for the movement amount variable y (step S51).

  Thereafter, the image generation unit 304 changes the Xf coordinate of the pointer image combining position from the value of the variable lastX (that is, the Xf coordinate value of the previous combining position) to the value of the movement variable x (that is, the Xp axis of the pointing device 100). (Direction displacement amount) is subtracted, and the Yf coordinate is a value obtained by subtracting the value of the movement amount variable y from the value of the variable lastY (step S52).

  In step S50, when the information reception unit 301 determines that the received information is stop request information (step S50; Yes), the image generation unit 304 uses a value (pointer OFF) indicating that the pointer image is not projected. The value of the pointer variable is updated (step S53).

  After step S45, step S49, step S52, and step S53 in FIG. 8, the image generation unit 304 in FIG. 10 moves the value of the variable lastX from the Xf coordinate value of the current synthesis position (that is, the value of the variable lastX). The value of the variable lastY is updated with the Yf coordinate value of the current composite position (step S54 in FIG. 9).

  Thereafter, the image generation unit 304 determines whether or not the value stored in the pointer variable is a value indicating that the pointer image is projected (pointer ON) (step S55). At this time, if the image generation unit 304 determines that the value stored in the pointer variable is a value indicating that the pointer image is projected (pointer ON) (step S55; Yes), the image generation unit 304 is searched in step S43 of FIG. The pointer image represented by the information retrieved in step S44 is synthesized with the pointer position updated in steps S48 and S52 of FIG. S56). Thereafter, the image generation unit 304 sets an image projected on the screen 500 (hereinafter referred to as a projection image) as the composite image generated in step S56.

  If it is determined in step S55 that the value stored in the pointer variable is not a value indicating that the pointer image is projected (pointer ON) (step S55; No), the image generation unit 304 performs step S43 in FIG. A designated image represented by the retrieved information is set as a projection image. Thereafter or after step S56, the information output unit 307 in FIG. 10 outputs information representing the projection image (hereinafter referred to as projection image information) to the image projection apparatus 400 in FIG. 1 (step S57). The image projection apparatus 400 projects a projection image represented by the output information onto the screen 500 in FIG. Thereafter, the above process is repeated from step S41.

  Next, the information transmission unit 308 in FIG. 10 transmits the projection image information to the pointing device 100 (step S58). Note that the pointing device 100 displays the projection image represented by the transmitted information on the image display unit 105 in FIG. 4 as described above.

  Note that the pointing device 100 of FIG. 1 and the image generation apparatus 300 may wirelessly communicate information without using the access point 200. Further, the image generation apparatus 300 may be mounted on the pointing device 100, and the pointing device 100 may wirelessly communicate information with the image projection apparatus 400 via or without the access point 200. Furthermore, the image generation apparatus 300 may be mounted on the image projection apparatus 400, and the pointing device 100 may wirelessly communicate information with the image projection apparatus 400 via or without the access point 200.

  Further, the pointing system 1 of FIG. 1 has been described as being configured by one image projection device 400 and one screen 500, but is not limited thereto. The pointing system 1 includes a plurality of image projecting apparatuses and a plurality of screens onto which images are projected by the plurality of image projecting apparatuses. A composite image represented by the output information may be projected on each screen. Here, with a pointing device that points using laser light, it is only possible to display points on an image displayed on one screen. However, according to this configuration, a plurality of pointers can be displayed simultaneously on a plurality of images displayed on a plurality of screens.

(Modification of Embodiment 1)
As shown in FIG. 1, the pointing system 1 according to the present embodiment has been described as including a pointing device 100, an access point 200, an image generation device 300, an image projection device 400, and a screen 500. However, the present invention is not limited to this, and as shown in FIG. 12, the pointing system 1 may be configured by an image display device 600 that is an image display unit instead of the image projection device 400 and the screen 500. The image display device 600 is configured by, for example, an LCD display, an EL display, or a PDP display. In this configuration, the image display device 600 displays an image represented by the information output from the image generation device 300, and the user points to the image displayed on the image display device 600 while operating the pointing device 100. Can be adopted.

  Further, in these configurations, the image generation apparatus 300 of FIG. 12 is mounted on the image display apparatus 600, and the pointing device 100 wirelessly communicates information with the image display apparatus 600 via or without the access point 200. Also good.

  According to these configurations, when the first operation, which is a predetermined operation, is detected, the composite image obtained by combining the pointer image is displayed at the initial position in order to combine the pointer image with the initial position of the predetermined image. The After that, when the pointing device 100 is displaced from the position where the first operation is detected, the composite position of the pointer image is changed from the initial position according to the detected displacement amount (that is, relative movement amount). Moved. For this reason, even if the calibration operation is not performed on the pointing device 100 in advance, the position where the pointer image is displayed can be easily pointed by the pointing device 100, so that the convenience of the pointing device 100 is improved compared to the conventional case. In addition, the difference between the position indicated by the pointing device 100 and the position where the pointer is displayed can be reduced more easily than in the past.

  In addition, according to these configurations, a predetermined image in which the pointer image is not combined is displayed after the second operation is performed until the first operation is performed. For this reason, it is possible to prevent the display of the predetermined image from being hindered by the pointer image. On the other hand, after the first operation is performed until the second operation is performed, after the composite image in which the pointer image is combined is displayed at the initial position, the amount corresponding to the displacement amount of the pointing device 100 is displayed. Thus, a combined image in which the pointer image is combined is displayed at the position moved from the initial position. For this reason, when the display of the pointer image is resumed, the pointer image is displayed at the initial position of the displayed composite image, so the difference between the position indicated by the pointing device and the position where the pointer is displayed can be made easier than before. Can be reduced.

  Further, according to these configurations, when the image is designated, the pointing device 100 displays the designated image on the image display unit 105 in FIG. 4, and the image projection apparatus 400 in FIG. The twelve image display devices 600 display a designated image or a synthesized image obtained by synthesizing the image and the pointer image. For this reason, since the user can confirm the image displayed by the image projection apparatus 400 or the image display apparatus 600 using the pointing device 100, the convenience improves compared with the past.

  Furthermore, according to these configurations, the pointing device 100 displays the same image as that displayed by the image projection device 400 of FIG. 1 and the image display device 600 of FIG. For this reason, even when the user cannot stare at the images displayed by the image projection apparatus 400 and the image display apparatus 600 of FIG. 12, the image projection apparatus 400 can be viewed by visually observing the image displayed on the pointing device 100. Alternatively, since the display position of the pointer pointing to the image displayed by the image display device 600 can be moved, the convenience is improved as compared with the conventional case.

(Embodiment 2)
Next, Embodiment 2 will be described. Note that, in the configuration of the pointing system according to the second embodiment, the description of the configuration common to the first embodiment is omitted.

  Similar to the pointing system 1 of the first embodiment, the pointing system of the second embodiment includes a pointing device 100, an access point 200, an image generation device 300, an image projection device 400, and a screen 500.

  After the user points the upper left point P1 of the image displayed on the screen 500 as shown in FIG. 13 with the pointing device 100, the user performs the third operation on the pointing device 100 and starts from the remaining four corner points P2. Point to P4 in order. In the present embodiment, the third operation is an operation of depressing the “9” key of the keyboard 100j in FIG. 3, and the fourth operation is an operation of depressing the “9” key. However, the present invention is not limited to this. It is not done.

  When the third operation is performed on the “9” key of the keyboard 100 j in FIG. 3, the CPU 100 a of the pointing device 100 executes the same process as the first operation time process in FIG. 5. Thereafter, the CPU 100a receives the interrupt signal from the timer 100m in FIG. 3 until the fourth operation is performed on the “9” key of the keyboard 100j in FIG. 3 (that is, until the push-up operation is performed). In addition, a movement range specifying process (not shown) is executed. The movement range specifying process is a process of specifying a movement range RM in which the pointing device 100 is moved so that the user points from the predetermined position to the display area RS on which the image of FIG. 13 is displayed. By executing this process, the CPU 100a further functions as a range specifying unit 106 as shown in FIG.

  When execution of a movement range specifying process (not shown) is started, the range specifying unit 106 calculates the Xw coordinate value and the Yw coordinate value of the pointing device 100 in the XwYwZw coordinate system based on the displacement amount detected by the displacement amount detection unit 103. calculate. Next, the range specifying unit 106 specifies the movement range RM of the pointing device 100 on the XwYw plane based on the calculated Xw coordinate value and Yw coordinate value of the pointing device 100.

  Thereafter, the range specifying unit 106 calculates the distance from the center of the specified moving range RM in the Xw direction to one end of the moving range RM in the Xw direction, and stores the calculated value in the variable xlen. Similarly, the range specifying unit 106 calculates the distance from the center in the Yw direction of the specified moving range RM to one end of the moving range RM in the Yw direction, and stores the calculated value in the variable ylen. Thereafter, the range specifying unit 106 ends the execution of the movement range specifying process.

  As in the first embodiment, the pointing device 100 according to the second embodiment executes a process corresponding to the timer interrupt process in FIG. 6 from the time when the first operation is detected until the second operation is detected. To do. This process is a timer interruption process as shown in FIG. 15, and the process of step S68 determines the displacement amount of the pointing device 100 in the Xw direction and the displacement amount in the Yw direction, the length of the movement range RM in the Xw direction, and This is different from step S28 of FIG. 6 executed in the first embodiment in that it is a process of obtaining normalized values using lengths in the Yw direction.

  Specifically, the process of step S68 is performed by multiplying the value obtained by subtracting the previous Xw coordinate value from the Xw coordinate value of the current pointing device 100 by 100, and then moving the value multiplied by 100 to the variable xlen. In this process, the value calculated by dividing the RM by the length in the Xw direction (that is, the percentage of the change amount of the Xw coordinate value with respect to xlen) is used as the normalized displacement amount in the Xw direction. The reason why the value obtained by subtracting the previous Xw coordinate value from the current Xw coordinate value is multiplied by 100 is also to prevent the occurrence of an error due to a digit loss.

  Similarly, in step S68, the value obtained by subtracting the previous Yw coordinate value from the Yw coordinate value of the current pointing device 100 is multiplied by “100”, and then the value multiplied by “100” is stored in the variable ylen. This is also a process of setting the percentage calculated by dividing by the value to the normalized displacement amount in the Yw direction.

  Here, as shown in FIG. 16A and FIG. 16B, in order to point the image displayed on the screen 500 using the pointing device 100, a moving range RM in which the user moves the pointing device 100 is It becomes wider as the user approaches the screen 500. Further, even if the distance between the screen 500 and the user is the same, the moving range RM for moving the pointing device 100 is different for each user.

  Therefore, according to these configurations, information indicating the displacement amount of the pointing device 100 normalized based on the specified movement range RM of the pointing device 100 is transmitted, and thus the image displayed on the screen 500 and Even if the distance to the pointing device 100 or the user using the pointing device 100 changes, if the displacement amount of the position indicated by the pointing device 100 is the same, the transmitted variation amount information can be made substantially the same.

  The image generation apparatus 300 according to the second embodiment executes processing similar to the image generation processing executed in the first embodiment. This process is an image generation process as shown in FIG. 17, and the process of step S91 is the center of the designated image F shown in FIG. 11 with respect to the normalized displacement amount Δx in the Xw direction of the pointing device 100. And the value obtained by multiplying the length XLEN from one end in the Xf direction by the number “100” used for preventing digit loss in step S68 of FIG. 15, and the normalized displacement Δy Is different from step S51 in FIG. 8 executed in the first embodiment in that the value obtained by multiplying the length YLEN from the center of the designated image F to one end in the Yf direction by the number “100” is divided. is doing. Note that information representing XLEN and information representing YLEN are stored in the information storage unit 303 in FIG.

  According to these configurations, since the movement range RM for moving the pointing device 100 is specified and the movement amount of the composite position of the pointer image is changed based on the specified movement range RM, the displayed image and the pointing device 100 are changed. The position indicated by the pointing device and the position where the pointer is displayed are changed even if the distance to the user changes, the user who uses the pointing device 100 changes, or the size of the displayed image changes. The difference can be reduced more than before. In particular, according to these configurations, the user moves the position of the pointing device 100 in the movement range RM while pointing the image displayed on the screen 500 from the third operation to the fourth operation. Therefore, the pointing device 100 can be calibrated (that is, the difference between the position pointed to by the pointing device and the position where the pointer is displayed) can be reduced.

(Modification of Embodiment 2)
Next, a modification of the second embodiment will be described. In the modification of the second embodiment, as shown in FIG. 18, an example will be described in which the user points an image displayed on the screen 500 using the pointing device 100 from a position diagonally forward to the right of the screen 500.

  The shape of the range RS where the image is displayed on the screen 500 shown in FIG. 18 is a rectangle. Further, the shape of the projected image represented by the information output to the image projection apparatus 400 that projects the image onto the screen 500 is also a rectangle as shown in FIG. However, since the user points to the image displayed on the screen 500 from the position of FIG. 18, the shape of the moving range RM for moving the pointing device 100 is a trapezoid. This is because the user visually recognizes the rectangular image displayed on the screen 500 from a position diagonally forward to the right of the screen 500.

  In the modification of the present embodiment, when the pointing device 100 specifies the moving range RM of the pointing device 100 with the range specifying unit 106 in FIG. 14, the information transmitting unit 102 displays the moving range information representing the moving range RM with the image generating apparatus 300. Send radio waves to

  When the image generation apparatus 300 receives the movement range information by the information reception unit 301 in FIG. 10, the information search unit 302 searches for image information representing the designated image stored in the information storage unit 303. Next, the image generation unit 304 changes the shape of the movement range RM represented by the received movement range information (that is, a trapezoid) to the shape of the designated image represented by the searched image information (that is, a rectangle). A conversion matrix to be converted is calculated.

  When the image generation processing of FIG. 17 is executed, the image generation unit 304 corrects the change amounts Δx and Δy represented by the information received from the pointing device 100 by converting them using the conversion matrix in step S91. Later, the movement amounts x and y of the composite position of the pointer image are calculated using the corrected changes Δx and Δy. Note that the image generation unit 304 may correct the calculated movement amounts x and y using the conversion matrix after calculating the movement amounts x and y of the combined position using the change amounts Δx and Δy. . Thereafter, the image generation unit 304 continues to execute the image generation process from step S92.

  According to these configurations, the movement amounts x and y of the composite position of the pointer image are corrected based on the conversion matrix for converting the movement range RM of the pointing device 100 into the display range RS where the image is displayed on the screen 500. For this reason, for example, even when the user is not facing the screen 500, the difference between the position indicated by the pointing device and the position where the pointer is displayed can be reduced as compared with the conventional case.

  Embodiment 1, Embodiment 1 of the present invention, Embodiment 2 and Embodiment 2 can be combined with each other.

  It should be noted that the present invention can be provided as a pointing device 100 that has a configuration for realizing the functions according to the first embodiment, the first modification, the second embodiment, or the second embodiment, as well as the application of the program. The existing pointing device can also function as the pointing device 100 according to the first embodiment, the modified example of the first embodiment, the second embodiment, or the modified example of the second embodiment. That is, a computer for controlling an existing pointing device with a program for realizing each functional configuration by the pointing device 100 exemplified in the first embodiment, the modification of the first embodiment, the second embodiment, or the modification of the second embodiment. By being applied so that it can be executed (such as a CPU), it can function as the pointing device 100 according to the first embodiment, the modification example of the first embodiment, the second embodiment, or the modification example of the second embodiment.

  Similarly, the program can be provided as an image generation apparatus 300 having a configuration for realizing the functions according to the first embodiment, the first modification, the second embodiment, or the second modification. By application, the existing image generation apparatus can be caused to function as the image generation apparatus 300 according to the first embodiment, the modification example of the first embodiment, the second embodiment, or the modification example of the second embodiment. That is, a program for realizing each functional configuration by the image generation apparatus 300 exemplified in the first embodiment, the modification example of the first embodiment, the second embodiment, or the modification example of the second embodiment is controlled by the existing image generation apparatus. By being applied so that it can be executed by a computer (such as a CPU), it can function as the image generation apparatus 300 according to the first embodiment, the first variation, the second embodiment, or the second variation. .

  Such a program distribution method is arbitrary. For example, the program can be distributed by being stored in a recording medium such as a memory card, a CD-ROM, or a DVD-ROM, or via a communication medium such as the Internet. .

Various embodiments and modifications of the present invention are possible without departing from the broad spirit and scope of the present invention. That is, although several embodiments of the present invention have been described, the above-described embodiments are for explaining the present invention and do not limit the scope of the present invention. The scope of the present invention includes the invention described in the scope of claims and its equivalents, not the embodiments.
The invention described in the scope of claims of the present application is appended below.

(Appendix 1)
Image generating means for generating a composite image by combining a pointer image representing a pointer and a predetermined image;
Image display means for displaying the generated composite image;
A displacement amount detecting means for detecting a displacement amount of the position of the pointing device indicating the displayed composite image;
An operation detection means for detecting an operation performed on the pointing device,
When a predetermined operation is detected, the image generation unit sets a combination position for combining the pointer image with the predetermined image as a predetermined initial position, and then according to the detected displacement amount. Move the composite position of the pointer image by the amount of movement;
A pointing system characterized by that.

(Appendix 2)
The operation detecting means further detects a second operation performed on the pointing device in addition to the first operation which is the predetermined operation,
The image generating means includes
When the first operation is detected, the composite position of the pointer image is set as the initial position,
Until the second operation is detected after the first operation is detected, the composite position of the pointer image is moved according to the detected displacement amount,
Until the first operation is detected again after the second operation is detected, the synthesis of the pointer image with respect to the predetermined image is stopped,
The image display means displays the composite image from when the first operation is detected until the second operation is detected, and after the second operation is performed, the first operation is performed again. Display the predetermined image until done
The pointing system according to appendix 1, wherein

(Appendix 3)
The operation detection means further detects a third operation and a fourth operation performed on the pointing device,
In order to point the displayed composite image from a predetermined position on the basis of a displacement amount of the pointing device detected from when the third operation is detected until the fourth operation is detected, the pointing A moving range specifying means for specifying a moving range in which the device is moved;
The image generation means changes a moving amount of the composite position of the pointer image based on the specified moving range.
The pointing system as set forth in Appendix 2, wherein

(Appendix 4)
Information storage means for storing information representing the predetermined image;
The image generation means uses the conversion matrix for converting the shape of the detected movement range into the shape of the predetermined image represented by the stored information, and calculates the movement amount of the composite position of the pointer image. to correct,
The pointing system according to Supplementary Note 3, wherein

(Appendix 5)
The operation detection means further detects a designation operation for designating the predetermined image,
The pointing device includes a portable terminal having the operation detection unit, an information transmission unit that transmits operation information representing a designated operation detected by the operation detection unit, and a unit that displays an image designated by the designation operation. Terminal,
The image display means displays a predetermined image designated by a designation operation represented by the transmitted operation information or a synthesized image obtained by synthesizing the designated predetermined image and the pointer image;
The pointing system according to any one of appendices 1 to 4, characterized in that:

(Appendix 6)
The image display means displays the composite image on the object by projecting the composite image generated by the image generation means onto the object.
The pointing system according to any one of appendices 1 to 5, characterized in that:

(Appendix 7)
Image generating means for generating a composite image by combining a pointer image representing a pointer and a document image representing a document;
Information receiving means for receiving information representing an operation performed on the pointing device pointing to the displayed composite image, and information representing a displacement amount of the pointing device;
When the operation represented by the received information is a predetermined operation, the image generation means sets the combination position for combining the pointer image to the document image as a predetermined initial position, and then performs the detection. Moving the composite position of the pointer image by a movement amount corresponding to the displacement amount made,
An image generation apparatus characterized by that.

(Appendix 8)
Computer
Image generating means for generating a composite image by combining a pointer image representing a pointer and a document image representing a document;
Functioning as information receiving means for receiving information representing an operation performed on the pointing device pointing to the displayed composite image and information representing a displacement amount of the pointing device;
When the operation represented by the received information is a predetermined operation, the image generation means sets the combination position for combining the pointer image to the document image as a predetermined initial position, and then performs the detection. Moving the composite position of the pointer image by a movement amount corresponding to the displacement amount made,
An image generation program characterized by that.

DESCRIPTION OF SYMBOLS 1 ... Pointing system, 100 ... Pointing device, 100a ... CPU, 100b ... ROM, 100c ... RAM, 100d ... Hard disk, 100e ... Media controller, 100f ... Wireless LAN card, 100h, video card, 100i, LCD, 100j, keyboard, 100k, speaker, 100l, acceleration sensor, 100m, timer, 101, operation detection unit, 102 ...... Information transmission unit 103 ... Displacement amount detection unit 104 ... Information reception unit 105 ... Image display unit 106 ... Range specifying unit 200 ... Access point 300 ... Image generating device, 300a ... CPU, 300b ... ROM, 300c ... RAM, 300d ... C Disk, 300e ... media controller, 300f ... wireless LAN card, 300g ... wired LAN card, 300h ... video card, 300i ... LCD, 300j ... keyboard, 300k ... speaker , 301... Information reception unit, 302... Information search unit, 303... Information storage unit, 304... Image generation unit, 307. ... Image projection device, 500 ... Screen, 600 ... Image display device

Claims (8)

Image generating means for generating a composite image by combining a pointer image representing a pointer and a predetermined image;
Image display means for displaying the generated composite image;
A displacement amount detecting means for detecting a displacement amount of the position of the pointing device indicating the displayed composite image;
An operation detection means for detecting an operation performed on the pointing device,
When a predetermined operation is detected, the image generation unit sets a combination position for combining the pointer image with the predetermined image as a predetermined initial position, and then according to the detected displacement amount. Move the composite position of the pointer image by the amount of movement;
A pointing system characterized by that. The operation detecting means further detects a second operation performed on the pointing device in addition to the first operation which is the predetermined operation,
The image generating means includes
When the first operation is detected, the composite position of the pointer image is set as the initial position,
Until the second operation is detected after the first operation is detected, the composite position of the pointer image is moved according to the detected displacement amount,
Until the first operation is detected again after the second operation is detected, the synthesis of the pointer image with respect to the predetermined image is stopped,
The image display means displays the composite image from when the first operation is detected until the second operation is detected, and after the second operation is performed, the first operation is performed again. Display the predetermined image until done
The pointing system according to claim 1. The operation detection means further detects a third operation and a fourth operation performed on the pointing device,
In order to point the displayed composite image from a predetermined position on the basis of a displacement amount of the pointing device detected from when the third operation is detected until the fourth operation is detected, the pointing A moving range specifying means for specifying a moving range in which the device is moved;
The image generation means changes a moving amount of the composite position of the pointer image based on the specified moving range.
The pointing system according to claim 2. Information storage means for storing information representing the predetermined image;
The image generation means uses the conversion matrix for converting the shape of the detected movement range into the shape of the predetermined image represented by the stored information, and calculates the movement amount of the composite position of the pointer image. to correct,
The pointing system according to claim 3. The operation detection means further detects a designation operation for designating the predetermined image,
The pointing device includes a portable terminal having the operation detection unit, an information transmission unit that transmits operation information representing a designated operation detected by the operation detection unit, and a unit that displays an image designated by the designation operation. Terminal,
The image display means displays a predetermined image designated by a designation operation represented by the transmitted operation information or a synthesized image obtained by synthesizing the designated predetermined image and the pointer image;
The pointing system according to any one of claims 1 to 4, wherein The image display means displays the composite image on the object by projecting the composite image generated by the image generation means onto the object.
The pointing system according to any one of claims 1 to 5, wherein: Image generating means for generating a composite image by combining a pointer image representing a pointer and a document image representing a document;
Information receiving means for receiving information representing an operation performed on the pointing device pointing to the displayed composite image, and information representing a displacement amount of the pointing device;
When the operation represented by the received information is a predetermined operation, the image generation means sets the combination position for combining the pointer image to the document image as a predetermined initial position, and then performs the detection. Moving the composite position of the pointer image by a movement amount corresponding to the displacement amount made,
An image generation apparatus characterized by that. Computer
Image generating means for generating a composite image by combining a pointer image representing a pointer and a document image representing a document;
Functioning as information receiving means for receiving information representing an operation performed on the pointing device pointing to the displayed composite image and information representing a displacement amount of the pointing device;
When the operation represented by the received information is a predetermined operation, the image generation means sets the combination position for combining the pointer image to the document image as a predetermined initial position, and then performs the detection. Moving the composite position of the pointer image by a movement amount corresponding to the displacement amount made,
An image generation program characterized by that.

Images