JP5477833B2 - Pointing device, pointing method and program - Google Patents

Description

  The present invention is used in, for example, a system that projects a computer screen onto a screen by a projector, and a pointing device with a camera having a function of pointing an arbitrary position on the screen and reflecting it on the computer screen. The present invention relates to a pointing method and a program used.

  In recent years, there has been known a system in which an arbitrary position on a screen is pointed and reflected on a computer screen using a so-called “camera pointer” with a camera (see, for example, Patent Document 1).

A camera is provided at the tip of the pointing device. By recognizing the center of the image (center of angle of view) captured by the camera as the pointing position (designated position) of the operator, A pointer is displayed at the corresponding position.
JP 2001-166881 A

  In the pointing device with a camera as described above, a pointer can be displayed at a position where the camera is directed. However, since the movement of the pointer depends on the movement of the camera, it is very difficult to point to a fine part on the screen while holding the apparatus body by hand and changing the direction of the camera.

  SUMMARY OF THE INVENTION The present invention has been made in view of the above points. A pointing device and a pointing device that can accurately point to an intended position by moving the pointer small in a situation where an operator is pointing to a small portion on the screen. An object is to provide a method and a program.

A pointing device according to claim 1 of the present invention is a pointing device for pointing an arbitrary position of a pointing region,
Photographing means for photographing the pointing area;
An image detection unit that detects an image corresponding to the pointing area from a captured image obtained by the imaging unit, and the pointing according to a relative relationship between the pointing area image detected by the image detection unit and the captured image. Pointer position calculating means for calculating the corresponding position of the area, storage means for storing the pointer position calculated by the pointer position calculating means, and displaying the pointer at the current pointer position calculated by the pointer position calculating means The pointer position is calculated from the first pointing means, the current pointer position calculated by the pointer position calculating means, and the pointer position from a predetermined time before and stored in the storage means, and the calculation is performed. The second to display the pointer at the designated pointer position And in computing means, a moving speed detecting means for detecting a moving speed of said pointer from a time change in the relative relationship between the pointing region image and the captured image, in the normal mode where the correction mode is canceled, the moving speed detection The correction mode is set when the state satisfying the first condition that the moving speed detected by the means is equal to or lower than the predetermined speed continues for a predetermined time or longer, and the moving speed is set to satisfy the second condition that is higher than the predetermined speed . The correction mode is maintained, the normal mode is set by canceling the correction mode when the second condition is satisfied, and the first condition is satisfied for less than a predetermined time in the normal mode. However, the correction mode switching means for maintaining the normal mode and the pointer display position by the first pointing means while the correction mode is released. Display control means for continuously performing the updating operation of the pointer display position by performing the updating operation of the pointer display position by the second pointing means a plurality of times while the correction mode is set. Features.

Further, Claim 2 of the present invention, the pointing device according to claim 1, wherein, prior Symbol first Ponting means, the display position of the pointer with respect to time changes in the relative relationship between the captured image and the pointing area image Change of the pointer display position with respect to the time change of the relative relationship between the pointing area image and the photographed image becomes small.

Furthermore, a third aspect of the present invention, the pointing device according to claim 1 or 2, before Symbol second pointing means comprises: a current pointer position calculated by the pointer position calculation means, by the storage means moving speed of the pointer stored is to calculate the pointer position from the pointer position of the time that continues to below a predetermined speed, and wherein the displaying the pointer on the pointer position issued the calculated.

According to a fourth aspect of the present invention, in the pointing device according to the third aspect , the second pointing means is a time during which the moving speed of the pointer stored in the storage means is kept below a predetermined speed. A reference position is calculated from the pointer position, a pointer position is calculated at a position between the pointer position calculation means and the current pointer position, and the pointer is displayed at the calculated pointer position. To do.

Also, a fifth aspect of the present invention, the pointing device according to any one of claims 1 to 4, in any of the second pointing means and the first Ponting means by said display control means It further comprises identification display means for displaying whether or not it has been switched.

A pointing method according to a sixth aspect of the present invention is a pointing method for pointing an arbitrary position of a pointing area by a pointing device with a camera, and shooting with the camera directed to the pointing area, An image corresponding to the pointing area is detected from the obtained captured image, a corresponding position of the pointing area is calculated according to a relative relationship between the pointing area image and the captured image, and the calculated pointer position And the pointer is displayed at the calculated current pointer position, and the pointer position is calculated from the calculated current pointer position and the stored pointer position from a predetermined time before to the present time. , to display the pointer to the pointer position issued the calculated, before Detecting a moving speed of the pointer from the time variation of the relative relationship between the captured image and the pointing area image, in the normal mode in which the correction mode is canceled, the moving speed is below a predetermined speed detected by the moving speed detecting means A correction mode is set when a state satisfying the first condition is continued for a predetermined time or longer, and this correction mode is maintained until the second condition that the moving speed is greater than the predetermined speed is satisfied. When the condition 2 is satisfied, the correction mode is canceled and the normal mode is set, and the normal mode is maintained even when the first condition is satisfied for less than a predetermined time in the normal mode, and the correction is performed. One of the updating operations of the pointer display position is continuously performed a plurality of times between the mode being released and the correction mode being set.

A program according to a seventh aspect of the present invention is a program executed by a computer that controls a pointing device with a camera for pointing at an arbitrary position in a pointing area, wherein the camera is placed in the pointing area. The corresponding position of the pointing area is determined according to the function of detecting an image corresponding to the pointing area from the captured image obtained when shooting with the pointing and the relative relationship between the pointing area image and the captured image. A function to calculate, a function to store the calculated pointer position, a function to display a pointer at the calculated current pointer position, the calculated current pointer position, and the stored predetermined time Calculate pointer position from previous and current pointer positions And a function of displaying a pointer on the pointer position issued the calculated, a function of detecting a moving speed of said pointer from a time change in the relative relationship between the pointing region image and the captured image, and the correction mode is canceled In the normal mode, a correction mode is set when a state satisfying the first condition in which the moving speed detected by the moving speed detecting means is equal to or lower than a predetermined speed continues for a predetermined time or longer. The correction mode is maintained until a large second condition is satisfied, and when the second condition is satisfied, the correction mode is canceled and the normal mode is set, and the normal mode is set for less than a predetermined time in the normal mode. A function of maintaining the normal mode even when the condition 1 is satisfied, and while the correction mode is canceled and when the correction mode is set , Characterized in that to realize the function of performing continuously one of the pointer display position a plurality of times operation of updating.

  According to the present invention, when the operator is pointing to a small portion on the screen, the intended position can be accurately pointed by moving the pointer small.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram showing a configuration of a projection system using a pointing device according to the first embodiment of the present invention.

  The system includes a pointing device 11, a projector 12, and a PC (Personal Computer) 13. The projector 12 and the PC 13 are connected via a communication cable 14, and the projector 12 projects and displays the display screen of the PC 13 on the screen 15 as a projection target.

  Note that the connection form between the projector 12 and the PC 13 uses the communication cable 14 in the example of FIG. 1, but wireless communication can also be used. The screen 15 is, for example, a white board, and is installed to face the projector 12. Reference numeral 16 in the figure denotes a projected image of the PC screen projected onto the screen 15.

  The pointing device 11 is for performing a pointing operation at an arbitrary position on the screen 15. A camera 10 is installed at the tip of the pointing device 11. The current pointing position (designated position) is detected using an image captured by the camera 10, and the position signal is transmitted to the PC 13 by wireless communication or the like. Send it out. As a result, the PC 13 performs processing such as displaying the pointer P at a corresponding position on the screen based on the position signal sent from the pointing device 11.

  FIG. 2 is a block diagram showing a circuit configuration of the PC 13 and the pointing device 11 constituting the projection system.

  In this system, the PC 13 is used as a device for displaying an image for presentation. The PC 13 includes a CPU 21, a main storage device 22, a display storage device 23, an external storage device 24, a display device 25, an input device 26, a communication device 27, and the like. Further, the projector 12 is connected to the PC 13 via the communication cable 14.

  The CPU 21 performs overall control of the PC 13 and performs display processing of an image for presentation. The main storage device 22 is composed of a ROM or the like, and stores various data necessary for the processing operation of the CPU 21. The display storage device 23 stores an image for presentation. The external storage device 24 includes a hard disk device or the like.

  The display device 25 includes an LCD (Liquid Crystal Display) or the like, and displays an image for presentation read from the display storage device 23. The image displayed at this time is transferred to the projector 12 via the communication cable 14 and projected onto the screen 15 shown in FIG.

  The input device 26 includes a keyboard and is used when inputting various data and giving instructions. The communication device 27 transmits and receives data to and from the communication device 34 on the pointing device 11 side. Note that the communication form between the PC 13 and the pointing device 11 may be wired or wireless. However, as shown in FIG. 1, the pointing device 11 is operated at a location away from the PC 13, and thus wireless is preferable.

  On the other hand, the pointing device 11 is provided with a CPU 31, a main storage device 32, an operation unit 33, a communication device 34, and the camera 10.

  The CPU 31 controls the entire pointing device 11 and executes a series of processes related to a pointing operation described later. When the processing executed by the CPU 31 is functionally shown, as shown in FIG. 3, the CPU 31 includes an image detection unit 31a, a pointing unit 31b, a movement state detection unit 31c, and a display control unit 31d.

  The image detection unit 31 a detects an image corresponding to the pointing area from the captured image obtained by the camera 10. The pointing unit 31b displays a pointer P at a corresponding position in the pointing area in accordance with the relative relationship between the pointing area image detected by the image detection unit 31a and the captured image. The movement state detection unit 31c detects the movement state of the pointer P from the time change of the relative relationship. The display control unit 31d switches to the correction mode based on the movement state of the pointer P detected by the movement state detection unit 31c, and reduces the change in the display position of the pointer P with respect to the time change of the relative relationship. Correct the display position of.

  The main storage device 32 stores various data necessary for the operation of the CPU 31 in addition to a program 32a for realizing a pointing operation. The main storage device 32 is provided with a history storage unit 32b for storing the history of the coordinate position where the pointer P is displayed.

  The operation unit 33 is provided with various operation buttons including an ON / OFF button 33a for turning on / off the pointer function. The communication device 34 transmits and receives data to and from the communication device 27 on the PC 13 side.

  The camera 10 is provided at the tip of the pointing device 11. The camera 10 continuously performs photographing operations at predetermined intervals (for example, at intervals of 1/30 seconds) with the pointer function turned on.

  Hereinafter, in order to simplify the description, it is assumed that the projection image 16 of the PC screen is displayed on the entire projection surface of the screen 15 serving as a pointing area.

  FIG. 4 is a diagram showing the relationship between the photographed image and the screen image included therein, and FIG. 5 is a diagram showing the position of the pointer with respect to the screen.

  First, the display screen of the PC 13 is projected on the screen 15 via the projector 12. The operator performs a pointing operation with the camera 10 provided at the tip of the pointing device 11 facing the screen 15.

  As shown in FIGS. 4 and 5, if the center point of the image 41 taken by the camera 10 (the center of the angle of view of the camera 10) is O, the center point O is the position pointed to by the operator. If the center point O is within the area of the screen image 42, the pointer P is displayed at a corresponding position on the screen 15 through the PC 13.

  As described above, the pointing device 11 can recognize the position where the camera 10 is directed as the pointing position of the operator, and can display the pointer P at the corresponding position on the screen 15. However, when operating at a location away from the screen 15, as shown in FIG. 6, since the screen 15 is photographed small, even if the operator slightly changes the direction of the camera 10, the pointer P moves greatly. There is a problem that the intended position cannot be accurately pointed.

  Further, even if the distance to the screen 15 is appropriate, if the screen 15 is operated from an oblique direction, the screen 15 is distorted as shown in FIG. Pointing operation for is difficult.

  Further, even when the same standing position is used, the movement of the pointing operation varies depending on the location that the operator wants to point to. That is, as shown in FIG. 8, when a presentation PC screen is projected on the screen 15, when the description content portion 16a in the projected image 16 is roughly pointed, the positional accuracy of the pointer P is Not as necessary for each. Therefore, the operator can perform a pointing operation while moving the camera 10 (pointing device 11) relatively quickly.

  On the other hand, as shown in FIG. 9, for example, when the “close” button (“x” mark) 16b is pinpointed to close the window, the position accuracy of the pointer P is required. Therefore, the operator must perform an operation of accurately positioning the pointer P on the “x” mark 16b while slowly moving the camera 10 (pointing device 11).

  Hereinafter, a method for solving such a problem will be described.

  FIG. 10 is a flowchart showing the processing operation of the pointing device according to the first embodiment. Note that the processing shown in this flowchart is executed when the CPU 31 provided in the pointing device 11 reads a program 32a stored in the main storage device 32.

  First, the pointer function is turned on by operating an ON / OFF button 33 a provided on the operation unit 33 of the pointing device 11. In this state, when the camera 10 provided at the tip of the pointing device 11 is pointed at the screen 15, continuous shooting is performed at a predetermined frame rate, for example, 1/30 second, as shown in FIG. Then, a captured image 41 having a horizontal pixel number W and a vertical pixel number H is sequentially obtained (step A11).

  When the CPU 31 obtains the photographed image 41 through the camera 10, the CPU 31 detects a region of the screen image 42 that becomes a pointing region from the photographed image 41, and extracts its outline (step A12). Note that since a method for detecting a region of a specific portion from a captured image is known, the description thereof is omitted here.

  Here, as described with reference to FIGS. 4 and 5, if the center point of the captured image 41 (the center of the angle of view of the camera 10) is O, the center point O is the position pointed to by the operator. The CPU 31 converts the center point O into a point (Xi-1, Yi-1) on the coordinate system of the screen image 42, and if it is within the area of the screen image 42, issues an instruction to the PC 13 to The pointer P is displayed at the corresponding position on 15 (step A13).

  When the next captured image 41 is obtained after 1/30 seconds, the CPU 31 converts the center point O of the captured image 41 into a point (Xi, Yi) on the coordinate system of the screen image 42 as described above. The pointer P is displayed at the corresponding position on the screen 15 (steps A14 and A15). If the pointer function is turned off by the operation of the ON / OFF button 33a after the pointer P is displayed (Yes in step A16), the processing here ends.

  On the other hand, if the pointer function is in the ON state (No in step A16), the CPU 31 subsequently obtains the point (Xi-1, Yi-1) obtained in the previous photographing and the point obtained in the current photographing ( Xi, Yi) are compared, and the moving speed of the pointer P displayed on the screen 15 is calculated (step A17). After the movement speed is calculated, (Xi, Yi) is replaced with (Xi-1, Yi-1) in preparation for processing of the next frame image (step A18).

  Here, when the moving speed of the pointer P is equal to or lower than the predetermined speed and the state continues for a predetermined time (for example, 2 seconds) or longer (Yes in Step A19), the CPU 31 switches from the normal mode to the correction mode ( In step A19a), the following processing is executed. The “correction mode” is a mode for correcting the display position of the pointer P. Here, the movement of the pointer P corresponding to the movement of the camera 10 is made smaller than that in the normal mode. . The “predetermined speed” is determined by the average speed of the camera 10 (pointing device 11) when the operator points to a fine part as described with reference to FIG.

  When the correction mode is switched, the CPU 31 first stores the coordinates of each point (Xi, Yi) of the pointer P displayed while the state of the predetermined speed or lower continues, in the history storage of the main storage device 32. A position obtained by reading from the unit 32b and averaging them is determined as a reference point (Xref, Yref) (step A20).

  At that time, when a new captured image 41 is obtained after 1/30 seconds have elapsed since the previous capturing (step A21), the CPU 31 uses the captured image 41 and the previous captured image 41 in the same manner as described above. Thus, the moving speed of the pointer P is calculated (steps A22 and A23). As a result, if the moving speed of the pointer P is faster than the predetermined speed (No in Step A24), the CPU 31 cancels the correction mode (Step A24a), and the relative relationship between the current captured image 41 and the screen image 42 is obtained. The pointer P is displayed at the position (Xi, Yi) obtained from (Step A25). After the pointer P is displayed, (Xi, Yi) is replaced with (Xi-1, Yi-1) (step A26) in preparation for processing of the next frame image, and then the process returns to step A14.

On the other hand, when the moving speed of the pointer P as still has long followed the following state for a predetermined speed (Yes in step A24), CPU 31, the position obtained from this relative relationship of the captured image 41 and the screen image 42 (Xi, Yi ) And the reference point (Xref, Yref) obtained in step A20 is obtained as a corrected display position. If the display position is within the area of the screen image 42, the pointer P is displayed there (see FIG. Step A27). In addition, after the pointer P is displayed, (Xi, Yi) is replaced with (Xi-1, Yi-1) in preparation for processing of the next frame image (step A28) until the pointer function is turned off (step A28). Step A29) and the process from step A21 are repeated.

FIG. 12 shows the display position of the pointer P in the correction mode.
In the normal mode, the pointer P is displayed at the position of the point (Xi, Yi) obtained from the relative relationship between the captured image 41 and the screen image 42, whereas in the correction mode, the point (Xi, Yi) and the reference point ( The pointer P is displayed at the midpoint between Xref and Yref). Therefore, even if the camera 10 is moved greatly, the pointer P can be moved little by little. When returning to the normal mode, the pointing device 11 is temporarily moved quickly. Then, since the condition of the correction mode is not met, the normal mode is restored.

  As described above, according to the first embodiment, it is determined from the moving speed of the pointer P that the operator is pointing a fine portion on the screen 15 and the mode is switched to the correction mode. In the correction mode, the movement of the pointer P with respect to the movement of the camera 10 is reduced. Accordingly, as shown in FIG. 9, for example, when the “close” button (“x” mark) 16b of the PC screen projected on the screen 15 is clicked, the pointer P can be moved in small increments for easy pointing. become.

  The same applies to the case where the screen 15 is photographed small as shown in FIG. 6 or the screen 15 is photographed as distorted as shown in FIG. Thus, the intended position can be easily pointed.

  Further, in the first embodiment, if the pointer P is moved slowly, the correction mode is continued, so that there is an advantage that fine instruction operations can be continuously performed over a wide range.

  Further, if the pointer P is moved even temporarily, it returns to the normal mode, so that it can be easily used separately from the correction mode.

  When switching to the correction mode, the operator may be notified of this by lighting the lamp. The lamp is installed at a position where it can be easily seen, such as the upper part of the main body of the pointing device 11 shown in FIG. The light turns on when the correction mode is switched in step A19a, and turns off when the correction mode is canceled in step A19a. Thus, the operator can move the pointer P and perform a pointing operation while being aware of the state when the mode is switched to the correction mode.

  In the above embodiment, the continuous shooting time interval (frame rate) is 1/30 second, but continuous shooting may be performed at other time intervals.

  The determination speed for switching to the correction mode may be arbitrarily set, or the determination speed for switching to the correction mode and the determination speed for returning to the normal mode may be changed.

  Further, the display position of the pointer P in the correction mode is not limited to the midpoint (that is, the position divided into 1: 1) between the reference point (Xref, Yref) and the point (Xi, Yi), but is divided into, for example, 1: 2. It may be a position.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  In the first embodiment, the moving speed is detected as the moving state of the pointer P, and the mode is switched to the correction mode when the state of the predetermined speed or lower continues continuously. In contrast, in the second embodiment, the movement range is detected as the movement state of the pointer P, and the correction mode is switched when the movement range is equal to or smaller than a predetermined size.

  Since the system configuration of FIG. 1 and the apparatus configuration of FIG. 2 are the same, only the processing operation as the second embodiment will be described here.

  FIG. 13 is a flowchart showing the processing operation of the pointing device according to the second embodiment. Similar to the first embodiment, the processing shown in this flowchart is executed by the CPU 31 provided in the pointing device 11 reading the program 32 a stored in the main storage device 32.

  As in the first embodiment, when the camera 10 provided at the tip of the pointing device 11 is pointed toward the screen 15 in a state where the pointer function is turned on, a predetermined value such as 1/30 second is predetermined. Continuous shooting is performed at the frame rate, and a captured image 41 having a horizontal pixel number W and a vertical pixel number H is sequentially obtained (step B11). Then, the pointer P is displayed at the position (Xi, Yi) on the screen corresponding to the center point of the captured image 41 based on the relative relationship between the captured image 41 and the screen image 42 included therein (steps B12 to B15). . If the pointer function is turned off by the operation of the ON / OFF button 33a after the pointer P is displayed (Yes in step B16), the processing here is terminated.

  On the other hand, if the pointer function is on (No in step B16), the pointer P is continuously displayed using the captured images 41 sequentially obtained by continuous shooting. At that time, the position information of each point on which the pointer P is displayed is stored in the history storage unit 32 b of the main storage device 32.

  Here, in the second embodiment, switching from the normal mode to the correction mode is performed as follows.

  In other words, the CPU 31 reads the position (Xi, Yi) of the pointer P displayed during the elapse of T seconds from the activation of the pointer function of the pointing device 11 from the history storage unit 32b, and these positions (Xi, Yi). ) Is determined as the movement range of the pointer P (step B17).

FIG. 14 shows this state.
It is assumed that “X” in the figure indicates the position where the pointer P is displayed for T seconds. The range of the rectangle 43 connected by the four points (A, B), (C, B), (A, D), and (C, D) that can include these display positions is the movement range of the pointer P for T seconds. Show.

  Next, the CPU 31 determines whether or not the vertical and horizontal lengths of the rectangle 43 are shorter than the predetermined lengths V and L (step B18). These V and L are the determination conditions for the correction mode. When one of the vertical and horizontal lengths of the rectangle is longer than V and L, that is, when the movement range of the pointer P is wider than a predetermined range (No in step B18), the mode is switched to the correction mode. Instead, the pointer P is displayed in the normal mode as it is.

  On the other hand, when the vertical and horizontal lengths of the rectangle 43 are shorter than V and L, respectively, the CPU 31 switches from the normal mode to the correction mode (step B18a) and executes the following processing.

  That is, the CPU 31 first obtains the coordinates of the center point from the coordinates of the four points (A, B), (C, B), (A, D), and (C, D) constituting the rectangle 43, and there Is defined as a reference point (Xref, Yref) (step B19).

  Subsequently, when a new photographed image 41 is obtained after 1/30 seconds have elapsed since the previous photographing (step B20), the CPU 31 converts the center point O of the photographed image 41 into the screen coordinate system. (Xi, Yi) is obtained (step B21).

  Here, as shown in FIG. 15, a rectangle 44 having a vertical length of 2V and a horizontal length of 2L centered on the reference point (Xref, Yref) is set as a cancellation condition of the correction mode. The CPU 31 determines whether or not the current position (Xi, Yi) is included in the range of the rectangle 44 of 2V in length and 2L in width (step B22). If the current position (Xi, Yi) is outside the range of the rectangle 44 (No in Step B22), the CPU 31 cancels the correction mode (Step B22a), and from the relative relationship between the current captured image 41 and the screen image 42. After displaying the pointer P at the obtained position (Xi, Yi) (step B23), the process returns to step B14.

  On the other hand, if the current position (Xi, Yi) is within the range of the rectangle 44 (Yes in step B22), the CPU 31 determines the current position (Xi, Yi) and the reference point (Xref, Yref obtained in step B19). ) Is determined as the corrected display position, and if the display position is within the area of the screen image 42, the pointer P is displayed there (step B24). Further, after the pointer P is displayed, the processing from step B20 is repeated until the pointer function is turned off.

FIG. 15 shows the display position of the pointer P in the correction mode.
In the normal mode, the pointer P is displayed at the position of the point (Xi, Yi) obtained from the relative relationship between the captured image 41 and the screen image 42, whereas in the correction mode, the point (Xi, Yi) and the reference point ( The pointer P is displayed at the midpoint between Xref and Yref). Therefore, even if the camera 10 is moved quickly, the pointer P can be moved little by little. When it is desired to return to the normal mode, the pointing device 11 is largely moved so as to exceed the range of the rectangle 44. Then, since the condition of the correction mode is not met, the normal mode is restored.

  As described above, according to the second embodiment, it is determined from the movement range of the pointer P that the operator is pointing a fine portion on the screen 15. In the correction mode, the movement of the pointer P with respect to the movement of the camera 10 becomes small as in the first embodiment, so that a fine portion on the screen 15 can be easily pointed.

  In the second embodiment, if the pointer P is moved within a predetermined range, the correction mode continues regardless of the speed at which the pointer P is moved. Convenient.

  Further, if the pointer P is moved largely even temporarily, it returns to the normal mode, so that it can be easily used separately from the correction mode.

  Further, when switching to the correction mode, the operator may be notified of this by lighting the lamp or the like.

  In the example of FIG. 13, whether the history point of the pointer P falls within the rectangular range of vertical V and horizontal L is set as the determination condition for the correction mode (step B18), and the condition for exiting the correction mode is 2V, both vertically and horizontally. , 2L (step B22). This is because the ratio of the position where the pointer P is displayed after entering the correction mode is 1: 1, that is, the midpoint between the reference point (Xref, Yref) and the point (Xi, Yi). This is because whether or not to return to the normal mode is determined depending on whether or not is within the rectangular range of vertical V and horizontal L.

  However, the size of the rectangle in step B18, the size of the rectangle in step B22, and the ratio to the reference points (Xref, Yref) and (Xi, Yi) for displaying the pointer P in the correction mode are not this combination. Alternatively, the sizes of the three elements may be determined regardless of their size.

  Further, the size of the rectangle in step B18 or step B22 may be changed according to the ratio of the screen image to the captured image.

  When the correction mode is entered by combining the first and second embodiments, the determination is made based on the moving speed of the pointer P as in the first embodiment, and when returning to the normal mode, the second embodiment is performed. You may make it determine by whether the pointer P protrudes from the rectangular range decided beforehand like a form. On the contrary, it may be determined whether the history point of the pointer P falls within a predetermined rectangular range when entering the correction mode, and may be determined based on the moving speed of the pointer P when returning to the normal mode. .

  In short, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the respective embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, the method described in the above-described embodiment is a program that can be executed by a computer, such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD-ROM, etc.), a semiconductor memory, etc. The program can be written on a medium and applied to various apparatuses, or the program itself can be transmitted through a transmission medium such as a network and applied to various apparatuses. A computer that implements this apparatus reads a program recorded on a recording medium or a program provided via a transmission medium, and performs the above-described processing by controlling operations by this program.

FIG. 1 is a diagram showing a configuration of a projection system using a pointing device according to the first embodiment of the present invention. FIG. 2 is a block diagram showing a circuit configuration of a PC and a pointing device constituting the projection system in the embodiment. FIG. 3 is a block diagram showing a functional configuration of a CPU provided in the pointing device according to the embodiment. FIG. 4 is a diagram showing the relationship between the captured image and the screen image included therein in the embodiment. FIG. 5 is a diagram showing the position of the pointer with respect to the screen in the same embodiment. FIG. 6 is a diagram showing a relationship between a captured image and a screen image included therein when operating at a location away from the screen in the embodiment. FIG. 7 is a diagram illustrating a relationship between a captured image and a screen image included therein when the screen in the embodiment is operated from an oblique direction. FIG. 8 is a diagram showing an example in the case of indicating a rough part of the PC screen on the screen in the embodiment. FIG. 9 is a diagram showing an example in the case of pointing a fine portion of the PC screen on the screen in the embodiment. FIG. 10 is a flowchart showing the processing operation of the pointing device in the embodiment. FIG. 11 is a diagram for explaining a relationship between a captured image and a screen image obtained by the pointing device according to the embodiment. FIG. 12 is a view for explaining the display position of the pointer P in the correction mode of the pointing device according to the embodiment. FIG. 13 is a flowchart showing the processing operation of the pointing device according to the second embodiment of the present invention. FIG. 14 is a diagram for explaining the relationship between the captured image obtained by the pointing device and the history point of the pointer P in the embodiment. FIG. 15 is a view for explaining the display position of the pointer P in the correction mode of the pointing device according to the embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Camera, 11 ... Pointing device, 12 ... Projector, 13 ... PC, 14 ... Communication cable, 15 ... Screen, 16 ... Projection image, 21 ... CPU, 22 ... Main storage device, 23 ... Display storage device, 24 ... External Storage device, 25 ... Display device, 26 ... Input device, 27 ... Communication device, 31 ... CPU, 32 ... Main storage device, 32a ... Program, 32b ... History storage unit, 33 ... Operation unit, 33a ... ON / OFF button, 34 ... Communication device, 41 ... Captured image, 42 ... Screen image, 43 ... Rectangular, 44 ... Rectangular, O ... Center point of captured image (center of camera angle of view), P ... Pointer.

Claims (7)

A pointing device for pointing at an arbitrary position in a pointing area,
Photographing means for photographing the pointing area;
Image detecting means for detecting an image corresponding to the pointing area from a photographed image obtained by the photographing means;
Pointer position calculating means for calculating a corresponding position of the pointing area according to the relative relationship between the pointing area image detected by the image detecting means and the captured image;
Storage means for storing the pointer position calculated by the pointer position calculation means;
First pointing means for displaying a pointer at the current pointer position calculated by the pointer position calculating means;
The pointer position is calculated from the current pointer position calculated by the pointer position calculating means and the pointer position from a predetermined time before and stored in the storage means, and the pointer is placed at the calculated pointer position. A second pointing means for displaying;
A moving speed detecting means for detecting a moving speed of the pointer from a temporal change in a relative relationship between the pointing area image and the captured image;
In normal mode the correction mode is canceled, and sets the correction mode if the first condition is satisfied state moving speed detected by the moving speed detecting means is below a predetermined speed continues for a predetermined time or longer, the The correction mode is maintained until the second condition that the moving speed is greater than the predetermined speed is satisfied. When the second condition is satisfied, the correction mode is canceled and the normal mode is set, and the normal mode is set. Correction mode switching means for maintaining the normal mode even when the first condition is satisfied for less than a predetermined time,
While the correction mode is cancelled, the pointer display position update operation by the first pointing means is continuously performed a plurality of times, and while the correction mode is set, the pointer display by the second pointing means is performed. And a display control means for continuously performing a position updating operation a plurality of times. The first Ponting means has the same change in the display position of the pointer with respect to the time change in the relative relationship between the pointing area image and the captured image,
The pointing device according to claim 1, wherein the second pointing unit reduces a change in the display position of the pointer with respect to a time change in a relative relationship between the pointing area image and the captured image. Before Stories second pointing means comprises: a current pointer position calculated by the pointer position calculation means, the pointer position of the time the moving speed of the pointer stored by the storage means continues being below a predetermined speed It calculates the pointer position from a pointing device according to claim 1 or 2, characterized in that to display the pointer in the pointer position issued the calculated.   The second pointing means obtains a reference position from the pointer position at a time when the moving speed of the pointer stored in the storage means continues below a predetermined speed, and calculates the current position calculated by the pointer position calculating means. 4. The pointing device according to claim 3, wherein the pointer position is calculated at a position between the pointer position and the pointer is displayed at the calculated pointer position.   5. The apparatus according to claim 1, further comprising: an identification display unit that displays in an identifiable manner whether the first control unit or the second pointing unit has been switched by the display control unit. The pointing device according to one item. A pointing method for pointing an arbitrary position of a pointing area by a pointing device with a camera,
Shoot with the camera pointing to the pointing area,
Detecting an image corresponding to the pointing area from a captured image obtained by the camera;
According to the relative relationship between the pointing area image and the captured image, the corresponding position of the pointing area is calculated,
Storing the calculated pointer position;
Displaying a pointer at the calculated current pointer position;
Calculating the pointer position from the calculated current pointer position and the stored pointer position from a predetermined time before to the present time, and displaying the pointer at the calculated pointer position;
Detecting the moving speed of the pointer from the time change of the relative relationship between the pointing area image and the captured image;
In normal mode the correction mode is canceled, and sets the correction mode if the first condition is satisfied state moving speed detected by the moving speed detecting means is below a predetermined speed continues for a predetermined time or longer, the The correction mode is maintained until the second condition that the moving speed is greater than the predetermined speed is satisfied. When the second condition is satisfied, the correction mode is canceled and the normal mode is set, and the normal mode is set. Even if the first condition is satisfied for less than a predetermined time, the normal mode is maintained,
The pointing method, wherein the pointer display position update operation is continuously performed a plurality of times while the correction mode is canceled and while the correction mode is set. A program executed by a computer for controlling a pointing device with a camera for pointing at an arbitrary position in a pointing area,
In the computer,
A function of detecting an image corresponding to the pointing area from a captured image obtained when shooting with the camera facing the pointing area;
A function of calculating a corresponding position of the pointing area according to a relative relationship between the pointing area image and the captured image;
A function of storing the calculated pointer position;
A function of displaying a pointer at the calculated current pointer position;
A function of calculating a pointer position from the calculated current pointer position and the stored pointer position from a predetermined time before to the present time, and displaying a pointer at the calculated pointer position;
A function of detecting a moving speed of the pointer from a temporal change in a relative relationship between the pointing area image and the captured image;
In normal mode the correction mode is canceled, and sets the correction mode if the first condition is satisfied state moving speed detected by the moving speed detecting means is below a predetermined speed continues for a predetermined time or longer, the The correction mode is maintained until the second condition that the moving speed is greater than the predetermined speed is satisfied. When the second condition is satisfied, the correction mode is canceled and the normal mode is set, and the normal mode is set. A function for maintaining the normal mode even when the first condition is satisfied for less than a predetermined time;
A program that realizes a function of continuously updating any one of the pointer display positions a plurality of times while the correction mode is canceled and while the correction mode is set. .

Images