JP2005215828A - Pointing device and method for displaying point image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for displaying a point image on a projected image for making it unnecessary to perform any complicate positioning, and for realizing the position control of the point image by an image indicating device oriented to an arbitrary place without using any laser which should not be directly viewed. <P>SOLUTION: This pointing device is configured to display the point image 103 on a screen 102 by operating an image indicating device 101, and to detect the movement of the image in an imaging visual field 104 picked up by the image indicating device 101, and to calculate the moving direction and movement quantity of the point image 103 corresponding to the movement of the image, and to compound the point image 103 with a display image projected on the screen 102 from a projecting device 105 based on the calculation result, and to control the position of the point image 103 corresponding to the change of the orientation of the image indicating device 101. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for controlling a point image displayed on a projected image or the like, and to a technique for controlling a display position of a point image using an image captured by an imaging device provided in an image instruction apparatus.

  A method of projecting an image on a screen and giving a presentation is known. At this time, a technique is known in which a point image (for example, an image of a light spot or an arrow) pointing to a specific image is superimposed and displayed on a screen.

  As such a technique, the techniques described in Patent Documents 1 to 4 are known. In Patent Document 1, in a configuration in which a point image is superimposed on an image projected on a screen from an image projection device, an ultrasonic wave is generated from an image instruction device for operating the point image, and the ultrasonic wave is provided on the screen side. A technique is described in which a point detected by an image sensor, a position pointed to by an image pointing device is specified, and a point image is displayed on a screen.

  Patent Documents 2 and 3 control the display position of a point image by irradiating infrared or instruction light from an image pointing device in the same technical field as Patent Document 1 and specifying the irradiation position on the screen. The technology is described.

  Patent Document 4 describes a technique for projecting a reference position for coordinate detection on a screen in a technique for irradiating an infrared ray from an image pointing device onto a screen and determining a display position of a point image.

Japanese Patent Laid-Open No. 2002-207466 Japanese Patent Laid-Open No. 11-271675 Japanese Patent Laid-Open No. 11-305940 Japanese Patent Laid-Open No. 11-85395

  However, the method using ultrasonic waves described in Patent Document 1 requires a receiving device that detects ultrasonic waves on the screen side, which increases the number of necessary devices and complicates the mechanism. In addition, the methods using infrared rays and indicator light described in Patent Documents 2 to 4 also require devices and mechanisms for detecting infrared rays and indicator light, and have the same problems as when ultrasonic waves are used.

  Further, in the above-described conventional technology, it is necessary to align the position of the point image with the image instruction device in advance. However, there are cases where presentations are performed in various setting environments, and it is troublesome and inconvenient to perform alignment each time it is used. Further, in the above-described prior art, the position of the point image cannot be controlled unless the image pointing device is directed toward the screen. Keeping the image pointing device facing the screen during explanation may limit the posture, body orientation, or body movement of the person performing the explanation (especially the movement of the hand holding the image pointing device). It was.

  The present invention solves the above-described problems in the conventional point image control technology, does not increase the number of devices as much as possible, has a simple configuration, does not require complicated alignment, and does not point the image pointing device on the screen. It is another object of the present invention to provide a technique capable of controlling a point image.

  The pointing device of the present invention calculates the movement direction and the movement amount of the point image corresponding to the movement of the image, the image movement detection means for detecting the movement of the image taken by the imaging device provided in the image pointing device. A calculation unit; and a signal generation unit configured to generate a signal for synthesizing the point image into the display image based on a calculation result of the calculation unit, and the display position of the point image is set to a directivity of the image instruction device. It is characterized by being moved corresponding to the change in direction.

  In the pointing device of the present invention, it is preferable that imaging is performed by a CCD camera provided in the image pointing device. The imaging means using a CCD camera is small and has high resolution, and has the convenience that the image pointing device can be used as a simple camera.

  According to the pointing device of the present invention, the movement direction and the movement amount of the point image are calculated from the movement of the image captured by the image pointing device (the movement of the image within the imaging field of view), and thereby the position of the point image is controlled. . In the present invention, the change in the pointing direction of the image pointing device is detected by the movement of the image picked up by the image pointing device in the imaging field of view. By moving the display position of the point image based on the detection result, the display position of the point image can be moved in accordance with the change in the pointing direction of the image pointing device.

  In the pointing device of the present invention, since the movement of the image captured by the image pointing device is detected, no sensor or the like is required other than the image sensor provided in the image pointing device, and the entire system can be simplified. Further, since the relative movement of the pointing position of the image pointing device is detected, the position of the point image may be controlled while the image pointing device is pointed at any appropriate place, and a complicated alignment operation is not required. Further, since the movement of the point image is controlled based on the relative movement of the image within the imaging field of view, the image pointing device may be pointed anywhere, and the degree of freedom during use is increased. Thereby, for example, when used for presentation, the presenter's posture, body orientation, and restrictions on body movement are reduced.

  In the pointing device of the present invention, the image movement detection means stores the first image data at at least one of the plurality of monitoring points set in the imaging screen at a predetermined time, and after the predetermined time, Comparing the second image data obtained from the plurality of monitoring points with the stored first image data, and detecting a difference between the first and second image data from the comparison; It is preferable to perform a process of calculating the moving direction and moving amount of the captured image from the difference between them.

  According to this aspect, by comparing the image data acquired with a time difference at the monitoring point, the movement of the captured image can be detected, and the movement direction and the movement amount can be calculated. According to this aspect, since only the images at the monitoring points set at a plurality of predetermined positions in the captured image need be analyzed, the amount of information to be handled can be reduced. For this reason, both low cost and reduction of processing time can be pursued. In particular, since the processing time can be reduced, the movement of the point image that smoothly follows the movement of the image pointing device can be realized.

  Further, in the processing by the image movement detection means, the monitoring point is preferably a set of pixels divided in a matrix. According to this aspect, since the image data at the monitoring point is handled as the dot information of the pixels arranged in a matrix, the processing of the pixel data can be simplified.

  The pointing device of the present invention preferably includes position control means for displaying the point image at a predetermined position in the display image regardless of the calculation result of the calculation means. According to this aspect, the point image can be initially displayed at a predetermined location regardless of the pointing position of the image instruction device. This function can be easily used when, for example, the point image is lost. If this function is used, a complicated alignment operation is not required, which is convenient.

  The pointing device of the present invention preferably includes a graphical user interface (GUI) operation means using a point image. According to this aspect, the personal computer can be operated using the graphical user interface on the screen projected on the screen by the projection device, for example.

  In other words, in a presentation using an image projected on a large screen, a graphical user interface operation is performed using a point image, and reference processing of various materials can be easily performed by clicking on the projection screen. it can.

  In the pointing device of the present invention, it is preferable that the pointing device includes an adjusting unit that adjusts the moving amount of the point image on the display image with respect to the moving amount of the image. According to this aspect, the amount of movement of the point image corresponding to the amount of movement of the pointing position of the image pointing device can be adjusted. This function is useful in that it can set the amount of movement of the point image according to each person's feeling corresponding to the difference in how the image pointing device moves depending on the person.

  The invention relating to the pointing device of the present invention can also be grasped as a point image display method. That is, the point image display method of the present invention includes an image movement detection step for detecting movement of a captured image, a calculation step for calculating a movement direction and a movement amount of the point image corresponding to the movement of the image, And a signal generation step of generating a signal for synthesizing the point image into the display image based on the calculation result of the calculation step.

  According to the present invention, the movement of the pointing target location of the image pointing device is detected from the movement of the image captured by the imaging means of the image pointing device, and the moving direction and the moving amount of the point image are calculated from the result. Less equipment is required and the configuration can be simplified. Further, since the image to be detected is not particularly limited, the point image can be controlled without the image pointing device being directed on the screen. That is, according to the present invention, there is provided a technology that can increase the number of devices as much as possible, has a simple configuration, does not require complicated alignment, and can control a point image without pointing the image pointing device on the screen. The

(First embodiment)
1-1. Configuration of Embodiment FIG. 1 is a conceptual diagram showing an outline of a presentation system using a pointing device of the present invention. In FIG. 1, there are an image instruction device 101, a screen 102, a point image 103, an imaging target 104 of the image instruction device 101, a projection device 105, a presenter (person who makes a presentation) 106, a point image control device 107, and a personal computer 108. It is shown.

  In this system, when the presenter 106 moves the image instruction apparatus 101, the point image 103 projected on the screen moves corresponding to the movement. That is, the movement of the image captured by the image instruction apparatus 101 within the imaging field of view is analyzed by the point image control apparatus 107, the movement direction and movement amount of the point image 103 are calculated, and the display position of the point image 103 is controlled accordingly. Is done.

  The image instruction device 101 captures an arbitrary imaged object directed by the presenter 106, sends the image to the point image control device 107, and sends various control signals to the point image control device 107.

  The personal computer 108 stores an image of the content to be presented that has been processed and created using appropriate application software, and outputs the image data to the point image control device 107 in accordance with a predetermined operation.

  The point image control device 107 generates a signal for controlling the position of the point image 103 on the screen 102 based on the signal sent from the image instruction device 101. Further, the point image control apparatus 107 performs a process of superimposing the point image on the image sent from the personal computer 108.

  FIG. 2 is a block diagram illustrating an example of the configuration of the image instruction device 101. 2 includes an imaging device 111, an image signal generation device 112, a control switch 113, a position reset switch 114, a movement amount adjustment dial 115, a control signal generation device 116, and a signal output device 117.

  The projection device 105 is a liquid crystal projector, a three-tube projector, or the like, and has a function of projecting an image signal sent from the point image control device 107 onto the screen 102.

  The imaging device 111 is a camera provided with a CCD imaging device. The image signal generation device 112 converts an image captured by the image sensor 111 into an appropriate electrical signal (image signal) suitable for transmission. The control switch 113 is a switch used when switching to a mode in which a captured image is directly projected, or when using a mouse function described later. For example, a function corresponding to a right click and a left click of the mouse can be assigned as a control switch. The position reset switch 114 is a switch for initial setting of the position of the point image 103. For example, when the position reset switch 114 is operated, the point image 103 can be forcibly displayed at the center of the screen 102.

  The movement amount adjustment dial 115 is a dial that adjusts the relationship between the change amount of the pointing direction or the pointing position of the image pointing device 101 and the movement amount of the point image 103 on the screen. For example, when adjustment is made so that the amount of movement of the point image 103 on the screen changes greatly compared to the amount of movement of the pointing position of the image instruction device 101, the point image 103 is moved only by moving the image instruction device 101 slightly. Can be set to move greatly.

  The control signal generation device 116 has a function of converting the operation contents of the control switch 113, the position reset switch 114, and the movement amount adjustment dial 115 into an electrical signal suitable for transmission to the point image control device 107. The signal output device 117 has a function of transmitting the electric signal generated by the image signal generation device 112 and the control signal generation device 116 to the point image control device 107 as a radio wave signal.

  FIG. 3 is a block diagram showing an example of the configuration of the point image control device 107 (see FIG. 1). The point image control device 107 shown in FIG. 3 includes a reception device 121, a signal separation device 122, an image capture device 123, a movement amount adjustment device 124, an initial setting position control device 125, an image analysis device 126, and a point image control signal generation device 127. The image synthesizing device 128 and the signal output device 129 are provided.

  The receiving device 121 has a function of receiving a radio signal from the image instruction device 101. The signal separation device 122 has a function of separating an image signal and various control signals included in the signal received by the reception device 121 and guiding them to a predetermined device. The movement amount adjustment device 124 has a function of receiving a control signal reflecting the adjustment contents of the movement amount adjustment dial 115 of the image instruction device 101 and adjusting the ratio of the movement amount of the point image 103. For example, the movement amount adjustment device 124 has a function of arbitrarily multiplying the movement amount corresponding to the movement of the image captured by the image instruction device 101 with respect to a predetermined reference value, and the adjustment amount dial 115 (FIG. 2). It has a function of changing the numerical value to be arbitrarily multiplied in accordance with the setting condition of (see).

  The image capturing device 123 has a function of capturing an image captured by the image capturing device 111 (see FIG. 2). The image analysis circuit 126 analyzes an image captured by the image capturing device 123 (an image captured by the image capturing device 111), and calculates a moving direction and a moving amount within the imaging field of view. The image analysis method will be described in detail later.

  The point image control signal generation device 127 uses the movement direction and movement amount of the point image 103 based on the movement direction and movement amount of the image captured by the image instruction device 101 calculated by the image analysis device 126 in the imaging field of view. Further, coordinate data of the position where the point image is to be displayed is generated based on the calculation result.

  In addition, the point image control signal generation device 127 responds to the signal from the movement amount adjustment device 124 so that the movement amount of the point image 103 becomes a value corresponding to the operation of the movement amount adjustment dial 115 (see FIG. 2). Perform processing. Further, when the point image control signal generation device 127 receives a signal for forcibly displaying the point image 103 at a predetermined position from the initial setting position control device 125, the point image control signal generation device 127 displays the point image 103 on the screen 102. Processing for displaying the position is performed.

  The image synthesizer 128 generates image data for displaying a point image at a position determined by the coordinate data generated by the point image control signal generator 127, and the image data is stored in the personal computer 108 (see FIG. 1). ) Is combined with the image sent from. By this processing, a composite image is generated in which the point image is displayed at a predetermined position in the image sent from the personal computer 108 (see FIG. 1).

  The signal output device 129 has a function of sending the image signal processed by the image composition device 128 to the projection device 105 (see FIG. 1). The signal output device 129 has a function of transmitting the image signal separated by the signal separation device 122 to the projection device 105.

  In the present embodiment, an optical zoom function in the projection device 105 may be operated by the image instruction device 101. In this case, the image instruction device 101 further includes a zoom adjustment operation switch, and the point image control device 107 further includes a zoom adjustment signal generation device. In this aspect, by operating the zoom adjustment operation switch of the image instruction device 101, a signal reflecting the operation result is sent to the point image control device 107, and the zoom adjustment of the point image control device 107 is received in response to the signal. In the signal generation device, a control signal for controlling the optical zoom function of the projection device 105 is generated, and the optical zoom function of the projection device 105 is controlled by the control signal.

1-2. Operation of Embodiment First, an example of the operation of the image instruction apparatus 101 will be described. FIG. 4 is a flowchart illustrating an example of the operation of the image instruction apparatus 101. First, it is determined whether the use of the image instruction device 101 is started, that is, whether an imaging start switch (not shown) is ON (step S11). If the switch is ON, imaging by the imaging device 111 (see FIG. 2) is performed. (Step S12), otherwise, step S11 is repeated.

  2 is converted into an image signal by the image signal generation device 112 and transmitted as a radio signal from the signal output device 117 to the point image control device 107 (see FIG. 1) (step). S13).

  The above processing is repeatedly performed in a state where the image instruction apparatus 101 is used. Then, the image signal of the imaging target 104 captured by the imaging device 111 of the image instruction device 101 is continuously transmitted to the point image control device 107.

  Next, an example of the operation of the point image control apparatus 107 will be described. FIG. 5 is a flowchart illustrating an example of image processing performed by the image analysis device 126 and the point image control signal generation device 127 of FIG. FIG. 6 is a conceptual diagram for explaining an example of an image analysis method. FIG. 6 shows an imaging field 133 that is imaged by the image instruction apparatus 101 (see FIG. 1). This imaging field 133 corresponds to the imaging object 104 in FIG. In FIG. 6, an arrow 132 indicates that the specific image 131 in the imaging visual field 133 is located at the position shown in FIG. 6A in accordance with the movement of the pointing direction by the image pointing device 101 according to the operation of the presenter 106. An example in the case of moving in the direction shown and moving to the position shown in FIG. 6B is shown.

  FIG. 6 shows an example in which five monitoring points 134 to 138 are provided in the imaging visual field 133. The monitoring points 134 to 138 are used to detect the movement of the captured image in the imaging visual field 133. That is, the image in the imaging visual field 133 is partially cut out at five positions by the monitoring points 134 to 138, and the data of the cut out images are compared at a predetermined time interval, whereby the imaging device 111 of the image pointing device 101 is compared. The movement of the captured image captured by is detected.

  The monitoring points 134 to 138 are divided into Xm × Yn dots (pixels) in a lattice shape. FIG. 6 shows an example in which the monitoring points 135 to 138 are divided into a 5 × 5 pixel matrix as an example. By these monitoring points 134 to 138, five parts are cut out from the captured image. Note that m and n are natural numbers excluding 0.

  Part of the captured image cut out at each monitoring point is represented by table data stored in the storage area (Xm, Yn). This table data becomes image data for specifying an image at each monitoring point.

  For example, in the example shown in FIG. 6, when a monochrome image is targeted, the image data at each of the monitoring points 134 to 138 is represented as 5 × 5 table data storing 0 or 1 pixel data.

  The monitoring points 134 to 138 are used to detect the movement of the captured image and to obtain basic data for calculating the moving direction and the moving amount of the image. As will be described in detail later, the moving direction and the moving amount of the image are calculated by analyzing temporal changes in the image data at the five monitoring points. The method for setting the monitoring point eliminates the need to handle image data relating to the entire image, thereby shortening the processing time required for the analysis and simplifying the hardware and software programs required for the analysis. Note that the setting of the monitoring points is not limited to the example shown in FIG. 6, and other forms may be employed for the number of setting locations and the setting positions.

  Hereinafter, an example of processing for detecting the movement of the captured image using the monitoring points 134 to 138 will be described. The image processing described below is performed in the image analysis device 126 and the point image control signal generation device 127 in FIG.

  First, image data (first image data) at the monitoring points 134 to 138 at a predetermined time is stored (step S21). The image data is stored in a memory (not shown) provided in the image analysis device 126.

  Image data is stored repeatedly according to a predetermined sampling frequency. Therefore, the image data (second image data) at the monitoring points 134 to 138 is stored again after a predetermined time has elapsed from the execution of step S21 (step S22).

  Next, the first image data and the second image data are compared to determine whether there is a difference in pixel data at a predetermined monitoring point (step S23). In step S23, the first image data and the second image data at each monitoring point are compared. That is, the first image data and the second image data at the monitoring point 134 are compared, the first image data at the monitoring point 135 and the second image data are compared, and the first image data at the monitoring point 136 is compared. A process of comparing the data and the second image data is performed.

  If there is at least one difference between the first image data and the second image data at the monitoring point, the process proceeds to step S24. Otherwise, the process returns to the previous stage of step S22. For example, if the presenter 106 shown in FIG. 1 does not move the image pointing device 101, the captured image does not move. Therefore, the first image data and the second image data described above are the same at each monitoring point, and step S23 is performed. The determination of NO is NO, and the processing after step S22 is repeated again.

  In step S24, it is determined whether there is a match between pixel data at different monitoring points. That is, it is determined whether the first image data of a certain monitoring point matches the second image data of another monitoring point. This determination is made for each monitoring point. For example, the first image data at the monitoring point 134 and the second image data at the monitoring points 135 to 138 are compared to determine whether there is a match between them, and the first image at the monitoring point 135 is further determined. The data and the second image data at the monitoring points 134 and 136 to 138 are compared, and it is determined whether there is a match between the data and the monitoring point 134, 136 to 138.

  If there is a match between the image data at different monitoring points, the process proceeds to step S25, and if not, the process returns to the previous stage of step S22.

  In step S25, the moving direction and the moving amount of the captured image are calculated from the positions of the two monitoring points that match the image data. Based on the movement direction and movement amount of the captured image calculated in step S25, the point image control signal generation device 127 (see FIG. 3) calculates the movement direction and movement amount of the point image (step). S26). The above process is repeatedly performed, and the position control of the point image according to the motion of the captured image is performed.

  Hereinafter, a more specific example of the position control of the point image will be described. FIG. 6 shows an example of a state in which the specific image 131 has moved from the position shown in FIG. 6A to the position shown in FIG. 6B. Here, it is assumed that the first image data is stored (step S21) in the state of FIG. 6A, and the second image data is stored (step S22) in the state of FIG. 6B. .

  In this case, the image data of the monitoring point 135 in FIG. 6A is different from the image data of the monitoring point 135 in FIG. In addition, the image data of the monitoring point 136 in FIG. 6A is different from the image data of the monitoring point 136 in FIG. Further, the image data of the monitoring point 137 in FIG. 6A is different from the image data of the monitoring point 137 in FIG. Therefore, the determination in step S23 is YES.

  Then, the image data of the monitoring point 135 in FIG. 6A matches the image data of the monitoring point 136 in FIG. 6B. In addition, the image data of the monitoring point 136 in FIG. 6A matches the image data of the monitoring point 137 in FIG. 6B. That is, the first image data and the second image data match at different monitoring points. Therefore, the determination in step S24 is YES.

  Then, the movement of the specific image 131 as indicated by the arrow 132 from the position shown in FIG. 6A to the position shown in FIG. That is, the moving direction and moving amount of the specific image 131 are calculated. Since the specific image 131 is a part of the captured image, the movement direction and the movement amount of the captured image are calculated by calculating the movement direction and the movement amount of the specific image. In this way, the moving direction and the moving amount of the captured image are calculated in the image analysis device 126 shown in FIG.

  As illustrated in FIG. 6, when the captured image moves in the direction of the arrow 132, the pointing position of the image pointing device 101 is moved in a direction opposite to the direction. For example, when the direction of the image pointing device 101 is moved so that the imaging target 104 moves in the direction of the arrow 109 in FIG. 1, the captured image captured by the imaging device 111 is in a direction different from the arrow 109 by 180 degrees. It moves in the direction of an arrow 132 (see FIG. 6). By utilizing this relationship, the point image control signal generation device 127 (see FIG. 3) performs the process of step S26, and the movement direction 110 and the movement amount of the point image 103 (see FIG. 1) are calculated. . Based on the calculation result, a signal for designating the display position of the point image is generated, and the point image is synthesized with predetermined coordinates in the image signal sent from the personal computer 108 in the image synthesis device 128.

As described above, when the presenter 106 changes the orientation of the image pointing device 101 and the pointing position moves in the direction of the arrow 109, the captured image of the imaging device 111 moves in the direction of the arrow 132 in FIG. The moving direction and moving amount of the captured image are calculated from the comparison of the image data between the monitoring points 134 to 138, and the moving direction and moving amount of the point image 103 are calculated based on the result, as indicated by the arrow 110. The display position of the point image 103 is controlled.

  The method of detecting the movement of the captured image using the monitoring points 135 to 138 described above has the advantage that the amount of data to be handled is small, so that the calculation speed can be increased and the responsiveness can be improved. . Therefore, the presenter can make a presentation using the point image 103 without feeling stress. Further, since the amount of data to be handled is small and the calculation is relatively simple, the necessary hardware is simple, and the cost and reliability can be improved.

  As a method for analyzing an image captured by the image instruction device 101 and detecting a change in a direction (or a pointing position) directed by the image instruction device 101, a feature point of the captured image (for example, brightness of an edge or the like) It is also possible to adopt a method of detecting a specific image by detecting a hue change point) and detecting movement of the image in the imaging field of view.

  In the present embodiment, the movement amount of the point image 103 can be adjusted by a movement amount adjustment device 124 (see FIG. 3), and the point image 103 corresponding to the change in the pointing position or the swing angle of the image instruction device 101. The amount of movement on the screen 102 can be arbitrarily adjusted.

  This adjustment is performed by operating the movement amount adjustment dial 115 (see FIG. 2) of the image instruction apparatus 101. That is, when the movement amount adjustment dial 115 is adjusted, a signal reflecting the adjustment content is generated by the control signal generation device 116 and sent from the image instruction device 101 to the point image control device 107. This control signal is sent from the signal separation device 122 to the movement amount adjustment device 124 via the reception device 121. The movement amount adjustment device 124 sets the movement amount of the point image 103 according to the operation state of the movement amount adjustment dial 115, and sends a signal designating the content to the point image control signal generation device 127.

  According to this aspect, on the screen 102 of the point image 103 according to the amount of movement of the imaging target 104 of the image instruction apparatus 101 according to the habit of how the image instruction apparatus 101 is moved by the presenter 106 shown in FIG. The amount of movement can be adjusted.

  In the present embodiment, the position of the point image 103 on the screen 102 can be forcibly aligned at an arbitrary timing. That is, the position of the point image 103 on the screen 102 can be forcibly displayed at the center of the screen.

  Hereinafter, a detailed example of this operation will be described. When the presenter 106 (see FIG. 1) operates the position reset switch 114 (see FIG. 2) of the image indicating device 101, a signal indicating that is generated by the control signal generating device 116, and the signal is sent from the signal output device 117. It is transmitted to the point image control device 107 (see FIGS. 1 and 3). The signal is received by the receiving device 121 of the point image control device 107 and sent to the initial setting position control device 125 through the signal separation device 122. In response to the signal, the initial setting position control device 125 sends a signal for displaying the point image 103 at a predetermined position on the screen 102 (for example, the center of the screen 102) to the point image control signal generation device 127. In response to this signal, the point image control signal generation device 127 generates a signal for displaying the point image 103 at a predetermined position on the screen 102 and sends it to the image composition device 128. Then, the image composition device 128 creates a composite image in which the point image 103 is displayed at the center of the image sent from the personal computer 108, and sends the composite image signal to the projection device 105. The projection device 105 projects this composite image on the screen 102. Thus, the point image 103 is forcibly displayed at the center of the screen 102.

  According to this aspect, the presenter 106 can initialize or reset the position of the point image 103 on the screen 102 at an arbitrary timing. With this function, when the presenter 106 (see FIG. 1) loses sight of the point image 103, a reset operation can be performed to redisplay the position of the point image at a predetermined position. This operation can also be used to set the initial position of the point image 103 when starting a presentation.

  In the configuration shown in FIG. 1, an image captured by the image instruction device 101 can be projected on a screen. In this case, in the image instruction device 101 of FIG. 2, the control switch 113 is operated to switch to the image capturing mode. The image picked up by the image pickup device 111 is converted into an image signal by the image signal generation device 112, and the signal is sent from the signal output device 117 to the point image control device 107 shown in FIG. In the point image control apparatus 107 shown in FIG. 3, the signal is received by the reception apparatus 121 and sent to the projection apparatus 105 (see FIG. 1) via the signal separation apparatus 122 and the signal output apparatus 129. An image captured by the image instruction device 101 is projected from the projection device 105 onto the screen 102. This aspect is effective when an image obtained by capturing a sample or the like is projected on a screen in a presentation.

  It is also possible to superimpose an image captured by the image instruction device 101 on an image from the personal computer 108 and project the composite image on the screen 102. In this case, the image signal from the image instruction device 101 received by the reception device 121 is received by the reception device 121 of the point image control device 107 and sent from the signal separation device 122 to the image synthesis device 128, where the image synthesis is performed. Is done.

(Second Embodiment)
2-1. Configuration of Embodiment In this embodiment, a point image position control technique using the present invention is combined with a graphical user interface (GUI) function provided in a personal computer. The graphical user interface refers to a user interface that uses a lot of graphics for displaying information to the user and can operate application software with a pointing device such as a mouse.

  FIG. 7 is a conceptual diagram showing an outline of another presentation system using the pointing device of the present invention. FIG. 8 is a block diagram showing an example of the configuration of the point image control apparatus 201 in FIG.

  This embodiment is an example in which the present invention is applied to a system that projects and displays a point image on a screen by using a control function of a point image (mouse pointer) normally provided in a personal computer.

  FIG. 7 shows a presenter 301, an image instruction device 302, an imaging target 303 imaged by the image instruction device 302, a point image control device 201, a personal computer 304, a USB (Universal Serial Bus) cable 305, and an image transmission cable 306. A display 307, a projection device 308, a screen 309 and a point image 310 of the personal computer 304 are shown.

  In the present embodiment, an image captured by the image control apparatus 302 is analyzed by the point image control apparatus 201 to calculate a moving direction and a moving amount of the imaging target of the image instruction apparatus 302. Based on the analysis result, the point image control apparatus 201 generates a USB (Universal Serial Bus) standard signal that indicates the display position of the point image 310, and sends the signal to the personal computer 304. The personal computer 304 uses the point image display position control function provided in the GUI (graphical user interface) function to generate an image in which the point image is located at a predetermined coordinate, and outputs the image to the projection device 308. The projection device 308 projects the image on the screen 309. Thus, the point image 310 can be displayed on the screen 309 by following the movement of the image instruction device 302.

  In this embodiment, the position control of the point image 310 is performed using a point image display function generally provided in the personal computer 304. This is different from the first embodiment in which a point image is combined with an image generated by a personal computer using an external device. In this embodiment, since the point image control function of the personal computer is used, it is possible to operate the GUI function using the point image.

  In the present embodiment, the image instruction apparatus 302 includes a plurality of control switches 113 in the configuration shown in FIG. 2, and includes switches corresponding to right click switches and left click switches provided in a normal mouse. Use what you have.

  The signal output from the point image control apparatus 201 and input to the USB input port of the personal computer 304 is the same as the signal input to the personal computer from a normal pointing device (for example, a mouse). The processing contents in the personal computer 304 for controlling the position of the point image are the same. Accordingly, it is possible to perform right-click and left-click processing operations using the image instruction device 302 in the same manner as a normal mouse. That is, a GUI operation using the image instruction device 302 can be performed.

  In the present embodiment, as shown in FIG. 8, the point image control device 201 shown in FIG. 7 includes a reception device 202, a signal separation device 203, an image capture device 205, a movement amount adjustment device 206, an initial setting position control device 207, An image analysis device 208, a point image control signal generation device 209, a USB interface device 210, and a signal output device 211 are provided.

  In FIG. 8, devices other than the USB interface device 210 have the same functions as those of the devices shown in FIG. The USB interface device 210 generates a USB standard signal that designates the position of the point image based on the signal generated by the point image control signal generation device 209, and transmits it to the personal computer 304 (see FIG. 7). It has the function to do. Also, the USB interface device 210 converts the right click operation signal and the left click operation signal using the point image sent from the image instruction device 302 (see FIG. 2) into a USB standard signal, and converts the signal into a personal standard signal. It has a function of transmitting to the computer 304 (see FIG. 7).

2-2. Operation of Embodiment In FIG. 7, when the presenter 301 moves the image instruction device 302 and changes its direction of orientation, the imaging target 303 relatively moves and the image in the imaging field of view moves. The image data including the movement of the image is sent from the image instruction device 302 to the point image control device 201. The point image control device 201 receives the image data by the reception device 202 and captures the image via the signal separation device 203. Sent to device 205. The image data captured by the image capturing device 205 is analyzed by the image analyzing device 208, and the motion of the image in the above-described imaging field of view is analyzed. By this analysis, the moving direction and moving amount of the image taken by the imaging device 111 (see FIG. 2) are calculated. The analysis method is the same as the method described in the first embodiment.

  Based on the analysis result of the image analysis device 208, the point image control signal generation device 209 calculates the movement direction and movement amount of the point image, and further generates the coordinate data of the point image based on the calculation result. The processing content in the point image control signal generation device 209 is the same as the method described in the first embodiment. The coordinate data of the point image output from the point image control signal generation device 209 is converted into a USB standard signal by the USB interface device 210 and sent to the USB port of the personal computer 304 via the USB cable 305.

  The personal computer 304 combines the point image with the presentation image based on the signal designating the display position of the point image sent from the point image control apparatus 201. This processing is the same as the processing of a signal from a pointing device (for example, a mouse) that is normally performed. An image (a point image is included in this image) generated by the personal computer 304 is sent to the projection device 308 and projected onto the screen 309 from the projection device 308. Thus, the position of the point image 310 displayed on the screen 309 can be controlled by changing the pointing position of the image pointing device 302.

  In the present embodiment, processing using a graphical user interface (GUI) can be performed on the screen 309 using the point image 310. For example, it is assumed that a screen using application software that can use the GUI is displayed on the screen 309. At this time, the position of the point image 310 on the screen can be operated by the pointing position of the image instruction device 302 as described above. Further, by operating the control switch 113 (see FIG. 2) of the image instruction device 302, operations corresponding to the right click and left click of the mouse performed in the normal operation of the personal computer are displayed on the screen 309. Can be done on the screen.

  For example, assume that the control switch 113 (see FIG. 2) of the image instruction apparatus 302 is operated and an operation corresponding to a left click is performed. In this case, a signal indicating the left click is generated by the control signal generation circuit 116, and the signal is sent from the signal output device 117 to the point image control device 201 (see FIG. 8). This signal is received by the receiving device 202 in the point image control device 201 and sent from the signal separation device 203 to the USB interface device 210, where it is converted into a USB standard signal and sent to the personal computer 304 (see FIG. 7). Sent. In the personal computer 304, the same processing as that performed by a normal mouse is performed, and a left click operation using the point image 310 is performed.

  In this manner, in the system shown in FIG. 7, the graphical user interface using the point image 310 can be operated using the image instruction device 302.

  In the present embodiment illustrated in FIG. 7, an image captured by the image instruction device 302 can be directly projected on the screen 309. In this case, the image signal of the image captured by the imaging device 111 of the image instruction device 302 is received by the receiving device 202 of the point image control device 201 in FIG. 8 and further sent from the signal separation device 203 to the signal output device 211. . The signal output device 211 sends the image signal to the personal computer 304 via the image transmission cable 306. The personal computer 304 processes the image signal using appropriate application software, sends the image signal to the projection device 303, and performs projection. The device 303 projects this image on the screen 309.

  Switching to a mode in which an image captured by the image instruction device 302 is projected onto the screen 309 may be performed by the control switch 113 (see FIG. 2).

  In the present embodiment shown in FIG. 7, the amount of movement of the point image 310 can be adjusted according to the change in the pointing direction or the pointing position of the image pointing device 302. In this case, in response to the operation of the movement amount adjustment dial 115 (see FIG. 2), a control signal for setting the movement amount of the point image is generated in the movement amount adjustment device 206 of the point image control device 201 (see FIG. 8). In the point image control signal generation device 209, processing is performed so that the position of the designated point image reflects the adjusted amount of movement.

  In the present embodiment shown in FIG. 7, the position of the point image 310 can be forcibly displayed again at a predetermined position. In this case, an operation of the position reset switch 114 (see FIG. 2) causes the initial setting position control device 207 (see FIG. 8) in the point image pointing device 201 to forcibly display the point image at a predetermined position. A signal is generated, and the point image control signal generation device 209 performs processing for forcibly displaying the point image at a predetermined position in the image projected on the screen 309.

  In the above examples, the case where an image is projected on a screen using a projection device and a point image is displayed on the projection screen has been described. However, a display device that displays an image may be a cathode ray tube, a liquid crystal display, a plasma display, or A display device using a suitable light emitting device may be used. In addition, the present invention is not limited to application to presentations, and can be used for various processes and operations using point images, games using images, and expression activities.

  In the above-described example, the image instruction device includes a signal output unit that outputs a signal for displaying the captured image on the display image. The image directly captured by the image instruction device is captured in the computer or displayed as a display image. Can be used. According to this function, for example, during the presentation, the brought-in sample can be captured by the camera of the image pointing device and projected on the screen to enhance the presentation effect.

  The present invention can be used to operate a point image displayed on a screen. For example, the present invention can be used in an apparatus for performing a presentation performed by manipulating a point image projected on a screen and a technique related thereto.

It is a conceptual diagram which shows the outline | summary of the presentation system using the pointing device of this invention. It is a block diagram which shows the structure of an image instruction | indication apparatus. It is a block diagram which shows the structure of a point image control apparatus. It is a flowchart explaining the process in an image instruction | indication apparatus. It is a flowchart explaining an image analysis process. It is a conceptual diagram explaining the analysis method of an image. It is a conceptual diagram which shows the outline | summary of the presentation system using the pointing device of this invention. It is a block diagram which shows the structure of a point image control apparatus.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Image instruction | indication apparatus, 102 ... Screen, 103 ... Point image, 104 ... Image pick-up object which the image instruction | indication apparatus 101 imaged, 105 ... Projection apparatus, 106 ... Presenter, 107 ... Point image control apparatus, 108 ... Personal computer , 109 ... moving direction of imaging target, 110 ... moving direction of point image, 111 ... imaging device, 112 ... image signal generating device, 113 ... control switch, 114 ... position reset switch, 115 ... movement amount adjusting dial, 116 ... control Signal generation device, 117 ... Signal output device, 121 ... Reception device, 122 ... Signal separation device, 123 ... Image capture device, 124 ... Movement adjustment device, 125 ... Initial setting position control device, 126 ... Image analysis device, 127 ... Point image control signal generation device, 128 ... Image composition device, 129 ... Signal output Device 131 ··· Image 132 · Image moving direction 133 · Imaging field of view · 134 · Monitoring point · 135 · Monitoring point · 136 · Monitoring point · 137 · · · Monitoring point · 138 · · · Monitoring point · 201 · Point image control device 202 ... Receiving device 203 ... Signal separating device 205 ... Image capturing device 206 ... Moving amount adjusting device 207 ... Initial setting position control device 208 ... Image analyzing device 209 ... Point image control signal generating device 210 ... USB Interface device 211... Signal output device 301. Presenter 302. Image instruction device 303. Image pickup object imaged by image instruction device 301 304 Personal computer 305 USB (Universal Serial Bus) cable 306 ... Image transmission cable, 307 ... Display, 308 ... Throw Device, 309 ... screen, 310 ... point image.

Claims (6)

Image movement detecting means for detecting movement of an image captured by an imaging device provided in the image instruction device;
A calculation means for calculating a movement direction and a movement amount of the point image corresponding to the movement of the image;
Signal generating means for generating a signal for synthesizing the point image into the display image,
A pointing device, wherein the display position of the point image is moved in correspondence with movement of an image captured by the image instruction device. In the image movement detection means,
Storing the first image data at at least one of the plurality of monitoring points set in the imaging screen at a predetermined time;
Compare the second image data obtained from the plurality of monitoring points with the stored first image data after the predetermined time point,
Detecting a difference between the first and second image data from the comparison;
The pointing device according to claim 1, wherein a moving direction and a moving amount of the captured image are calculated from a difference between the image data.   The pointing device according to claim 2, wherein the monitoring point is a set of pixels divided in a matrix.   The pointing device according to claim 1, further comprising a movement amount adjusting unit that adjusts a movement amount of the point image corresponding to the movement amount of the image.   5. The pointing device according to claim 1, further comprising a control signal generating device that generates a control signal for controlling the graphical user interface operating means. An image movement detection step for detecting movement of the captured image;
A calculation step for calculating a movement direction and a movement amount of the point image in correspondence with the movement of the image;
A point image display method comprising: a signal generation step of generating a signal for synthesizing a point image into a display image based on the calculation result of the calculation step.

Images