JP2006040110A - Pointing device, and method for displaying point image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for displaying a point image on a projected image that eliminates complicated alignment, dispenses with laser light, which should not be viewed directly, and enables the position control of a point image with an image pointer directed to an arbitrary position. <P>SOLUTION: In a structure where the image pointer 101 is operated to display a point image 103 on a screen 102, as to an image movement in an imaging field 104 imaged by the image pointer 101, a position with a large density change in the imaging field is selected as a feature point, and the movement of the feature point in the imaging field is tracked to detect an orientation change of the image pointer 101. The moving direction and moving distance of the point image 103 are calculated from the result, and the point image 103 is synthesized with a display image projected on the screen 102 from a projector 105. The position of the point image 103 is controlled according to the orientation change of the image pointer 101. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a technique for controlling a point image displayed on a projected image or the like, and to a technique for controlling a display position of a point image using an image captured by an imaging device provided in an image instruction apparatus.

  A method of projecting an image on a screen and giving a presentation is known. At this time, a technique is known in which a point image (for example, an image of a light spot or an arrow) pointing to a specific image is superimposed and displayed on a screen.

  As such a technique, the techniques described in Patent Documents 1 to 4 are known. In Patent Document 1, in a configuration in which a point image is superimposed on an image projected on a screen from an image projection device, an ultrasonic wave is generated from an image instruction device for operating the point image, and the ultrasonic wave is provided on the screen side. A technique is described in which a point detected by an image sensor, a position pointed to by an image pointing device is specified, and a point image is displayed on a screen.

  Patent Documents 2 and 3 control the display position of a point image by irradiating infrared or instruction light from an image pointing device in the same technical field as Patent Document 1 and specifying the irradiation position on the screen. The technology is described.

  Patent Document 4 describes a technique for projecting a reference position for coordinate detection on a screen in a technique for irradiating an infrared ray from an image pointing device onto a screen and determining a display position of a point image.

Japanese Patent Laid-Open No. 2002-207466 Japanese Patent Laid-Open No. 11-271675 Japanese Patent Laid-Open No. 11-305940 Japanese Patent Laid-Open No. 11-85395

  However, the method using ultrasonic waves described in Patent Document 1 requires a receiving device that detects ultrasonic waves on the screen side, which increases the number of necessary devices and complicates the mechanism. In addition, the methods using infrared rays and indicator light described in Patent Documents 2 to 4 also require devices and mechanisms for detecting infrared rays and indicator light, and have the same problems as when ultrasonic waves are used.

  Further, in the above-described prior art, it is necessary to match the position designated by the image pointing device with the position of the point image in advance. However, there are cases where presentations are performed in various setting environments, and it is troublesome and inconvenient to perform alignment each time it is used. Further, in the above-described prior art, the position of the point image cannot be controlled unless the image pointing device is directed toward the screen. Always pointing the image pointing device to the screen during a presentation places restrictions on the posture, body orientation, or body movement (particularly the movement of the hand holding the image pointing device). This is not preferable.

  The present invention solves the above-described problems in the conventional point image control technology, does not increase the number of devices as much as possible, has a simple configuration, does not require complicated alignment, and does not point the image pointing device on the screen. It is another object of the present invention to provide a technique capable of controlling a point image.

  The pointing device according to the present invention includes an image movement detection unit that detects movement of an image captured by an imaging device included in the image instruction apparatus within an imaging field of view, and a movement direction and movement of the point image corresponding to the movement of the image. A calculation means for calculating the amount and a signal generation means for generating a signal for synthesizing the point image in the display image, wherein the image movement detection means has a difference in shading depending on the location in the captured image. It has a function of selecting one or a plurality of areas exceeding a predetermined level as a monitoring point.

  According to the pointing device of the present invention, the position control of the point image displayed on the screen, the display, or the like can be controlled by the position (or direction) to which the image pointing device is directed. That is, a portion where the difference in shading depending on the location exceeds a predetermined level is selected as a feature point (monitoring point) from the image captured by the image pointing device, and its relative movement in the imaging field of view is tracked. Thus, a change in the pointing direction of the image pointing device is detected, and the movement of the point image is controlled based on the detection result.

  That is, when the image pointing device is moved, a relative movement of the image in the imaging field occurs when the direction of the image is changed. When this movement is detected, the gradient of light and shade in the imaging field is detected. A large place (for example, a contour portion of a specific object) is dynamically selected as a feature point. By doing so, it is possible to detect the movement of the feature point in the imaging field without error.

  Since the movement of the feature point in the imaging visual field is opposite to the movement in the direction in which the image pointing device is directed, the direction in which the point image is moved can be determined by using this fact. On the other hand, the movement amount of the point image can be calculated by setting the correspondence between the movement amount of the point image and the movement amount of the feature point in the previous imaging field to a predetermined relationship.

  In this way, the feature point is a place where the gradient of shading is large, and by detecting the movement of the feature point in the imaging field of view, the change in the pointing direction of the image pointing device is detected, and the point displayed on the screen or the like based on the change You can move the image. In other words, the display position of the point image can be controlled by moving the image pointing device directed to an appropriate place and moving the image pointing device.

  In this method, an image of a specific article or the like having a large change in shading and its outline are recognized as feature points, so that inconveniences such as increased image recognition reliability and difficulty in recognizing image movement can be avoided. As a result, an accurate and reliable operation can be obtained.

  That is, in the pointing device of the present invention, the image movement detection unit acquires the monitoring pattern of the monitoring point selected from the images captured in the first imaging, and a predetermined time has elapsed since the first imaging. The monitoring pattern is searched from the captured image in the second imaging performed later, and the position of the monitoring pattern in the imaging field of view is compared by comparing the coordinates of the acquired monitoring pattern with the coordinates of the searched monitoring pattern. It is preferable to adopt a mode having a function of detecting the movement amount and calculating the movement direction and the movement amount of the captured image in the imaging visual field from the change in position.

  The point image refers to a display (for example, an arrow or the like) indicating an image (for example, a chart or a map) displayed on a screen or the like for use in a presentation. In general, the presenter gives an explanation and gives a presentation while moving the point image on the display surface and pointing to a chart or the like.

  In the pointing device of the present invention, it is preferable that the pointing device further includes a movement amount adjusting unit that adjusts the movement amount of the point image corresponding to the movement amount of the captured image. According to this aspect, it is possible to adjust the amount of movement of the point image on the display screen according to the degree of change in the direction in which the image pointing device is directed. That is, even if the image pointing device is moved a little, the point image is set to move greatly, even if the image pointing device is moved greatly, the point image is set not to move much, Can be adjusted.

  In the pointing device of the present invention, it is preferable that the pointing device further includes a control signal generating device for generating a control signal for controlling the graphical user interface operating means. Graphical User Interface is a user interface that uses graphics for displaying information to the user and can perform most operations with a pointing device such as a mouse. Say.

  According to this aspect, it is possible to operate various application software using the point image displayed on the screen or the like with the same feeling as the operation of a normal personal computer.

  If this aspect is utilized, a presentation using various application software can be performed, for example, with the operation of a graphical user interface on a screen. For example, when an image downloaded directly from the Internet is displayed on a screen and a presentation is performed, operations such as switching images and opening a link destination screen can be performed using the point image displayed on the screen.

  The pointing device of the present invention can also be grasped as a pointing method. That is, the pointing method of the present invention includes an image movement detection step for detecting movement in an imaging field of an image captured by an imaging apparatus included in the image pointing device, and a movement direction of the point image corresponding to the movement of the image. And a movement step for calculating the amount of movement and a signal generation step for generating a signal for synthesizing the point image in the display image. The image movement detection step includes: It can be grasped as a point image display method including a step of selecting one or a plurality of areas whose differences exceed a predetermined level as monitoring points.

  In the point image display method, the image movement detection step includes acquiring a monitoring pattern of the monitoring point selected from the first captured image, and capturing the image after a predetermined time has elapsed since the first imaging. The step of searching for the monitoring pattern from the second captured image and the change of the position of the monitoring pattern in the imaging field of view by comparing the coordinates of the acquired monitoring pattern with the coordinates of the searched monitoring pattern. It is preferable to provide an aspect including a detecting step and a step of calculating a moving direction and a moving amount of the captured image from a change in the position of the monitoring pattern.

  According to the present invention, the movement of the pointing target location of the image pointing device is detected from the movement of the image captured by the imaging means of the image pointing device, and the moving direction and the moving amount of the point image are calculated from the result. Less equipment is required and the configuration can be simplified.

  Further, since the image to be detected is not particularly limited, it is possible to control the point image without pointing the image pointing device on the screen. At this time, a feature point such as the contour of the imaging target that can be easily tracked is searched for, and the relative movement in the imaging field of the captured image corresponding to the movement of the image pointing device is detected by paying attention thereto. Therefore, it is possible to obtain an advantage that the operability is not limited by the object to which the image pointing device is directed. In addition, since a portion with a clear shading difference that can reliably track the movement is used as a feature point, it is possible to suppress errors and uncertainties associated with the operation.

  As described above, according to the present invention, the number of devices is not increased as much as possible, the configuration is simple, no complicated alignment is required, and the point image can be reliably controlled without the image pointing device being directed on the screen. Technology is provided.

(First embodiment)
1-1. Configuration of Embodiment FIG. 1 is a conceptual diagram showing an outline of a presentation system using a pointing device of the present invention. 1 includes an image instruction device 101, a screen 102, a point image 103, an imaging target 104 of the image instruction device 101, a projection device 105, a presenter (person who makes a presentation) 106, a point image control device 107, and a personal computer 108. It is shown.

  In this system, when the presenter 106 moves the image instruction apparatus 101, the point image 103 projected on the screen moves corresponding to the movement. That is, the movement of the image captured by the image instruction apparatus 101 within the imaging field of view is analyzed by the point image control apparatus 107, the movement direction and movement amount of the point image 103 are calculated, and the display position of the point image 103 is controlled accordingly. Is done.

  The image instruction device 101 captures an arbitrary imaged object directed by the presenter 106, sends the image to the point image control device 107, and sends various control signals to the point image control device 107.

  The personal computer 108 stores an image of the content to be presented that has been processed and created using appropriate application software, and outputs the image data to the point image control device 107 in accordance with a predetermined operation.

  The point image control device 107 generates a signal for controlling the position of the point image 103 on the screen 102 based on the signal sent from the image instruction device 101. Further, the point image control apparatus 107 performs a process of superimposing the point image on the image sent from the personal computer 108.

  FIG. 2 is a block diagram illustrating an example of the configuration of the image instruction device 101. 2 includes an imaging device 111, an image signal generation device 112, a control switch 113, a position reset switch 114, a movement amount adjustment dial 115, a control signal generation device 116, and a signal output device 117.

  The projection device 105 is a liquid crystal projector, a three-tube projector, or the like, and has a function of projecting an image on the screen 102 based on an image signal sent from the point image control device 107.

  The imaging device 111 is a camera provided with a CCD imaging device. The image signal generation device 112 converts an image captured by the image sensor 111 into an appropriate electrical signal (image signal) suitable for transmission. The control switch 113 is a switch used when switching to a mode in which a captured image is directly projected, or when using a mouse function described later. For example, a function corresponding to a right click and a left click of the mouse can be assigned as a control switch. The position reset switch 114 is a switch for initial setting of the position of the point image 103. For example, when the position reset switch 114 is operated, the point image 103 can be forcibly displayed at the center of the screen 102.

  The movement amount adjustment dial 115 is a dial that adjusts the relationship between the change amount of the pointing direction or the pointing position of the image pointing device 101 and the movement amount of the point image 103 on the screen. For example, when adjustment is made so that the amount of movement of the point image 103 on the screen changes greatly compared to the amount of movement of the pointing position of the image instruction device 101, the point image 103 is moved only by moving the image instruction device 101 slightly. Can be set to move greatly.

  The control signal generation device 116 has a function of converting the operation contents of the control switch 113, the position reset switch 114, and the movement amount adjustment dial 115 into an electrical signal suitable for transmission to the point image control device 107. The signal output device 117 has a function of transmitting to the point image control device 107 the electric signal generated by the image signal generation device 112 and the control signal generation device 116 as a radio wave signal.

  FIG. 3 is a block diagram showing an example of the configuration of the point image control device 107 (see FIG. 1). The point image control device 107 shown in FIG. 3 includes a reception device 121, a signal separation device 122, an image capture device 123, a movement amount adjustment device 124, an initial setting position control device 125, an image analysis device 126, and a point image control signal generation device 127. The image synthesizing device 128 and the signal output device 129 are provided.

  The receiving device 121 has a function of receiving a radio signal from the image instruction device 101. The signal separation device 122 has a function of separating an image signal and various control signals included in the signal received by the reception device 121 and guiding them to a predetermined device. The movement amount adjustment device 124 has a function of receiving a control signal reflecting the adjustment contents of the movement amount adjustment dial 115 of the image instruction device 101 and adjusting the ratio of the movement amount of the point image 103. For example, the movement amount adjustment device 124 has a function of arbitrarily multiplying the movement amount corresponding to the movement of the image captured by the image instruction device 101 with respect to a predetermined reference value, and the adjustment amount dial 115 (FIG. 2). It has a function of changing the numerical value to be arbitrarily multiplied in accordance with the setting condition of (see).

  The image capturing device 123 has a function of repeatedly capturing an image captured by the image capturing device 111 (see FIG. 2) at a predetermined sampling interval. The image analysis device 126 analyzes the image captured by the image capturing device 123 (the image captured by the image capturing device 111), and calculates a relative moving direction and moving amount within the imaging field of view. The image analysis device 126 includes a storage device (memory) necessary for image analysis. The image analysis method will be described in detail later.

  The point image control signal generation device 127 determines the movement direction of the point image 103 based on the relative movement direction and movement amount of the image captured by the image instruction device 101 calculated by the image analysis device 126 within the imaging field of view. The movement amount is calculated, and coordinate data of the position where the point image is to be displayed is generated based on the calculation result.

  In addition, the point image control signal generation device 127 responds to the signal from the movement amount adjustment device 124 so that the movement amount of the point image 103 becomes a value corresponding to the operation of the movement amount adjustment dial 115 (see FIG. 2). Perform processing. Further, when the point image control signal generation device 127 receives a signal for forcibly displaying the point image 103 at a predetermined position from the initial setting position control device 125, the point image control signal generation device 127 displays the point image 103 on the screen 102. Processing for displaying the position is performed.

  The image synthesizer 128 generates image data for displaying a point image at a position determined by the coordinate data generated by the point image control signal generator 127, and the image data is stored in the personal computer 108 (see FIG. 1). ) Is combined with the image sent from. By this processing, a composite image is generated in which the point image is displayed at a predetermined position in the image sent from the personal computer 108 (see FIG. 1).

  The signal output device 129 has a function of sending the image signal processed by the image composition device 128 to the projection device 105 (see FIG. 1). The signal output device 129 has a function of transmitting the image signal separated by the signal separation device 122 to the projection device 105.

  In the present embodiment, an optical zoom function in the projection device 105 may be operated by the image instruction device 101. In this case, the image instruction device 101 is further provided with a zoom adjustment operation switch, and the point image control device 107 is further provided with a zoom adjustment signal generation device. In this aspect, by operating the zoom adjustment operation switch of the image instruction device 101, a signal reflecting the operation result is sent to the point image control device 107, and the zoom adjustment of the point image control device 107 is received in response to the signal. In the signal generation device, a control signal for controlling the optical zoom function of the projection device 105 is generated, and the optical zoom function of the projection device 105 is controlled by the control signal.

1-2. Operation of Embodiment First, an example of the operation of the image instruction apparatus 101 will be described. FIG. 4 is a flowchart illustrating an example of the operation of the image instruction apparatus 101. First, it is determined whether the use of the image instruction device 101 has been started, that is, whether an imaging start switch (not shown) is turned on (step S401). If the switch is turned on, imaging by the imaging device 111 (see FIG. 2) is performed. (Step S402), otherwise, step S401 is repeated.

  The image picked up by the image pickup device 111 is converted into an image signal by the image signal generation device 112 and transmitted as a radio wave signal from the signal output device 117 to the point image control device 107 (see FIG. 1) (step S403).

  The above processing is repeatedly performed in a state where the image instruction apparatus 101 is used. Then, the image signal of the imaging target 104 captured by the imaging device 111 of the image instruction device 101 is continuously transmitted to the point image control device 107.

  Hereinafter, an example of a method for controlling the position of the point image 103 displayed on the screen 102 will be described. When the presenter 106 moves the image instruction device 101 held in the hand, the motion is detected as a relative image motion in the imaging field captured by the imaging device 111, and the point image 103 is converted based on the motion of the image. Control to move on the screen 102 is performed. For example, if the image pointing device 101 is moved and the imaging target 104 is moved in the direction of the arrow 109, the movement is analyzed by image analysis, and control is performed so that the point image 103 is moved in the direction of the arrow 110. .

  According to this control, the presenter 106 can move the point image 103 freely on the screen 102 by directing the image pointing device 101 held in an arbitrary position and moving it appropriately.

  Hereinafter, an example of this control will be described in detail. FIG. 5 is a flowchart illustrating an example of image processing performed in the image analysis device 126 and the point image control signal generation device 127 of FIG.

  First, the image analysis device 126 determines a monitoring point for acquiring basic data for detecting the movement of the image sent from the imaging device 111 within the imaging visual field (step S501). Details of the process for determining the monitoring point will be described later.

  When the monitoring point is determined, the coordinate data of the monitoring point and the monitoring pattern data at the monitoring point are stored. Since the monitoring point is, for example, m × n pixel matrix, the monitoring pattern is obtained as table data of this matrix pixel. For example, when a black and white image is a target and the monitoring point is a pixel matrix of 10 × 10 pixels, table data in which white is set to 0 and black is set to a 10 × 10 matrix is obtained.

  When the monitoring pattern is stored, data that matches the monitoring pattern stored in step S502 is searched from the image data captured by the image capturing device 123 at a predetermined sampling interval (step S503).

  If the search in step S503 could not be performed, the process returns to the previous stage of step S501, and step S501 and the subsequent steps are executed again. If the search has been performed, the process proceeds to step S505.

  In step S505, the coordinate data of the searched monitoring pattern is compared with the coordinate data of the monitoring point stored in step S502. From this comparison result, the relative movement of the image picked up by the image pickup apparatus 111 within the image pickup field of view is calculated.

  In step 505, when the movement direction and movement amount of the captured image within the imaging field of view of the imaging device 111 are calculated, the movement direction and movement amount of the point image are calculated based on the calculation result (step S506). In this way, the moving direction and moving amount of the point image 103 according to the change in the directing direction of the image instruction device 101 are calculated.

  For example, when the pointing direction of the image instruction apparatus 101 is not moved, the coordinate data of the searched monitoring pattern is the same as the coordinate data of the monitoring point stored in step S502. Therefore, the calculation result of step S506 is as follows: the moving direction of the point image → no change, the moving amount of the point image → zero, and the process of not moving the point image 103 shown in FIG. 1 is performed.

  On the other hand, let us consider a case where the image pointing device 101 is moved and the position of the pointing target is moved in the direction indicated by the arrow 109 in FIG. In this case, the image in the imaging field of the imaging device 111 moves in the direction opposite to the arrow 109 in the imaging field by 180 °, and the selected monitoring point also moves in the same way in the imaging field. Therefore, the monitor pattern image searched in step S503 moves to the upper right location with respect to the original location in the imaging field of view. By comparing the coordinates of the image of the moved monitoring pattern with the coordinates of the monitoring point selected in step S501, the moving direction and amount of movement of the monitoring pattern image in the imaging field of view are calculated. For example, in this case, a predetermined movement amount in the upper right direction of the monitoring pattern in the imaging field of view is calculated.

  In step S506, the moving direction calculated in step S505 is converted as the opposite direction, whereby the moving direction of the point image 103 on the screen 102 is calculated, and the monitoring pattern image calculated in step S505 is captured. The movement amount in the visual field is converted into the movement amount of the point image 103 on the screen 102.

  As will be described later, the correspondence relationship between the movement amounts can be adjusted. For example, when the pointing direction of the image pointing device 101 is slightly moved, the point image 103 moves greatly or moves correspondingly. These settings can be adjusted.

  Hereinafter, an example of the processing content of step 501 will be described in detail. FIG. 6 is a flowchart illustrating an example of processing for determining a monitoring point. FIG. 7 is a diagram illustrating an example of a monitoring point setting state.

  First, the setting of the monitoring point in this example will be described. FIG. 7 shows a configuration in which a monitoring point group in which the imaging field of view is divided into five is set, and each monitoring point group is further divided into a plurality of monitoring points.

  In this example, the monitoring point group is set in five areas in the imaging field of view, ie, the upper left, upper right, lower left, lower right, and center of the imaging field, which are divided into four parts. The monitoring point groups 701, 701, 703 and 704 are composed of eight divided monitoring points, and the monitoring point group 705 has a configuration in which a monitoring point is set in the center in addition to the eight divided monitoring points. It has become.

  In determining the monitoring point, a process of selecting a monitoring point group is first performed on an image captured by the imaging device 111 and captured at a predetermined sampling timing by the image capturing device 123 (step S601).

  The selection of the monitoring point group is performed in the order of, for example, the monitoring point group 701 → the monitoring point group 702 →... The monitoring point group 705 → the monitoring point group 701.

  When a monitoring point group is selected, a monitoring point is selected from the monitoring point group (step S602). For example, in the case of the monitoring point group 701, the monitoring point is selected by monitoring point (1-1) → monitoring point (1-2) →... Monitoring point (1-8) → monitoring point (1-1). ), And so on.

  When the monitoring point is selected, the image data of the selected monitoring point is detected (step S603), and it is determined whether the change rate of shading in the monitoring point exceeds a preset threshold value (step S604).

  Here, it is determined whether or not there is a significant difference in light and shade (that is, a difference in contrast) that can be used as a feature point by looking at the change rate of light and shade in the monitoring point. For example, when a specific object is imaged, the contour portion of the image of the object shows a clear difference in light and shade, so the change rate of light and shade (the change rate of the position of light and shade) increases accordingly. In this way, whether a feature point is suitable for detecting the movement of an image by setting a predetermined threshold value for the change rate of shading in the monitoring point and determining whether or not the threshold value is exceeded. Can be determined.

  If the determination in step S604 is YES, it is decided to use the monitoring point for data acquisition for image analysis (step S606). If the determination in step S604 is NO, the image within the same monitoring point group is determined. It is determined whether or not there is a monitoring point that has not been detected (that is, unselected) (step S605). If there is an unselected monitoring point, the next monitoring point is selected (step S607). If there is no selected monitoring point, the process advances to step S608 to select the next monitoring point group, and a process of searching for a monitoring point that can be used as a feature point from the monitoring point group (that is, the process after step S602) is executed. The

  For example, consider a case where the monitoring point group 701 is selected in step S601. In this case, the monitoring point (1-1) is first selected according to the selection order set as monitoring point (1-1) → (1-2) → (1-3) →... (Step S602). ).

  Then, the image data of the monitoring point (1-1) is detected (step S603), and it is determined whether or not the change rate of shading in the monitoring point (1-1) is below a predetermined threshold (step S604). ). At this time, if the change in shading is such that it cannot be used as a feature point, the determination in step S604 is no and the process proceeds to step S605. In this case, since the monitoring point (1-1) is selected in step S602, the determination in step S605 is YES, and the next monitoring point (1-2) is selected (step S607), the procedure after step S603 is performed again.

  Thus, the search for the monitoring points in the monitoring point group 701 is sequentially performed in accordance with the determined order until a monitoring point having a certain difference in positional shading is found. If a monitoring point having a difference in density that can be used to detect image movement is not found, step S605 is NO, the next monitoring point group 702 is selected, and the monitoring point group 701 is compared with the case. Similar processing is executed.

  In this manner, each monitoring point in each monitoring point group is sequentially searched until a density (for example, an edge portion of an image of a specific article) that can be used as a feature point for detecting the motion of the image is found, decide.

  According to the embodiment exemplified above, since the process in which the region where the difference in shading exceeds the predetermined level is selected as the monitoring point is performed, the place where the image pointing device 101 is directed is a ceiling or a wall, for example. Even if it is a point-poor portion, a feature point is automatically found, and a change in the pointing direction of the image pointing device 101 is detected using the feature point, and the movement of the point image 103 is controlled based on the change. Can do.

  For this reason, even when the image pointing device 101 is generally directed to a ceiling or wall having few feature points, it is possible to obtain an advantage that the inconvenience that the position control of the point image 103 becomes difficult does not occur. . Such superiority is also effective for presentations in dark venues.

  In the example described above, the case where one feature point is selected has been described. However, a configuration may be adopted in which a plurality of feature points are selected.

  In the present embodiment, the movement amount of the point image 103 can be adjusted by a movement amount adjustment device 124 (see FIG. 3), and the point image 103 corresponding to the change in the pointing position or the swing angle of the image instruction device 101. The amount of movement on the screen 102 can be arbitrarily adjusted.

  This adjustment is performed by operating the movement amount adjustment dial 115 (see FIG. 2) of the image instruction apparatus 101. That is, when the movement amount adjustment dial 115 is adjusted, a signal reflecting the adjustment content is generated by the control signal generation device 116 and sent from the image instruction device 101 to the point image control device 107. This control signal is sent from the signal separation device 122 to the movement amount adjustment device 124 via the reception device 121. The movement amount adjustment device 124 sets the movement amount of the point image 103 according to the operation state of the movement amount adjustment dial 115, and sends a signal designating the content to the point image control signal generation device 127.

  According to this aspect, on the screen 102 of the point image 103 according to the amount of movement of the imaging target 104 of the image instruction apparatus 101 according to the habit of how the image instruction apparatus 101 is moved by the presenter 106 shown in FIG. The amount of movement can be adjusted.

  In the present embodiment, the position of the point image 103 on the screen 102 can be forcibly aligned at an arbitrary timing. That is, the position of the point image 103 on the screen 102 can be forcibly displayed at the center of the screen.

  Hereinafter, a detailed example of this operation will be described. When the presenter 106 (see FIG. 1) operates the position reset switch 114 (see FIG. 2) of the image indicating device 101, a signal indicating that is generated by the control signal generating device 116, and the signal is sent from the signal output device 117. It is transmitted to the point image control device 107 (see FIGS. 1 and 3). The signal is received by the receiving device 121 of the point image control device 107 and sent to the initial setting position control device 125 through the signal separation device 122.

  In response to the signal, the initial setting position control device 125 sends a signal for displaying the point image 103 at a predetermined position on the screen 102 (for example, the center of the screen 102) to the point image control signal generation device 127. In response to this signal, the point image control signal generation device 127 generates a signal for displaying the point image 103 at a predetermined position on the screen 102 and sends it to the image composition device 128.

  Then, the image composition device 128 creates a composite image in which the point image 103 is displayed at the center of the image sent from the personal computer 108, and sends the composite image signal to the projection device 105. The projection device 105 projects this composite image on the screen 102. Thus, the point image 103 is forcibly displayed at the center of the screen 102.

  According to this aspect, the presenter 106 can initialize or reset the position of the point image 103 on the screen 102 at an arbitrary timing. With this function, when the presenter 106 (see FIG. 1) loses sight of the point image 103, a reset operation can be performed to redisplay the position of the point image at a predetermined position. This operation can also be used to set the initial position of the point image 103 when starting a presentation.

  In the configuration shown in FIG. 1, an image captured by the image instruction device 101 can be projected on a screen. In this case, in the image instruction device 101 of FIG. 2, the control switch 113 is operated to switch to the image capturing mode. The image picked up by the image pickup device 111 is converted into an image signal by the image signal generation device 112, and the signal is sent from the signal output device 117 to the point image control device 107 shown in FIG. In the point image control apparatus 107 shown in FIG. 3, the signal is received by the reception apparatus 121 and sent to the projection apparatus 105 (see FIG. 1) via the signal separation apparatus 122 and the signal output apparatus 129. An image captured by the image instruction device 101 is projected from the projection device 105 onto the screen 102. This aspect is effective when an image obtained by capturing a sample or the like is projected on a screen in a presentation.

  It is also possible to superimpose an image captured by the image instruction device 101 on an image from the personal computer 108 and project the composite image on the screen 102. In this case, the image signal from the image instruction device 101 received by the reception device 121 is received by the reception device 121 of the point image control device 107 and sent from the signal separation device 122 to the image synthesis device 128, where the image synthesis is performed. Is done.

(Second Embodiment)
2-1. Configuration of Embodiment In this embodiment, a point image position control technique using the present invention is combined with a graphical user interface (GUI) function provided in a personal computer.

  FIG. 8 is a conceptual diagram showing an outline of another presentation system using the pointing device of the present invention. FIG. 9 is a block diagram showing an example of the configuration of the point image control apparatus 201 in FIG.

  This embodiment is an example when the present invention is applied to a system that projects and displays a point image on a screen using a control function of a point image (mouse pointer) provided in a personal computer.

  In FIG. 8, a presenter 301, an image instruction device 302, an imaging target 303 imaged by the image instruction device 302, a point image control device 201, a personal computer 304, a USB (Universal Serial Bus) cable 305, and an image transmission cable 306 A projection device 308, a screen 309 and a point image 310 are shown.

  In the present embodiment, the point image control apparatus 201 analyzes an image captured by the image control apparatus 302, thereby calculating a movement direction and a movement amount of the image captured by the image instruction apparatus 302 in the imaging field of view. Based on the analysis result, the point image control apparatus 201 generates a USB (Universal Serial Bus) standard signal that indicates the display position of the point image 310, and sends the signal to the personal computer 304. The personal computer 304 uses the point image display position control function provided in the GUI (graphical user interface) function to generate an image in which the point image is located at a predetermined coordinate, and outputs the image to the projection device 308. The projection device 308 projects the image on the screen 309. Thus, the point image 310 can be displayed on the screen 309 by following the movement of the image instruction device 302.

  In this embodiment, the position control of the point image 310 is performed using a point image display function generally provided in the personal computer 304. This is different from the first embodiment in which a point image is combined with an image generated by a personal computer using an external device. In this embodiment, since the point image control function of the personal computer is used, it is possible to operate the GUI function using the point image.

  In the present embodiment, the image instruction apparatus 302 includes a plurality of control switches 113 in the configuration shown in FIG. 2, and includes switches corresponding to right click switches and left click switches included in a normal mouse. Use things.

  The signal output from the point image control apparatus 201 and input to the USB input port of the personal computer 304 is the same as the signal input to the personal computer from a normal pointing device (for example, a mouse). The processing contents in the personal computer 304 for controlling the position of the point image are the same. Accordingly, it is possible to perform right-click and left-click processing operations using the image instruction device 302 in the same manner as a normal mouse. That is, a GUI operation using the image instruction device 302 can be performed.

  In the present embodiment, as shown in FIG. 9, the point image control device 201 shown in FIG. 8 includes a reception device 202, a signal separation device 203, an image capture device 205, a movement amount adjustment device 206, an initial setting position control device 207, An image analysis device 208, a point image control signal generation device 209, a USB interface device 210, and a signal output device 211 are provided.

  In FIG. 9, devices other than the USB interface device 210 have the same functions as those of the devices shown in FIG. The USB interface device 210 generates a USB standard signal that designates the position of the point image based on the signal generated by the point image control signal generation device 209, and transmits it to the personal computer 304 (see FIG. 8). It has the function to do. Also, the USB interface device 210 converts the right click operation signal and the left click operation signal using the point image sent from the image instruction device 302 (see FIG. 2) into a USB standard signal, and converts the signal into a personal standard signal. It has a function of transmitting to the computer 304 (see FIG. 8).

2-2. Operation of Embodiment In FIG. 8, when the presenter 301 moves the image instruction device 302 and changes its directing direction, the imaging target 303 relatively moves, and the image in the imaging field of view moves. The image data including the motion of the image is sent from the image instruction device 302 to the point image control device 201. The point image control device 201 receives the image data by the reception device 202 and captures the image via the signal separation device 203. Sent to device 205. The image data captured by the image capturing device 205 is analyzed by the image analysis device 208, and the motion of the image in the above-described imaging field of view is analyzed. By this analysis, the moving direction and moving amount of the image taken by the imaging device 111 (see FIG. 2) within the imaging field of view are calculated. The analysis method is the same as the method described in the first embodiment.

  Based on the analysis result of the image analysis device 208, the point image control signal generation device 209 calculates the movement direction and movement amount of the point image, and further generates the coordinate data of the point image based on the calculation result. The processing content in the point image control signal generation device 209 is the same as the method described in the first embodiment. The coordinate data of the point image output from the point image control signal generation device 209 is converted into a USB standard signal by the USB interface device 210 and sent to the USB port of the personal computer 304 via the USB cable 305.

  The personal computer 304 combines the point image with the presentation image based on the signal designating the display position of the point image sent from the point image control apparatus 201. This processing is the same as the processing of a signal from a pointing device (for example, a mouse) that is normally performed. An image generated by the personal computer 304 (this image includes a point image) is sent to the projection device 308 and projected from the projection device 308 onto the screen 309. Thus, the position of the point image 310 displayed on the screen 309 can be controlled by changing the pointing position of the image pointing device 302.

  In the present embodiment, processing using a graphical user interface (GUI) can be performed on the screen 309 using the point image 310. For example, it is assumed that a screen using application software that can use the GUI is displayed on the screen 309. At this time, the position of the point image 310 on the screen can be operated by the pointing position of the image instruction device 302 as described above. Further, by operating the control switch 113 (see FIG. 2) of the image instruction device 302, operations corresponding to the right click and left click of the mouse performed in the normal operation of the personal computer are displayed on the screen 309. Can be done on the screen.

  For example, assume that the control switch 113 (see FIG. 2) of the image instruction apparatus 302 is operated and an operation corresponding to a left click is performed. In this case, a signal indicating the left click is generated by the control signal generation circuit 116, and the signal is sent from the signal output device 117 to the point image control device 201 (see FIG. 9). This signal is received by the receiving device 202 in the point image control device 201 and sent from the signal separation device 203 to the USB interface device 210, where it is converted into a USB standard signal and sent to the personal computer 304 (see FIG. 8). Sent. In the personal computer 304, the same processing as that performed by a normal mouse is performed, and a left click operation using the point image 310 is performed.

  In this way, in the system shown in FIG. 8, the graphical user interface using the point image 310 can be operated using the image pointing device 302.

  In the present embodiment illustrated in FIG. 8, an image captured by the image instruction device 302 can be directly projected on the screen 309. In this case, an image signal of an image captured by the imaging device 111 of the image instruction device 302 is received by the reception device 202 of the point image control device 201 and further sent from the signal separation device 203 to the signal output device 211. The signal output device 211 sends the image signal to the personal computer 304 via the image transmission cable 306. The personal computer 304 processes the image signal using appropriate application software, sends the image signal to the projection device 303, and performs projection. The device 303 projects this image on the screen 309.

  Switching to a mode in which an image captured by the image instruction device 302 is projected onto the screen 309 may be performed by the control switch 113 (see FIG. 2).

  In the present embodiment shown in FIG. 8, the amount of movement of the point image 310 can be adjusted according to the change in the pointing direction or the pointing position of the image pointing device 302. In this case, in response to the operation of the movement amount adjustment dial 115 (see FIG. 2), a control signal for setting the movement amount of the point image is generated in the movement amount adjustment device 206 of the point image control device 201 (see FIG. 9). In the point image control signal generation device 209, processing is performed so that the position of the designated point image reflects the adjusted amount of movement.

  In the present embodiment shown in FIG. 8, the position of the point image 310 can be forcibly displayed again at a predetermined position. In this case, an operation of the position reset switch 114 (see FIG. 2) causes the initial setting position control device 207 (see FIG. 9) in the point image instruction device 201 to forcibly display the point image at a predetermined position. A signal is generated, and the point image control signal generation device 209 performs processing for forcibly displaying the point image at a predetermined position in the image projected on the screen 309.

  In the above examples, the case where an image is projected on a screen using a projection device and a point image is displayed on the projection screen has been described. However, a display device that displays an image may be a cathode ray tube, a liquid crystal display, a plasma display, or A display device using a suitable light emitting device may be used. In addition, the present invention is not limited to application to presentations, and can be used for various processes and operations using point images, games using images, and expression activities.

  In the above-described example, the image instruction device includes a signal output unit that outputs a signal for displaying the captured image on the display image. The image directly captured by the image instruction device is captured in the computer or displayed as a display image. Can be used. According to this function, for example, during the presentation, the brought-in sample can be captured by the camera of the image pointing device and projected on the screen to enhance the presentation effect.

  The present invention can be used to operate a point image displayed on a screen. For example, the present invention can be used in an apparatus for performing a presentation performed by manipulating a point image projected on a screen and a technique related thereto.

It is a conceptual diagram which shows the outline | summary of the presentation system using the pointing device of this invention. It is a block diagram which shows the structure of an image instruction | indication apparatus. It is a block diagram which shows the structure of a point image control apparatus. It is a flowchart explaining the process in an image instruction | indication apparatus. It is a flowchart explaining the analysis method of an image. It is a flowchart explaining the analysis method of an image. It is a front view which shows the monitoring point setting state. It is a conceptual diagram which shows the outline | summary of the presentation system using the pointing device of this invention. It is a block diagram which shows the structure of a point image control apparatus.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Image instruction | indication apparatus, 102 ... Screen, 103 ... Point image, 104 ... Image pick-up object which the image instruction | indication apparatus 101 imaged, 105 ... Projection apparatus, 106 ... Presenter, 107 ... Point image control apparatus, 108 ... Personal computer , 109 ... moving direction of imaging target, 110 ... moving direction of point image, 111 ... imaging device, 112 ... image signal generating device, 113 ... control switch, 114 ... position reset switch, 115 ... movement amount adjusting dial, 116 ... control Signal generation device, 117 ... Signal output device, 121 ... Reception device, 122 ... Signal separation device, 123 ... Image capture device, 124 ... Movement adjustment device, 125 ... Initial setting position control device, 126 ... Image analysis device, 127 ... Point image control signal generation device, 128 ... Image composition device, 129 ... Signal output 201: Point image control device, 202 ... Reception device, 203 ... Signal separation device, 205 ... Image capture device, 206 ... Movement amount adjustment device, 207 ... Initial position control device, 208 ... Image analysis device, 209 ... Point Image control signal generator 210 ... USB interface device 211 ... Signal output device 301 ... Presenter 302 ... Image instruction device 303 ... Object to be imaged by image instruction device 301 304 ... Personal computer 305 ... USB (Universal Serial Bus) cable, 306 ... image transmission cable, 308 ... projection device, 309 ... screen, 310 ... point image.

Claims (6)

An image movement detecting means for detecting movement in an imaging field of an image captured by an imaging device provided in the image instruction device;
Calculating means for calculating the moving direction and moving amount of the point image in correspondence with the movement of the image;
Signal generating means for generating a signal for synthesizing the point image into the display image,
The image movement detection means includes
A pointing device having a function of selecting, as a monitoring point, one or a plurality of regions in which a difference in shading depending on a location exceeds a predetermined level in a captured image. The image movement detection means includes
A monitoring pattern of the monitoring point selected from the images captured in the first imaging is acquired, and the monitoring pattern is acquired from the captured image in the second imaging performed after a predetermined time has elapsed since the first imaging. By searching and comparing the coordinates of the acquired monitoring pattern with the coordinates of the searched monitoring pattern, a change in the position of the monitoring pattern in the imaging field of view is detected. The pointing device according to claim 1, wherein the moving direction and the moving amount in the imaging field of view are calculated.   The pointing device according to claim 1, further comprising a movement amount adjusting unit that adjusts a movement amount of the point image corresponding to a movement amount of the captured image.   4. The pointing device according to claim 1, further comprising control signal generation means for generating a control signal for controlling the graphical user interface operation means. An image movement detection step for detecting movement in an imaging field of an image captured by an imaging device provided in the image instruction device;
A calculation step for calculating a movement direction and a movement amount of the point image in correspondence with the movement of the image;
A signal generating step for generating a signal for synthesizing the point image into the display image,
The image movement detection step includes:
A point image display method comprising a step of selecting, as a monitoring point, one or a plurality of areas in which a difference in shading depending on a place exceeds a predetermined level in a captured image. The image movement detection step includes:
Obtaining a monitoring pattern of the monitoring point selected from the first captured image;
Retrieving the monitoring pattern from a second captured image captured after a lapse of a predetermined time from the first imaging;
Detecting a change in the position of the monitoring pattern in the imaging field by comparing the coordinates of the acquired monitoring pattern with the coordinates of the searched monitoring pattern;
The point image display method according to claim 5, further comprising a step of calculating a moving direction and a moving amount of the captured image from a change in position of the monitoring pattern.

Images