JPH11345086A - Pointing position detecting device and method, cursor position control method, presentation system and information storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pointing position detector capable of automatically and accurately recognizing a pointing position in a display area without using anything special as a point indicator. SOLUTION: This pointing position detector is provided with a CCD camera 14 for picking up the image of the display area 12 where images are displayed and a binarization processing part 112 and a pointing coordinate detection part 116 for position detection for detecting the prescribed position of the area of the real image of an indication and a shadow area which is the image area of an indication image included in an image pickup area as the pointing position based on the image pickup signals of the CCD camera 14.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pointing position detecting device and method for automatically detecting a pointing position in a display area, a cursor position controlling method, a presentation system, and an information storage medium.

[0002]

2. Description of the Related Art It is assumed that a presenter gives a presentation while looking at a screen projected on a screen, for example. In such a case, the presenter proceeds with the presentation while pointing to an arbitrary point on the screen using the pointing device.

In this case, the presenter can easily point to an arbitrary point on the screen, and the pointing position can be reliably recognized by the system.
It is important to reflect this in image processing.

However, in the conventional system, when the presenter points to a desired point on the screen,
A special pointing device must be used, which is extremely inconvenient for the user.

For example, when a laser pointer is used to point to an arbitrary point on the screen, it is difficult to recognize the pointing position unless the periphery of the screen is dark to some extent. In addition, since the point position is displayed as a small point, there is a problem that the pointing position is difficult to understand from this plane.

When a presentation is performed using a pointing device such as a mouse while moving the position of a cursor displayed on a screen, the presenter must interrupt the presentation and operate the mouse each time. For this reason, there was a problem that the presentation could not be performed smoothly and the usability was extremely poor.

The present invention has been made in view of such problems, and has as its object to automatically and accurately determine a pointing position in a display area without using a special pointing device. Pointing position detection device and method that can be recognized
Cursor position control method, presentation system,
An object of the present invention is to provide an information storage medium.

[0008]

In order to achieve the above object, a pointing position detecting device according to the present invention comprises: an image pickup means for picking up a display area on which an image is displayed; and a pointing image included in the picked up display area. And a position detecting means for detecting a pointing position from the image area.

[0009] The method of the present invention includes an imaging step of imaging a display area on which an image is displayed, and a position detection step of detecting a pointing position from an image area of a pointing image included in the imaged display area. It is characterized by the following.

Here, the position detecting means or step may be formed so as to detect a predetermined position of a shadow area of the pointing image included in the display area as a pointing position.

Further, the position detecting means or step may be formed so as to detect a predetermined position of a real image area of the pointing image included in the display area as a pointing position.

Thus, for example, when the presenter points to a predetermined position with a predetermined pointing tool, for example, a pointing tool such as a stick, the pointing position is changed to a shadow area or a real image of the pointing image included in the captured image. It can be detected based on the area.

Therefore, it is possible to realize a pointing position detecting device which is extremely easy to use.

Further, the presentation system of the present invention comprises: an image display means for displaying an image; an image pickup means for picking up an image of a display area on which the image is displayed; It is characterized by including position detecting means for detecting a pointing position, and cursor control means for controlling the position of a cursor included in an image displayed by the image display means based on the detected pointing position.

Further, the information storage medium of the present invention, based on an image pickup signal of a display area, information for detecting a predetermined position of an image area of a pointing image included in the display area as a pointing position, and Information for controlling the position of a cursor included in the image displayed by the image display means based on the pointing position;
Is stored.

Also, the method of the present invention includes an image display step of displaying an image, an imaging step of capturing an image of a display area on which the image is displayed, and pointing from an image area of the designated image included in the captured display area. A position detection processing step of detecting a position; and a cursor control step of controlling a position of a cursor included in an image displayed by the image display means based on the detected pointing position.

According to the present invention, when the presenter indicates a desired pointing position on the display area with a predetermined pointing device, for example, a bar-shaped pointing device, a finger, or the like, the pointing position is detected. , The cursor position is controlled. This allows the presentation to proceed smoothly.

In the present invention, a slightly larger pointing portion is formed at the tip of a pointing tool for forming a shadow or a real image of the pointing image, for example, a bar-shaped pointing tool, and this pointing section is formed by using a non-reflective member. It is preferable to form it. By doing so, the tip of the pointing rod can reliably form a shadow or a real image in the captured image, so that the pointing position can be detected more accurately.

Further, according to the presentation system of the present invention, based on the detected pointing position,
The position of the cursor displayed on the display area is controlled. By doing so, the cursor moves following the position indicated by the presenter, so that the pointing position can be displayed in a very easy-to-understand manner not only for the presenter but also for the viewer of this presentation. Can be. As a result, the presentation can proceed in a natural manner for both the presenter and the viewer, so that a presentation system that is easy to use and easy to understand can be realized.

Further, the position detecting means, information or step may be formed so as to detect a predetermined position of a shadow area of the pointing image included in the display area as a pointing position.

Thus, for example, a desired pointing position of the display area pointed by the presenter is
It can be detected as a shadow area on the display area.

Further, the position detecting means, information or step may be formed so as to detect a predetermined position of a real image area of the pointing image included in the display area as a pointing position.

Thus, for example, a desired pointing position in the display area indicated by the presenter can be detected as a real image area on the display area.

As described above, according to the present invention, when the presenter points to a predetermined position with the pointing tool such as a bar, the pointing position is changed to the shadow area of the pointing tool on the display area or the real image area. Can be detected as

It is preferable that the position detecting means, the information or the step is formed such that a bar-shaped image area included in the display area is regarded as an image area of the pointing image, and a tip position thereof is detected as a pointing position. .

Thus, from among all the detection targets,
A detection target other than the bar-shaped image area is determined as noise and is not recognized as an instruction image. For this reason,
An accurate pointing position can be detected without being affected by noise, environmental conditions, and the like.

Furthermore, since the tip of the image area of the bar-shaped pointing image is detected as the pointing position, the presentation proceeds as if the user were presenting while pointing at a blackboard or the like with a finger or a stick. be able to.

It is preferable that the position detecting means, the information or the step is formed so as to capture a bar-shaped image area as an image area of the pointing image from the continuity of the image area included in the display area.

By doing so, the noise image included in the captured image and the designated image can be more reliably separated and recognized, so that the pointing position can be detected more reliably.

[0030] It is preferable that the cursor control means, information or process displays a plurality of cursors simultaneously in the display area and independently controls the position of each cursor.

Further, the cursor control means, information or step includes means, information or step for fixing and displaying the cursor at a desired position in the display area.

By doing so, the cursor moves following the position indicated by the presenter, so that the viewer can easily recognize the position indicated by the presenter as the cursor position.

Further, there is the following example as a case of performing cursor control for a plurality of designated images. For example, when the presenter and the participant individually indicate the pointing position, in addition to controlling the cursor position at the position indicated by the presenter, the cursor is also moved to the position of the instruction image from the participant (listener). If you can control, you can see at a glance where each other wants to explain or ask a question. For this reason, natural discussion and discussion can be realized, and a presentation with a high degree of understanding can be realized.

Further, when the presenter gives a comparative explanation of two contents on the display screen, the first cursor is directly positioned at the first explanation point, and the first cursor is arranged at that position, and the cursor is fixedly displayed at that position. By pointing directly back to the point you want to compare and explain next,
The viewer can easily find the two designated portions because the two designated portions can be viewed at the same time.

Even if the presenter is in a state where he cannot leave the display area to search for the material and cannot give an instruction, the cursor position is kept, so that it is possible to smoothly return to the explanation. Therefore, an effective presentation can be realized without distracting the attention of the participants.

The cursor control means, the information or the step may be provided at a position which does not overlap the image area of the designated image.
In addition, it is preferable to control the position of the cursor so as to indicate the detected pointing position.

As described above, by displaying the display position of the cursor at a position on the display area that does not overlap with the position pointed by the presenter, the pointing position can be easily understood by a third party listening to the presentation. Can be conveyed accurately.

[0038] The cursor control means, information or process may control the size of the cursor to be displayed according to the size of the tip of the image area of the bar-shaped pointing image.

For example, when the tip of the designated image shielding area is small, the cursor is displayed large, so that the viewer looking at the display area can easily recognize the pointing position or the cursor.

Further, the apparatus or system of the present invention includes a screen having the display area, the image display means comprises image projection means for projecting an image on the display area, and the optical means of the imaging means comprises: The imaging conditions may be controlled in conjunction with the optical means of the image projection means.

Further, in the information storage medium of the present invention, the information for controlling the optical means of the image pickup means includes an image pickup condition which is linked with the optical means constituting the image projection means for projecting an image on the display area. It is preferable to include information for controlling

Usually, the presenter manually or automatically adjusts the focus and zoom of the optical means such as the lens of the image projection means while watching the display screen on the display area. On the other hand, the optical adjustment of the imaging means is often performed by the presenter while viewing the captured image. According to the present invention, by controlling the optical adjustment of the image pickup means in conjunction with the optical adjustment of the image projection means, the presenter is free from bothersome adjustment work (operation), and the usability of the system is improved.

Preferably, the apparatus or the presentation system of the present invention includes a screen having the display area, and the image projecting means is arranged to project an image from the front or the rear of the screen.

At this time, the front type presentation system that projects an image from the front of the screen, which is a display area, can proceed with the presentation while projecting a large screen for presentation on the screen from the image display means. However, in such a case, it is often difficult to directly touch the screen to indicate the position. Even in such a case, according to the present invention, the presenter is located between the imaging means and the screen, and creates a shadow on the screen so as to block a part of the optical path of the projection means, or to reduce the shadow of the imaging means. A desired position can be recognized as a pointing position without directly touching the screen by forming a real image of the pointing means in the captured image so as to block a part of the optical path.

Further, even in a rear-type presentation system that projects an image from behind the display area, a shadow area or a real image area of the designated image appears at a position where the presenter points to the display area. Therefore, the pointing position can be detected from the shadow or the real image.

[0046]

Next, a preferred embodiment of a presentation system to which the present invention is applied will be described in detail with reference to the drawings.

(1) Overall Description FIG. 1 shows an example of a presentation system when a front projection device is used as an image display device. An image for a predetermined presentation is projected from a projector 10 provided substantially in front of the screen. The presenter 30 gives a presentation to a third party while pointing the area on the screen where the image is displayed on the screen, that is, the desired position of the image in the display area 12 with the pointing stick 40.

FIG. 4 shows an example where the presenter 30 is giving a presentation.

As shown in FIG. 4A, the presenter 3
When 0 indicates a desired position of the display area 12 on the screen using the pointing rod 40, as shown in FIG. 4B, the display area 12, a part of the presenter 30, and the pointing rod 40 are moved to the display area. CCD camera 14 which is provided substantially in front of 12 and functions as an imaging means
Is captured as a captured image 20.

Here, an elongated rod-like shadow formed by blocking the light emitted from the projector 10 by the pointing rod 40 is referred to as a shadow area 300 of the pointing image. In addition, a part of the presenter 30 and the actual image of the pointing stick 40 that are captured in the captured image 20 are referred to as a real image area 302 of the pointing image, and these pieces of information (target) used in the pointing position detection processing.
Are referred to as a detection target 304.

The pointing position on the display area 12 indicated by the presenter 30 using the shadow area 300 of the pointing stick 40 is determined by the shadow area 30 of the pointing image projected on the display area 12 in the captured image 20.
It is detected as a leading end position 310 of zero. That is, the tip position 310 of the shadow area 300 of the bar-shaped instruction image is automatically detected as the pointing position, and predetermined data processing is performed.

Similarly, the display area 12 to which the presenter 30 designates using the real image area 302 of the pointing stick 40 is designated.
The upper pointing position is detected as the leading end position of the real image area 302 of the instruction image projected on the display area 12 in the captured image 20.

According to the present embodiment, the projector 10
The data processing is performed so that the cursor 200 projected from is displayed at the recognized pointing position. That is, in the display image projected on the display area 12, the cursor 200 included in this image
It moves following the zero pointing position. Thus, a third party listening to the presentation of the presenter 30 can recognize the pointing position accurately and easily from the position of the cursor 200 displayed on the display area 12.

As will be described later, the corners of the display area 12, particularly the four corners, have a comparatively lower luminance level than the central part. In such an area, the system according to the embodiment is configured to supplementarily illuminate the corner portion of the display area 12 using the light source 16 so that the shadow or the real image of the indicator rod 40 can be accurately detected. I have.

FIG. 2A shows an example of a rear type presentation system. This presentation system projects an image of the projector 10 from behind the display area 12 and captures the projected image using a CCD camera 14 provided behind the display area 12 and functioning as an imaging unit. An image projected by the projector 10 is projected onto the display area 12 via the reflection plates 18-1 and 18-2.

FIG. 2B shows another example of a rear-type presentation system. This system is a C-type display system in which an image of a display area 12 is provided substantially outside the front.
Images are taken by the CD camera 14.

In such a rear type system,
The presenter 30 moves the display area 1 with the pointing stick 40.
2 by pointing to a desired position on
The shadow of 0 and the real image appear in the captured image 20. The projected shadow is defined as a shadow area 300 of the designated image, and the real image is defined as a real image area 302 of the designated image.
Can be imaged.

Then, the shadow area 300 of the designated image or the leading end position 310 of the real image area 302 of the designated image is automatically recognized as the pointing position, and the above-described data processing is performed.

The image display means of the rear type system to which the present invention is applied is not limited to the projector 10 which is a projection type display device as described above.
An RT, a direct-view liquid crystal display panel, a PDP (plasma display panel), or another display device may be used.

FIG. 3 shows a specific example of the pointing rod 40. The pointing rod 40 used in the present embodiment has a wide pointing portion 42 provided at the tip of the rod portion. The pointing unit 42 is made of a material having a low reflectance. By doing so, since the light is not reflected at the tip of the pointing rod, a clear indication object 304 of the pointing image can be formed in the imaging area. Can be detected.

In the front type system shown in FIG. 1, the presenter 30 makes the pointing rod 40 contact the display area 12 directly by positioning the pointing rod 40 on the optical path between the projector 10 and the display area 12. Instead, the shadow area 300 of the designated image can be formed in the captured image 20 of the CCD camera 14. Similarly, by positioning the pointing rod 40 on the optical path between the display area 12 and the CCD camera 14, the real image area 302 of the pointing image is formed in the captured image 20 without bringing the pointing rod 40 into direct contact with the display area 12. can do. Therefore, the presenter 30 can give a presentation while pointing to a desired position on the display area 12 even from a position relatively distant from the display area 12.

FIG. 5 schematically shows a functional block diagram of the system according to the embodiment.

The system according to the present embodiment includes an input source 100 for supplying an image to the projector 10 and a CCD camera 1
And a processing unit 110 for automatically detecting a pointing position from the captured image of No. 4. The detailed configuration of the processing unit 110 will be described later.

Further, in the present embodiment, the projector 10 and the CCD camera 14 are formed as an integrated unit, but they may be arranged separately. In the case of such separate arrangement, particularly when performing a presentation while enlarging to a large screen in a dedicated large conference room or hall, by using together with the existing equipment prepared in the venue in advance, the type of equipment and Since it can be used while being deformed freely according to the installation location and surrounding environment, versatility and usability are improved.

The projector 10 is configured so that the projection size and direction can be set arbitrarily.
The CD camera 14 is also configured so that its imaging area can be set arbitrarily.

Further, the CCD camera 14 detects the target image 30 to be detected by the pointing rod 40 of the presenter 30.
Polarizing plates 10a and 14a are provided in each optical path of the projector 10 and the CCD camera 14, respectively, so that the image 4 can be captured as a clear image. These polarizing plates 10a and 14a are set so that the polarization directions thereof are different from each other. This allows CC
The D camera 14 can reliably capture the detection target 304 of the designated image from the captured image information without being significantly affected by the light emitted from the projector 10.

The CCD camera 14 has a zoom function for enlarging and reducing the image pickup area, so that an area larger than the display area 12 can be set as an image pickup area and an area smaller than the display area 12 can be set as an image pickup area. Is configured.

As a result, the imaging area can be arbitrarily set according to the usage environment and the imaging content, so that the usability of the system is extremely improved. When the imaging region is set wider than the display region, it is not necessary to align the imaging region with the display region.

In particular, by setting an imaging area that includes the presenter 30, it is possible to detect a more appropriate pointing position in consideration of the position of the presenter 30.

Further, the resolution of the CCD camera 14 can be substantially increased by setting the imaging range narrow, so that a specific area displayed on the display area 12 is imaged at a high resolution and a desired image is obtained. Image processing can also be performed.

(2) Detection of Pointing Position FIG. 6 shows a specific functional block diagram of the processing section 110.

In the present embodiment, the processing section 110
Binarization processing unit 112, pointing coordinate detection unit 11
6. It is configured to include the arithmetic processing unit 118. Such a processing unit 110 is, specifically, a CPU, various programs,
This is realized using a ROM as an information storage medium for storing data and the like, a RAM functioning as a work area, and the like.

The image signal output from the CCD camera 14 is input to a binarization processing unit 112. In the present embodiment, the CCD camera 14 outputs a monochrome image signal.

In this embodiment, the binarization processing section 11
2 compares the imaging signal with a reference value Vref, extracts a detection target 304 such as a shadow or a real image of the designated image from the captured image,
The pointing coordinate detector functions as a position detector that detects a pointing position.

Then, the binarization processing section 112 compares the luminance data of the image pickup signal output from the CCD camera 14 with the reference value Vref for binarization.
A process for extracting a detection target 304 of the designated image from among the images captured in the step is performed, and the processed data is output to the pointing coordinate detection unit as binary image data.

The pointing coordinate detection unit 116 extracts a block of the detection target 304 from the binarized image data output from the binarization processing unit 112, and points the pointing end of the detection target 304 by the pointing rod 40. And outputs the detection result to the arithmetic processing unit 118.

In the present embodiment, the pointing coordinate detecting unit 116 is configured to specify the pointing image based on the continuity of the image of the detection object 304 extending in a bar shape, and to detect the leading end as the pointing coordinates. I have. Therefore, the detection accuracy of the pointing position can be improved as compared with the case where the corner of the image of the detection target 304 is simply detected as the pointing coordinates.

The details will be described below.

FIG. 7A is a flowchart for explaining the outline of the pointing coordinate detecting process of the pointing coordinate detecting section 116. In the present embodiment, a detection process using the shadow area 300 of the instruction image as the detection target 304 of the instruction image will be particularly described. However, the real image area 302 of the instruction image also has a low luminance in the captured image 20. By extracting a region, a similar processing method can be applied.

First, from the binarized image data output from the binarization processing unit 112, the shadow area 300
Is performed. That is, as shown in FIG. 7B, the binarized image data output from the binarization processing unit 112 includes the presenter's shadow 30a and the pointing stick 40.
And a shadow area 300 of the instruction image, which is a shadow of the instruction image.

In step S10, the pointing coordinate detecting section 116 performs a process of extracting only the shadow area 300 of the designated image from the binarized image data as shown in FIG. 7B. This extraction process is performed by extracting an image portion of a rod-shaped continuous shadow from the binary image shown in FIG. 7B.

Next, the pointing coordinate detecting section 116
Performs the pointing position detection process in step S12. That is, a process is performed to specify where the pointing position is in the shadow area 300 of the bar-shaped instruction image extracted in FIG. 7B. That is, as shown in FIG. 7B, a process of determining which of the points A and B at both ends of the shadow area 300 of the bar-shaped pointing image is the pointing position is performed.

Here, as shown in FIG. 7B,
The binarized image data input from the binarization processing unit 112 includes the presenter's shadow 30a that is continuous with the B side of the shadow area 300 of the designated image. Therefore, the point A side not in contact with the shadow 30a of the presenter is specified as the pointing position. Therefore, as shown in FIG. 7B, the tip of the shadow area 300 of the bar-shaped instruction image
That is, the coordinates of the point A are detected as the pointing position and output to the arithmetic processing unit 118.

As described above, in the present embodiment, 2
When the binarization processing unit 112 obtains binarized image data representing a shadow area as shown in FIG.
At 0, a bar-shaped shadow image portion is extracted as the shadow area 300 of the pointing image. Next, at step S12, which side of both ends of the shadow area 300 of the extracted pointing image is the pointing position is specified. A process for detecting the coordinates of the position is performed. By doing so, it is possible to increase the pointing position detection accuracy as compared with the case where a simple corner portion of a shadow is specified as a pointing position from the binarized image data output from the binarization processing unit 112. it can.

Hereinafter, steps S10 and S1 will be described.
The process 2 will be described in more detail.

(2-1) Details of Step S10 FIG. 8 shows a specific example of the process of step S10, that is, the process of extracting the shadow area 300 of the designated image from the binarized image data output from the binarization processing unit 112. A specific example of the algorithm to be extracted is shown.

Here, the shadow area 300 of the bar-shaped pointing image is shown.
Is considered to be equivalent to the extraction of an elongated shape, and the processing is performed.

First, an elongated shadow is detected, and then the area of the elongated shadow is detected. Then, a region having a large area is recognized as having continuity, and is extracted as a shadow region 300 of the designated image. Hereinafter, the details will be described.

First, when the binarized image data 400 shown in FIG. 8B is input from the binarization processing unit 112, the pointing coordinate detection unit 116 starts the operation of extracting the shadow area 300 of the designated image (step S20). ).

As described above, the binarized image data 400 includes the shadow 30a of the presenter and the shadow area 300 of the pointing image which is the shadow of the pointing stick 40.

At steps S22 and S24, the input image data 400 is subjected to horizontal and vertical HPF processing, respectively. Here, the HPF process refers to a process of removing a shadow portion having a continuous length of a predetermined length or more in each of the horizontal and vertical line directions.

Next, the images obtained by the processing in steps S22 and S24 are combined in step S26 to obtain binary image data 402 from which the presenter's shadow 30a has been removed. That is, only the shadow image having a thin portion in the horizontal or vertical direction remains, and the other shadow images are removed.

Then, the obtained binary image data 402
Then, LPF processing for removing a shadow having a certain length or less is performed in step 28 to obtain the binary image data 404 to be labeled. Thereby, all the fine noises 440 are removed.

Further, in steps S30, S32, and S34, detection noise 450 other than the shadow area 300 of the designated image that cannot be removed in step 28 is removed. The process of removing the detection noise 450 is performed by first performing a labeling process in step S30, calculating an area for each label in step S32, and extracting a shadow having an area greater than or equal to a certain value in step S34.

That is, the lump of each detection noise 450 included in the binary image data 404 to be labeled is divided into individual regions, and A, B... E are labeled as shown in FIG. obtain.

Next, the number of pixels for each label (region) is counted. The actual processing may be performed simultaneously with the labeling processing. Thus, the area of the shadow area for each of the labels A, B,... E is calculated. Then, the label having a small area is determined as noise and removed from the labeling processing data 406 to obtain binary image data 408 including only the shadow area 300 of the designated image. In step S34, the label having the maximum area may be extracted. When detecting a plurality of pointing positions as described later, the label having the maximum area and the label having the next largest area are extracted. You may do it.

As an extraction method other than the area calculation in S32, the shadow area 300 of the designated image may be extracted from a continuous bar-shaped label. In this case, if the shape of the extracted labeling processing data 406 is rod-shaped and continuous, it can be determined as a shadow generated by the pointing rod 40, but if the detection noise 450 is small and not continuous, S22, It is determined as the detection noise of the binary image data that cannot be completely removed in each of the filtering processes in S24 and S28.

By the series of processes in steps S20 to S36, the presenter's shadow 30a and noise are removed from the binary image data 400 output from the binarization processing unit 112, and only the shadow region 300 of the elongated pointing image is removed. Can be obtained as binary image data 408.

The binary image data 4 subjected to the HPF processing
02, if the noise included is small, step S2
Step 8 does not need to be performed.

(2-2) Details of Step S12 FIG. 9 shows a flowchart for determining a pointing position from the shadow area 300 of the designated image included in the binary image data 408 obtained in the processing steps S20 to S30. A specific algorithm is shown.

In this process, the position of the most corner of the shadow area 300 of the designated image is obtained as pointing coordinates. or,
Even when the tip of the shadow area 300 of the designated image has a certain area, the coordinates of the corners are specified as pointing coordinates.

When this processing is started (step S4)
0), first, in step S42, 2 shown in FIG.
Shadow area 300 of the instruction image included in value image data 408
Are detected at both ends A and B in the longitudinal direction. A specific example of this detection processing will be described later with reference to FIG.

Next, in step S44, it is determined whether or not point B and the shadow image other than the shadow area 300 of the designated image are in contact or close to each other.

If YES is determined here, FIG.
As shown in (b), in the binary image data 400 (the binary image data before the processes of steps S22 to S34 are performed) output from the binarization processing unit 112, the point B is in contact with the shadow 30a of the presenter. I have. Therefore, in this case, the point A on the opposite side is obtained as the pointing coordinates.

If NO is determined in the step S44, then in a step S46, the point B is moved to or near the edge of the screen as in the binary image data 410 shown in FIG. 9B. It is determined whether it exists. FIG. 9 (B)
This is based on the assumption that the presenter 30 points a desired position on the display area with a long pointing stick from outside the screen 410. In such a case, the point B in the shadow area 300 of the designated image exists at the edge of the screen 410, and the point A on the opposite side is obtained as the pointing coordinate A.

If NO is determined in the step S46, then in a step S48, the shadow area 30 of the designated image is determined.
It is determined whether the shadow other than 0 is in contact with or close to the point A. Here, if it is determined as YES,
Point B is obtained as the pointing position.

If NO is determined in the step S48, it is determined in a next step S50 whether the point A is a point on the edge of the screen or a point close thereto. If the determination is YES here, the point B is obtained as the pointing position, and if the determination is NO, a detection error is output.

As described above, in the present embodiment, by the processing of steps S44 and S46, the pointing position can be accurately detected even when the presenter 30 stands on the left side toward the display area 12, and the processing of S48 and S46 can be performed.
By the processing in S50, the pointing position can be accurately detected even when the presenter 30 stands on the right side toward the display area 12. In this way, it is determined which of the points A and B at both ends of the shadow area 300 of the designated image is the pointing position, and this pointing position is detected as the (x, y) two-dimensional coordinates.

(2-3) Details of Step S42 FIG. 10 shows a specific example of the process of step S42 in FIG. Here, three binary image data including the shadow area 300 of the instruction image are shown. FIG. 10 (A)
Indicates that the shadow area 300 is in the horizontal direction, FIG. 10B is the vertical direction,
FIG. 10 (C) shows a case in which it is directed obliquely. The shape and size of the shadow area 300 of the pointing image change depending on the shape of the pointing rod 40 used. For example,
The shape of the shadow area 300 of the pointing image may be any of a round shape, a square shape, and a polygonal shape. In the present embodiment, a case of a long and thin elliptical shape will be described as an example.

In order to obtain the coordinates of both ends of the shadow area 300 of such an instruction image, first, as shown in FIG. 10, the upper left coordinates of the display area 12 are set to the origin (X, Y) =
As (0,0), the shadow area 3 of the designated image in the image
The four points where x and y of 00 are minimum and maximum (the end points on the coordinates, that is, the points where no detected coordinates exist afterwards in any direction) are obtained as a, b, c, and d. . a (Xmin, Ya) b (Xmax, Yb) c (Xc, Ymin) d (Xd, Ymax) And a, b, c, and d satisfy the condition of (Xmax−Xmin)> (Ymax−Ymin). If so, the points a and b are
42, the points A and B at both ends of the shadow area 300 of the designated image are specified. When a, b, c, and d satisfy the condition of (Xmax−Xmin) <(Xmax−Ymin), c , D are specified as points A and B at both ends of the shadow area 300 of the designated image.

Therefore, taking the shadow area 300 of the pointing image shown in FIG. 10 as an example, FIGS. 10A and 10C show a and b coordinates at the end points in the longitudinal direction, and FIG. The c and d coordinates are detected as longitudinal end points.

The pointing coordinate detecting section 116 according to the present embodiment always detects a position specified later as a pointing position based on the time-series data of the instruction using the pointing rod 40, and converts the detected position data. It is configured to output.

Thus, pointing coordinate detecting section 116 of the present embodiment detects the coordinates of the tip of shadow area 300 of the designated image as the pointing position, and outputs the detected position data to arithmetic processing section 118.

In the present embodiment, a portion formed by blocking the captured image 20 on the display area 12 with the pointing rod 40 is extracted as the shadow area 300 of the pointing image, and the pointing position is detected. A pointing part other than this, for example, a shadow formed by the finger of the presenter 30 may be recognized as the shadow area 300 of the pointing image, and the tip position of the finger may be automatically recognized as the pointing position.

In this embodiment, the CCD camera 14
Output a black and white image signal from the image signal, and use the binarization processing unit 112 and the pointing coordinate detection unit 116 to separate the shadow area 300 of the designated image from this image signal and detect the pointing position. Even when a color image signal is output from the image pickup device, the luminance level of the image signal and the reference value Vref are obtained by using the same method as in the above-described embodiment.
May be compared to extract the shadow of the pointing image to detect the pointing position.

(2-4) Shading Countermeasures Depending on the use environment of the presentation system, for example, as shown in FIG.
A part of the image projected above, in particular, a corner part, may have a low-luminance area 320a whose luminance level is lower than other areas. In such a case, the CCD camera 1
When the image is picked up at step 4 and binarization processing is performed by the binarization processing unit 112, the low-luminance area 320b is extracted as a shadow as shown in FIG. In particular, the shadow area 30 of the instruction image
When 0 overlaps with the low luminance area 320b, it is difficult to accurately detect the pointing position.

In order to solve such a problem, in the system according to the present embodiment, the illuminating means 16 such as a light or an LED array is used to intensively illuminate the low-luminance area 320a having a low luminance level on the display area 12. I do. By doing so, since the binarized image processed by the binarization processing unit 112 has no portion corresponding to the low-luminance region 320a, the shadow region 300 of the instruction image is converted from the binarized image data. Extraction can be performed well, and the pointing position can be detected.

(3) Data Processing Based on Detected Position Data The arithmetic processing unit 118 in FIG. 6 performs various data processing and image processing based on the pointing position detected data thus input.

In this embodiment, the arithmetic processing unit 1
Reference numeral 18 functions as the camera control unit 122, the cursor control unit 120, and the automatic reference value setting unit 114.

(3-1) Details of Camera Control Unit 122 The camera control unit 122 includes an operator operation unit 130,
Alternatively, based on information input from a PJ (projector) optical control unit 150 of the projector 10, various optical controls of the CCD camera 14, such as focus control, are performed. Here, particularly, the latter optical control based on information input from the PJ optical control unit 150 of the projector 10 will be described.

FIG. 22 is a diagram for explaining a method of automatically performing the optical adjustment of the imaging lens of the CCD camera 14 in conjunction with various adjustment operations such as focus and zoom adjustment of the projection lens of the projector 10.

FIG. 22A is a block diagram for explaining the overall configuration.

Normally, the presenter 30 manually adjusts the PJ lens adjustment mechanism 152, which is the optical adjustment mechanism of the projection lens of the projector 10, in order to optimally focus the display image projected from the projector 10. Or automatically by using an automatic adjustment mechanism. The adjustment amount of the PJ lens adjustment mechanism 152 is detected by the sensor 154 and input to the camera control unit 122.

The camera control section 122 has a lens control program 126 incorporated in advance to control the optical adjustment amount of the CCD lens based on the detection data of the sensor 154.
Execute The execution result is the C
The CCD optical control unit 22 is supplied to the CD optical control unit 22 and controls the CCD lens adjustment mechanism 24 that performs optical adjustment of the imaging lens. Thus, the projector 1
The imaging lens of the CCD camera 14 can be adjusted in conjunction with the control of the 0 projection lens.

As a result, the projector 10 and the C
Although it was necessary to adjust each lens of the CD camera 14 individually, according to the present embodiment, the projector 1
When the presenter 30 manually adjusts the projection lens 152 of 0 or automatically adjusts it,
Since the fine optical adjustment of the camera 14 is performed automatically,
The troublesome and troublesome work of fine optical adjustment of the CCD camera 14 can be omitted, and the operability and usability can be improved.

FIG. 22B is an algorithm for the processing of FIG. First, in step S202, the PJ lens adjustment mechanism 152 is operated by the user, and the adjustment (variable) amount is detected by the sensor 154 (step S2).
04). Next, based on the variable amount detected by the sensor 154, a control amount in the CCD lens adjustment mechanism 24 is obtained by the lens control program 126 (step S206), and the adjustment mechanism of the CCD lens is automatically controlled in step S208. .

The cursor control unit 120 operates the cursor 200 so as to indicate the detected pointing position.
Is performed. That is, the cursor 200 included in the image projected from the projector 10 is
The input source 100 is controlled so as to move following the pointing position of the input.

The automatic reference value setting section 114 performs processing for automatically setting the reference value Vref for binarization used in the binarization processing section 112 to an optimum value.

Next, details of the cursor control unit 120 will be described.

(3-2) Details of the Cursor Control Unit 120 The cursor control unit 120 determines whether the shadow area 3
The position of the cursor 200 is controlled so as not to overlap with 00 and to indicate the detected pointing position.

FIG. 11A shows an example of position control of the cursor 200. In this example, the display position of the cursor 200 is controlled at a position separated by a certain distance on the extension of the shadow area 300 of the instruction image. Specifically, the continuity of the shadow area 300 of the pointing image, which is the shadow of the pointing rod 40, is recognized, and the position control is performed so that the cursor 200 is positioned on an extension of the pointing area.

At this time, the cursor 200 is displayed one cursor away from the position where the pointing rod 40 and the cursor 200 do not overlap, for example, the tip of the pointing rod 40.

As a result, the cursor 200 moves to the pointing rod 40.
The pointing position of the presenter 30 can be reliably and easily displayed by the cursor 200 without being hidden by the shadow of.

Further, the cursor control unit 120 may be configured to control the size, shape, color, and the like of the cursor 200 displayed according to the thickness of the tip of the shadow area 300 of the designated image. For example, when the shadow area 300 of the instruction image is thin, it is difficult to identify the exact position of the tip end position. Therefore, by displaying the cursor 200 in a large size, the viewer viewing the display area 12 can point to the pointing position. Will be easier to understand.

Here, as shown in FIG. 11A, when the cursor 200 is displayed on the extension of the pointing rod 40, there is a possibility that the cursor 200 is hidden or an area that cannot be displayed appears at the periphery of the display area 12. There is.

For this reason, as shown in FIG. 11B, the pointing position approaches the peripheral portion of the display screen (FIG. 11B), and a fixed distance =
Only when d cannot be maintained, the cursor 200 may be displayed near the pointing position (shown in the figure) or overlapped (shown in the figure). Further, the size, color, shape (direction of the arrow), and the like of the cursor 200 may be controlled according to the distance between the tip of the shadow area 300 of the instruction image and the cursor 200 and the overlapping area. In the figure, is an example in which the cursor size and shape are modified,
Indicates that the color of the cursor has changed. By doing so, the cursor 200 does not become a shadow of the pointing rod 40 even at the peripheral portion of the display screen and does not become invisible to the viewer, and the pointing position is more easily understood.

FIG. 21 is a diagram for explaining an algorithm used when the cursor size is changed.

First, in step S162, as the area calculation processing in step S32 in FIG.
d, width = t is calculated. Here, in the present embodiment, a case where the end portion has a shape different from the shape of the branch portion of the indicator rod 40 has been described, but the tip portion has the same shape as the branch portion of the indicator rod 40. May be. In this case, only the length = d 'of the tip portion of the pointing rod 40 is cut out, and the tip shape may be considered as length = d' and width = t.

Next, the detection target 30 of the detected pointing image is detected.
4 is compared with the length = L which can be set arbitrarily in advance (step S164). If d <L, it can be determined that the tip shape is smaller than the setting. The size of the cursor 200 is largely changed. on the other hand,
If d> L, it can be determined that the tip is sufficiently large.

Next, in step S166, the width = t of the detection target 304 of the detected pointing image is compared with the width value k that can be arbitrarily set in advance. When t <k, similarly to the comparison of the lengths, it is determined that the tip shape is smaller than the setting, and the size of the cursor 200 is greatly changed. On the other hand, if t> k, it is determined that the tip is sufficiently large.

By performing such a process each time step S34 is performed, real-time cursor control becomes possible.

Further, the cursor control unit 120 operates as shown in FIG.
The display position of the cursor 200 may be controlled as shown in FIG.

For example, when the presenter 30 gives a presentation while standing on the left side toward the display area 12, the display area 12 is divided into a narrow area 220 at the right end and an area 230 other than the right end, and furthermore, the display area 12 at the right end is further divided. The area 220 is divided into an upper area 220a and another lower area 220b. Then, when the pointing position is in the area 230 other than the right end, the cursor 200 is displayed to the horizontal left just beside the pointing position, and when the pointing position is in the upper right area 220a, the cursor 200 is displayed below the pointing position. When the cursor 200 is displayed upward, and the pointing position is in the lower area 220b other than the upper right end, the cursor 200 is displayed downward above the pointing position.
Thereby, when the presenter stands on the left side of the display area 12 and makes a presentation, it is possible to prevent a situation in which the cursor 200 is hidden at the right end of the screen.

[0144] Also, if the presenter is in the display area 1
If you stand on the right end of 2 and give a presentation,
The regions 220a and 220b may be provided at the left end of the projection region, and the cursor 200 may be displayed in the same manner. In this case, it is preferable that the cursor 200 be displayed rightward horizontally on the main area 230 right beside the pointing position.

When the display of the cursor 200 is controlled as shown in FIG. 12A, the cursor control unit 120 causes the presenter 30 to move the cursor 200 to the right side of the display area 12 by the input operation from the operator operation unit 130 as described above. The display direction of the cursor may be switched between a case where the presentation is performed while standing and a case where the presentation is performed while standing on the left side.

By doing so, a presentation system that is easier for the user to use can be realized.

FIG. 12 (B) shows a specific example of an algorithm for displaying a cursor at a position horizontally separated by a certain distance from the tip coordinates of the shadow area 300 of the designated image, as shown in FIG. 12 (A). It is shown.

In step S100, the algorithm for displaying the cursor is started, and in step S102, the coordinates (x, y) of the tip of the pointing position are detected.

Next, in step S104, the tip coordinates (x ', y') of the cursor are calculated based on the detected coordinates (x, y), and the cursor coordinates (x ', y') are calculated.
y ′) is determined (S106).

Next, in step S108, processing for controlling and displaying the cursor position at the cursor coordinates (x ', y') is performed, and it is possible to display the cursor at a desired position on the display screen. I can do it.

FIG. 19 is a diagram for explaining a method of independently controlling the display of a plurality of cursors corresponding to a plurality of pointing positions when a plurality of pointing positions are detected.

First, in step S122, the process of extracting the indication image detection target 304 in step S34 of FIG. 8 is performed, and in step S124, the number of extracted labels is checked. Here, when the number of extracted labels is one, one cursor may be displayed so as to be processed normally. When the detection targets 304 of the plurality of designated images are extracted in step S124, the number is calculated in step S126, and processing for displaying a cursor corresponding to the number is performed in step S128.

A series of such processes are repeated to display a cursor at the position of the designated image.

FIG. 20 shows a specific example in which a plurality of cursors are displayed by the above-described method, and the display position of one of the cursors is fixed, and the position of only another cursor is freely controlled. Here is the algorithm.

As shown in FIG. 20, the presenter 3
0 indicates a desired position on the display area 12 using the pointing rod 40, and displays the first cursor 202 at the pointing position (steps S142 and S14).
4).

The presentation system according to the present embodiment is provided with a function switch 140 to which predetermined functions described later are assigned. By operating this function switch 140,
A command to temporarily stop the movement of the first cursor 202 is sent (step S146), and the display position of the first cursor 202 is fixed as shown in FIG. 20 (step S148).

Next, as shown in FIG. 20, the user points on the display area 12 other than the above, and the second cursor 204 is displayed at the new pointing position (steps S150 and S152).

As a result, two new and old pointing positions can be indicated simultaneously and separately. Especially,
It is preferable that the second cursor 204 displayed later is subjected to a process for distinguishing the second cursor 204 from the first cursor 202 displayed earlier, such as changing the color and shape from the first cursor 202 or blinking. .

The first cursor 20 fixedly displayed
2 may be formed so that the display position is fixed until the presenter 30 operates the function switch 140 again in step S154, or the fixed display may be released after a certain time has elapsed. The first cursor 202 whose fixed display has been released in this way is
Since it is not detected as the pointing image on the display area 12, it disappears from the screen as shown in FIG. On the other hand, the second cursor 204 that is continuously instructed is displayed as it is following the instruction on the screen,
After this, the second cursor 204 is as if the first cursor 2
02.

In addition to the functions described above, the function switch 140 is assigned a function capable of performing operations similar to mouse left click, right click, and the like.

The function switch 140 may be provided integrally with the pointing rod 40, or may be formed separately so that the presenter 30 can be carried around.

With the above configuration, the presenter 30 guides the cursor 200 to a predetermined position using the pointing rod 40, and operates the function switch 140 here. The same processing as when a left click or a right click is performed can be performed.

Further, the function switch 140
By assigning an underline function or handwritten character input to the image, the user can also input a handwritten character by drawing an underline at a predetermined position of the image displayed on the display area 12 or moving the pointing position.

The function switch 140
By assigning a function for temporarily stopping the movement of the cursor 200 to the presenter, in a state where the presenter 30 has moved the cursor to a predetermined position using the pointing rod 40,
By operating the function switch 140, the display position of the cursor 200 is changed to the function switch 140 next.
Can be fixed until is operated.

(4) Automatic setting of reference value Vref Next, the reference value Vre set by the reference value automatic setting section 114
Details of f will be described.

In order to reliably extract the shadow area 300 of the designated image from the image pickup signal output from the CCD camera 14, it is necessary to set the reference value Vref of the binarization threshold to an optimum value.

This reference value is determined when the image projected on the display area 12 is captured by the CCD camera 14.
It is necessary to set the value to be equal to or less than the value of the lowest luminance level included in the image signal. For example, when a monochrome imaging signal is output from the CCD camera 14 as in the present embodiment, it is necessary to set the reference value to a value equal to or lower than the black luminance level. This is because the lowest luminance level on the screen is black, whereas the luminance level of the shadow area 300 of the designated image is lower than the black luminance level.

FIG. 14 (A) shows an image captured by the CCD camera 1.
4 shows the luminance level of each area obtained by imaging.

Here, a pure white image (display white area, the same figure) is displayed on the display area 12 from the projector.
A black image (displayed black area, same figure) is projected, and a part of the display area 12 is shielded to form a state where the projected image is not displayed on a part of the display area 12 (real black area, same figure). did. FIG. 14A shows the luminance level of each region obtained by capturing this image with the CCD camera 14. The shadow (detected black area, same figure) formed by the pointing rod 40 is the black luminance level (VLC) of the video.
And the true black luminance level (VB), so that the shadow area 300 of the designated image is reliably extracted, so that the reference value Vref for binarization is used.
Must be set to a value smaller than the black luminance level (VLC) of the video and larger than the true black luminance level (VB). It should be noted that the brightness level in FIG.
4 represents the luminance level of one horizontal scanning line.

FIG. 14 (A) shows an ideal display area environment.
The brightness level of each area obtained when an image is projected on the upper side is shown. The brightness level when the image projected on the actual display area 12 is captured by the CCD camera 14 is, for example, as shown in FIG. become that way. FIG. 14 (B)
Is a luminance level when the luminance level of the image projected on the display area 12 is slightly higher in the center of the display area (shown in the figure) and becomes lower toward the edge of the display area (shown in the figure). Display area 1
A similar situation occurs in a different situation if there is a window near 2 or the indoor lighting environment is uneven.

Therefore, if the level difference ΔV (hereinafter referred to as a margin) between the black level 300a of the shadow area 300 of the designated image and the reference value Vref is set too small, the shadow area 300 of the center designated image is reliably detected. The situation that cannot be done arises. That is, as shown in the binarization processing output of FIG. 14B, when the luminance level 300a of the shadow area 300 (shown in the figure) of the center instruction image greatly varies, the shadow area 30
The luminance level 300a of 0 may not fall below the reference value Vref. In this case, the shadow area 300 cannot be reliably extracted.

For this reason, the reference value automatic setting section 114 of this embodiment is configured so that the reference value Vref can be automatically set to an optimum value.

The details will be described below.

FIG. 15A shows the display area 12.
A state in which the image to be projected is divided into a plurality of areas is shown above, and here, the image is divided into nine areas.

Then, the reference value automatic setting unit 1 of the embodiment
Reference numeral 14 can set a unique reference value Vref for each divided area.

For example, in divided areas 1, 2, and 3, the reference value is set to Vr as shown in FIG.
ef1, Vref2, and Vref3 are set.

As a result, the shadow area 300 of the designated image can be reliably detected even in the central portion of the screen having a high luminance level (portion shown in FIG. 14B).

Also, a reference value Vr unique to each divided area
In order to set ef, the calibration pattern image generation unit 124 according to the present embodiment causes the projector 10 to project a predetermined calibration pattern image onto the display area 12 on the screen. At this time, the reference value automatic setting unit 114 collates the data of the projected calibration pattern image with the image pickup signal obtained from the CCD camera 14 and sets an optimum reference value Vref for each divided area.

Here, the calibration pattern image shown in FIG. 15B is projected onto the display area 12. The calibration pattern image 500 includes nine areas 500-1, 500- corresponding to the divided areas.
Each area includes a black region 510 at the center and a white region 512 around the black region 510. Thus, each area 500-1, 500-2,.
.. By arranging a pattern in which regions 510 and 512 having different luminance levels of white and black are arranged in 500-9, the reference level of each divided area is set to the optimum value from the calibration pattern image 500. be able to.

When the reference value is set, the calibration pattern image projected from the projector may be formed based on black having the lowest luminance level. For example, a calibration pattern image in which all regions of the calibration pattern image are all set to the black level may be projected.

However, even when the same black is projected on the display area 12, the brightness of the black image picked up by the CCD camera 14 differs depending on whether the image is entirely black or a white area exists around the image. The levels will be slightly different.

In this embodiment, even in such a case,
In order to ensure that a reference value can be set to each optimal value, a calibration pattern combining black areas 510 and white areas 512 having different luminance levels is set in each area 500-1, 500-2,.
500-9, and the combined image is projected on the display area 12 as the calibration pattern image 500. Thereby, the reference value for binarization can be automatically set to an optimum value for each divided area.

FIG. 16 is a flowchart of the reference value automatic setting operation when the presentation system is started.

The reference value automatic setting section 114 starts automatic setting of the reference value when the system is started (step S6).
0). First, it is detected that the video projection switch of the projector 10 has been turned on (step S62), and FIG.
The calibration pattern image shown in (B) is projected from the projector 10 onto the display area 12 (step S6).
4).

Then, the reference value automatic setting unit 114
The brightness level data of the calibration pattern image that is known in advance is compared with the brightness level of the corresponding position of the imaging signal output from the CCD camera 14, and the optimum reference value Vref is set for each divided area (step). S66).

Here, for each area, from the image pickup signal,
The luminance level of the signal corresponding to the black area 510 of the calibration pattern image is read, and a reference value Vref is set for each divided area with a predetermined margin for the luminance level.

After the completion of the setting, the reference value automatic setting unit 114 terminates the projection of the calibration pattern image, switches the image input of the projector 10 to the normal image input mode (step S68), and performs the reference value automatic setting operation. Is ended (step S70).

As described above, according to the present embodiment, when starting up the system, the threshold value Vref for the binarization processing is set.
Can be automatically set to an optimal value for each divided region.

FIG. 17 shows a flowchart in the case where the reference value Vref is reset after the system is started.

When the user operates the reference value reset switch 132, the reference value automatic setting unit 114 detects the operation of the reset switch 132 (step S82), and starts the reference value reset operation of steps S84 to S90. .

Steps S84 to S88 are the same as steps S64 to S68 shown in FIG. 16, and a detailed description thereof will be omitted.

As described above, according to the present embodiment, the reference value can be automatically set to the optimum value even after the system is started up. For example, when the use environment changes during the presentation. (For example, when the outside of the window becomes brighter or darker, and the brightness level on the display area 12 changes), the user operates the reference value reset switch 132 to
The reference value Vref for the value conversion process can be automatically set to an optimum value according to the usage environment at that time.

FIG. 18 shows steps S64 and S64 in FIG.
66 and a specific algorithm for explaining the detailed processing of steps S84 and S86 in FIG.

First, in step S182, the reference value Vre
Preset the value of f or use the previously used reference value V
Select whether to use the value of ref as it is.

When the environment is completely the same as or equivalent to the previous use environment, No may be selected and the reference value Vref used last time may be used. In this case, the setting of the reference value can be omitted, which is convenient.

On the other hand, when Yes is selected in step S182 to preset the value of the reference value Vref, the number of regions for setting the reference value is specified (step S184), and its size is specified (step S186). . Thus, an area as shown in FIG. 15A can be determined. Next, in step S188, the type of the calibration pattern as shown in FIG. 15B is selected.

Next, luminance data for each divided area is detected (step S190), and the detected luminance data is compared with luminance data stored in advance in step S192. Based on the comparison result, the value of the reference value Vref is controlled to a desired value (step S194), and the value is stored (step S196). Next, in step 198, FIG.
6, steps S64 and S66, and step S of FIG.
The processing of 84 and S86 ends.

As described above, according to the present embodiment,
It is possible to automatically set the reference value Vref to an optimum value, and to accurately extract the shadow area 300 of the designated image from the imaging signal.

Further, in the present embodiment, a case where a different reference value is automatically set for each area has been described as an example. However, if necessary, a single reference value may be automatically set for all areas. You may comprise.

(5) Hardware Configuration of the Processing Unit Next, an example of a hardware configuration capable of realizing the processing unit 110 will be described with reference to FIG. In the device shown in the figure, a CPU 1000, a ROM 1002, a RAM 100
4, information storage medium 1006, image generation IC 1010, I
Are connected via a system bus 1016 so as to be able to transmit and receive data to and from each other. And I / O ports 1020-1 and 1020
-2,..., CCD camera 14, function switch 140, operator operation unit 130, projector 1
50, connected to other devices.

The information storage medium 1006 mainly stores programs and image data.

The CPU 1000 controls the entire apparatus and performs various data processing according to programs stored in the information storage medium 1006, programs stored in the ROM 1002, and the like. The RAM 1004 is storage means used as a work area or the like of the CPU 1000.
006 or given contents of ROM 1002 or CPU
1000 calculation results and the like are stored. A data structure having a logical configuration for realizing the present embodiment is constructed on the RAM or the information storage medium.

1 to 7, FIG. 10, FIG. 11, FIG.
The various processes described in 3 to 15 are described in FIGS.
2. an information storage medium 1006 storing a program for performing the processing shown in the flowcharts of FIGS. 16 to 22, a CPU 1000 operating according to the program, and an image generation I
This is realized by C1010 or the like. The image generation IC1
The processing performed in step 010 or the like may be performed by software using the CPU 1000 or a general-purpose DSP.

[0204] The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the present invention.

[0205] For example, a CCD camera or the like that captures an image as a color image may be used as the imaging means as necessary.

As for the method of extracting the real image area 302 of the pointing image, in addition to the detection of the luminance level based on the image pickup signal as described above, the present embodiment uses the arm of the presenter 30 or the pointing rod 4.
It may be detected from the movement of 0.

[Brief description of the drawings]

FIG. 1 is a schematic explanatory diagram of a front type presentation system to which the present invention is applied.

FIG. 2 is a schematic explanatory diagram of a rear-type presentation system to which the present invention is applied. FIG. 2A is an explanatory diagram in the case where a CCD camera is built in, and FIG.
FIG. 4 is an explanatory diagram in the case where it is attached externally.

FIG. 3 is an explanatory diagram showing an example of a pointing rod used in the present embodiment.

FIG. 4A is an explanatory view of a presentation operation, and FIG. 4B is an explanatory view of a shadow of a presenter and a pointing stick and a real image of the pointing stick in a captured image at this time.

FIG. 5 is a functional block diagram of the system according to the embodiment.

FIG. 6 is a detailed functional block diagram of a system according to the present embodiment.

FIG. 7 is an explanatory diagram of a process of extracting a shadow of a designated point from an imaging signal of a CCD camera and detecting a pointing position.

FIG. 8 is an explanatory diagram of a specific process for extracting a shadow of a designated point from an imaging signal of a CCD camera.

FIG. 9 is an explanatory diagram of specific processing for detecting a pointing position from a shadow of an extracted designated point.

FIG. 10 is an explanatory diagram of a process of detecting the coordinates of both ends of the shadow of the designated point.

FIGS. 11A and 11B are explanatory diagrams showing an example of cursor display based on a pointing position. FIG. 11A is an explanatory diagram in a case where the cursor is on an extension of the pointing rod, and FIG. FIG. 5 is an explanatory diagram for describing a method of displaying a cursor.

12A and 12B are explanatory diagrams showing another example of cursor display based on a pointing position. FIG. 12A is an explanatory diagram in a case where the direction of the cursor is changed, and FIG. 12B is a diagram showing a horizontal cursor. FIG. 5 is an explanatory diagram of an algorithm for displaying.

FIG. 13 is an explanatory diagram showing an example of a countermeasure for shading in the present embodiment.

14A and 14B are explanatory diagrams of a setting process of a binarization threshold value Vref for extracting a shadow of a designated point from an image pickup signal. FIG. 14A is an explanatory diagram in an ideal state, and FIG. FIG. 3C is an explanatory diagram in a case where there is a luminance difference in the display area, and FIG. 3C is an explanatory diagram in a case where the reference value is changed to improve the luminance difference.

FIG. 15A is an explanatory diagram of a process for dividing a display region into a plurality of regions and setting a unique reference value for each region; FIG. FIG. 9 is an explanatory diagram illustrating an example of a calibration pattern image used when setting a value.

FIG. 16 is a flowchart of an operation of automatically setting a reference value when the system is started.

FIG. 17 is a flowchart of a reference value resetting operation after the system is started.

FIG. 18 is a flowchart illustrating details of an operation of automatically setting a reference value in the present embodiment.

FIG. 19 is a flowchart illustrating details of a cursor display operation corresponding to a plurality of designated points in the present embodiment.

FIG. 20 is a flowchart illustrating details of a cursor position fixed display operation according to the present embodiment.

FIG. 21 is a flowchart illustrating a cursor size changing operation according to the present embodiment.

FIG. 22 is a block diagram and a flowchart for explaining an automatic setting operation of the optical control unit of the CCD camera in the present embodiment.

FIG. 23 is an explanatory diagram of a hardware configuration of a processing unit according to the present embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Projector 10a, 14a Polarizer 12 Display area 14 CCD camera 20 Captured image 22 CCD lens 24 CCD lens adjustment mechanism 40 Pointer 42 Pointing unit 110 Processing unit 112 Binarization processing unit 114 Automatic reference value setting unit 116 Pointing coordinate detecting unit 118 arithmetic control unit 120 cursor control unit 124 calibration pattern image generation unit 126 lens control program 132 reference value reset switch 140 function switch 150 PJ optical control unit 152 PJ lens 154 PJ lens adjustment mechanism 156 sensor 200 cursor 202 first cursor 204 first Two cursor 230 Area other than right end of display area 220 Lower right end of display area 300 Shadow of designated image 302 Real image of designated image 304 Detection pair Elephant 320 Low brightness area 400 Binary image data 440 Noise 450 Detection noise 500 Calibration pattern image 510 Black area 512 White area

Claims (26)

[Claims] 1. An image pickup means for picking up an image of a display area on which an image is displayed, and a position detecting means for detecting a pointing position from an image area of an instruction image included in the picked up display area. Pointing position detection device. 2. The pointing position detection device according to claim 1, wherein the position detection unit detects a predetermined position of a shadow area of the pointing image included in the display area as a pointing position. 3. The pointing position detecting device according to claim 1, wherein the position detecting means detects a predetermined position of a real image area of the pointing image included in the display area as a pointing position. 4. The presentation system according to claim 1, wherein the position detection unit detects a pointing position from an image area of a bar-shaped pointing image included in the display area. 5. The method according to claim 1, wherein the position detection unit captures a bar-shaped image region as an image region of the pointing image from the continuity of the image region included in the display region, and determines a position of a tip end thereof. A pointing position detecting device for detecting a pointing position. 6. An image display means for displaying an image, an image pickup means for taking an image of a display area on which an image is displayed, and a position detection for detecting a pointing position from an image area of a designated image included in the taken display area. And a cursor control means for controlling a position of a cursor included in an image displayed by the image display means based on the detected pointing position. 7. The presentation system according to claim 6, wherein the position detecting means detects a predetermined position of a shadow area of the pointing image included in the display area as a pointing position. 8. The presentation system according to claim 6, wherein the position detecting means detects a predetermined position of a real image area of the pointing image included in the display area as a pointing position. 9. The presentation system according to claim 6, wherein the position detection unit detects a pointing position from an image area of a bar-shaped pointing image included in the display area. 10. The position detection unit according to claim 6, wherein the position detection unit captures a rod-shaped image region as an image region of the pointing image from the continuity of the image region included in the display region, and sets a preceding position as a pointing position. A presentation system characterized by detecting. 11. The presentation system according to claim 6, wherein said cursor control means simultaneously displays a plurality of cursors in said display area and independently controls the position of each cursor. 12. The cursor control unit according to claim 6, wherein the cursor control unit performs cursor position control so as to indicate a detected pointing position at a position that does not overlap an image area of the pointing image. A presentation system characterized by the following. 13. The cursor control unit according to claim 6, wherein the cursor control unit controls a size of a cursor to be displayed according to a size of a tip of an image area of the bar-shaped instruction image. Presentation system characterized by the following. 14. The cursor control device according to claim 6, wherein the cursor control device includes a position fixing device for fixing and displaying the cursor at a desired position in the display area. Presentation system. 15. The imaging device according to claim 1, further comprising a screen having the display area, wherein the image display means comprises image projection means for projecting an image on the display area, and the optical means of the imaging means. Is a system for controlling imaging conditions in conjunction with optical means of said image projection means. 16. The display device according to claim 1, further comprising a screen having the display area, wherein the image projection unit is arranged to project an image from the front or the rear of the screen. Presentation system. 17. An image display means based on information for detecting a predetermined position of an image area of an instruction image included in the display area as a pointing position based on an image pickup signal of the display area, and based on the detected pointing position. And a computer-readable information storage medium storing information for controlling the position of a cursor included in an image displayed by the computer. 18. The information storage according to claim 17, wherein the information for position detection includes information for detecting a pointing position of a presenter from an image area of a bar-shaped pointing image included in the display area. Medium. 19. The information according to claim 17, wherein the information for position detection is such that a rod-shaped image area is regarded as an image area of the pointing image from the continuity of the image area included in the display area, and a tip position thereof is a pointing position. An information storage medium characterized by including information to be detected as an information. 20. The information processing apparatus according to claim 17, wherein the information for controlling the cursor includes information for simultaneously displaying a plurality of cursors in the display area and independently controlling the position of each cursor. Information storage medium. 21. The cursor according to claim 17, wherein the information for controlling the cursor is a position that does not overlap with the image area of the pointing image and that indicates a detected pointing position. An information storage medium containing information for controlling the position of the information storage medium. 22. The cursor according to claim 17, wherein the information for controlling the cursor is a size of a cursor to be displayed according to a size of a tip of an image area of the bar-shaped pointing image. An information storage medium including information for controlling the information storage. 23. The information according to claim 17, wherein the information for controlling the cursor includes position fixing information for fixing and displaying the cursor at a desired position in the display area. An information storage medium characterized by the following. 24. The information processing apparatus according to claim 17, wherein the information for controlling the optical unit of the imaging unit is linked to an optical unit constituting an image projecting unit that projects an image on the display area. An information storage medium including information for controlling an imaging condition. 25. An imaging step of imaging a display area on which an image is displayed, and a position detection step of detecting a pointing position from an image area of an instruction image included in the imaged display area. Pointing position detection method. 26. An image display step of displaying an image, an imaging step of capturing an image of a display area on which the image is displayed, and a position detection of detecting a pointing position from an image area of a pointing image included in the captured display area. A cursor position control method, comprising: a processing step; and a cursor control step of controlling a position of a cursor included in an image displayed by the image display means based on the detected pointing position.