JP3821194B2 - Pointing position detecting device and method, presentation system, information storage medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a pointing position detection apparatus and method for automatically detecting a pointing position in a display area, a presentation system, and an information storage medium.
[0002]
[Background Art and Problems to be Solved by the Invention]
Assume that a presenter makes a presentation while watching a screen displayed on a display area, for example. The presenter proceeds with the presentation while pointing to any point in the display area using the pointing device.
[0003]
In this case, it is important that the presenter can easily point to an arbitrary point on the display area, and the pointing position is surely recognized by the system and reflected in the image processing.
[0004]
However, the conventional system has a problem that it is extremely inconvenient for the user because a special pointing device must be used when the presenter indicates a desired point on the display area.
[0005]
For example, when a laser pointer is used to point to an arbitrary point on the display area, the pointing position is difficult to understand unless the periphery of the display area is dark to some extent. Further, since the point position is displayed as a small point, there is a problem that the pointing position is difficult to understand from this surface.
[0006]
In addition, when a presentation is performed using a pointing device such as a mouse while moving the cursor position displayed on the display area, the presenter must interrupt the presentation and perform a mouse operation each time. For this reason, there was a problem that presentation could not be performed smoothly and it was extremely inconvenient.
[0007]
The present invention has been made in view of such problems, and its object is to automatically and accurately recognize the pointing position in the display area without using a special point pointing tool. An object of the present invention is to provide a pointing position detection apparatus and method, a presentation system, and an information storage medium.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, a pointing position detection device of the present invention includes:
Imaging means for imaging a display area in which an image is displayed;
A reference value setting means for setting a reference value for separating the image area of the instruction image from the picked-up image based on the image pickup signal of the image pickup means;
Position detection means for comparing the imaging signal and the reference value to extract an image area of the instruction image and detecting a pointing position from the image area of the instruction image;
Including
The reference value setting means is configured to be able to change the reference value.
[0009]
The pointing position detection method of the present invention includes
An imaging process for imaging a display area in which an image is displayed;
A position detection step of comparing the imaging signal with a reference value for separating an image area of the instruction image and extracting the image area of the instruction image, and detecting a pointing position from the image area of the instruction image;
Including
The position detection step includes
The method includes a step of changing a reference value for separating the image area of the instruction image from the captured image based on the imaging signal of the imaging means.
[0010]
According to the present invention, in the display area where an image is displayed, the image area of the instruction image generated by the operation of the operator is extracted from the captured image of the imaging means, and the pointing position is automatically detected.
[0011]
Here, when the imaging signal output from the imaging means is a monochrome image, for example, a binarization threshold value may be set as a reference value in order to cut out the image area of the instruction image from the imaging signal.
[0012]
This reference value is basically set to a value equal to or less than the value of the black level luminance of the image picked up by the image pickup means. This is because the color with the lowest luminance on the screen is black, and the luminance of the image area of the instruction image is below the luminance level of black.
[0013]
At this time, if the margin between the black level of the image displayed on the display area and the reference value for binarization is set too large, the luminance level of the image area of the instruction image does not decrease below the reference value. This causes a problem that the image area of the instruction image cannot be detected.
[0014]
Conversely, if the margin of the reference value is too small, an image target that is not an image area of the instruction image is erroneously recognized as an image area of the instruction image. For example, when the luminance in the display area slightly changes depending on the surrounding environmental conditions, there arises a problem that it is erroneously recognized as the image area of the instruction image.
[0015]
Therefore, in such a system, there is a problem that it is difficult to detect an image area of an appropriate instruction image unless the reference value is set to an appropriate value according to the use environment.
[0016]
In the present invention, the reference value for separating the image area of the instruction image from the imaging signal can be changed. As a result, the above-described problems can be solved, the image area of the instruction image formed on the display area can be accurately extracted, and the pointing position can be automatically detected.
[0017]
The presentation system of the present invention
Image display means for displaying an image;
Imaging means for imaging a display area in which an image is displayed;
A reference value setting means for setting a reference value for separating the image area of the instruction image from the picked-up image based on the image pickup signal of the image pickup means;
Position detection means for comparing the imaging signal and the reference value to extract an image area of the instruction image and detecting a pointing position from the image area of the instruction image;
Cursor control means for controlling the position of the cursor included in the image displayed by the image display means based on the detected pointing position;
Including
The reference value setting means is configured to be able to change the reference value.
[0018]
  The present invention also provides:
  For controlling the presentation systemprogramIs a computer-readable information storage medium in which is stored,
  SaidprogramIs
  Reference value setting for setting a reference value for separating the image area of the instruction image from the captured image from the image pickup signal of the image pickup meansmeansWhen,
  Position detection for comparing the imaging signal with the reference value to extract an image area of the instruction image and detecting a pointing position from the image area of the instruction imagemeansWhen,
  Cursor control for controlling the position of the cursor included in the image displayed from the image display means based on the detected pointing positionLet the computer function as a means,
  Reference value settingmeansIs
  Change the reference valueThisIt is characterized by.
[0019]
The present invention also provides a method for detecting a pointing position in a presentation.
An imaging process for imaging a display area in which an image is displayed;
A position detection step of comparing the imaging signal with a reference value for separating an image area of the instruction image and extracting the image area of the instruction image, and detecting a pointing position from the image area of the instruction image;
A cursor control step for controlling the position of a cursor included in the image displayed by the image display means based on the detected pointing position;
Including
The position detection step includes
The method includes a step of changing a reference value for separating the image area of the instruction image from the captured image based on the imaging signal of the imaging means.
[0020]
According to the present invention, the presenter indicates a desired pointing position on the display area with a predetermined pointing tool, such as a bar-shaped pointing tool or a finger. Then, the pointing position is detected from the imaging signal of the imaging means, and the cursor position is controlled. Therefore, it is possible to realize a presentation system that is extremely convenient for users and third parties who listen to presentations.
[0021]
In particular, in the present invention, the reference value for separating the image area of the instruction image from the imaging signal is configured to be changeable. As a result, the image area of the instruction image formed on the display area can be accurately extracted, and as a result, the pointing position can be automatically detected and the cursor position can be controlled satisfactorily.
[0022]
Further, the position detection means, information or process includes
A predetermined position in the shadow area of the instruction image included in the display area may be detected as a pointing position.
[0023]
Thereby, for example, a desired pointing position in the display area indicated by the presenter can be detected as a shadow area on the display area.
[0024]
Further, the position detection means, information or process includes
You may form so that the predetermined position of the real image area | region of the instruction | indication image contained in the said display area may be detected as a pointing position.
[0025]
Thereby, for example, a desired pointing position of the display area pointed to by the presenter can be detected as a real image area on the display area.
[0026]
Thus, according to the present invention, when the presenter points to a predetermined position with an indicator such as a stick, the pointing position is detected as a shadow area or a real image area of the indicator on the display area. The cursor position can be controlled well.
[0027]
Further, the position detection means, information or process includes
It is preferable that a bar-shaped image area included in the display area is captured as an image area of the instruction image, and a tip position thereof is detected as a pointing position.
[0028]
As a result, detection targets other than the bar-shaped image region are determined as noise from all detection targets and are not recognized as instruction images. Therefore, accurate pointing position detection can be performed without being affected by noise, environmental conditions, and the like.
[0029]
Furthermore, since the tip of the image area of the stick-shaped instruction image is detected as the pointing position, the user can proceed with the presentation as if he / she made a presentation while pointing a blackboard or the like with a finger or a stick. .
[0030]
Further, the position detection means, information or process includes
It is preferable to form the bar-shaped image area as the image area of the instruction image because of the continuity of the image areas included in the display area.
[0031]
By doing in this way, since the other image which is noise contained in the captured image and the instruction image can be separated and recognized more reliably, it is possible to more reliably detect the pointing position.
[0032]
The apparatus and system of the present invention are
Calibration pattern image generating means for displaying a predetermined calibration pattern image on the display area from the image display means,
The reference value setting means includes
It is preferable that a reference value for extracting an image area of the instruction image is automatically set based on the calibration pattern image data and the imaging signal.
[0033]
Furthermore, information stored in the information storage medium of the present invention is:
Including calibration pattern control information for displaying a predetermined calibration pattern image on the display area from the image display means;
The reference value setting information is
It is preferable that information for automatically setting a reference value for extracting an image area of the instruction image is included based on the data of the calibration pattern image and the imaging signal.
[0034]
Specifically, the calibration pattern image is displayed from the image display unit toward the display area when the image display unit is started up or updated during the process.
[0035]
At this time, the reference value setting means automatically calculates a reference value for extracting the image area of the instruction image from the captured image based on the imaging signal obtained by imaging the displayed calibration pattern and the luminance data of the calibration pattern image. Set. For example, when the imaging signal is a monochrome image signal, a reference value having an optimum margin is automatically set for the black luminance level.
[0036]
At this time, the apparatus and system of the present invention
Including instruction means for setting a reference value,
When a command for setting a reference value is input by the instruction means,
The calibration pattern image generating means includes
A calibration pattern image is displayed on the display area from the image display means;
The reference value setting means includes
It is preferable that a reference value for extracting an image area of the instruction image is automatically set based on the calibration pattern image data and the imaging signal.
[0037]
Even in a presentation system used in the same place, the optimum reference value often changes depending on the time of use and the surrounding environment. For example, the optimum reference value is different between when used in the daytime and when used at night. Even when used in the same time zone, the optimum reference value is often different between a sunny day and a cloudy day.
[0038]
In addition, when the presentation takes a long time, the ambient brightness differs between the start of the presentation and the time when a predetermined time elapses, and the optimum reference value often varies.
[0039]
Even in such a case, according to the present invention, the calibration pattern image may be displayed on the display area from the image display unit by instructing the resetting of the reference value using the instruction unit. Thereby, the reference value setting means automatically resets the optimum reference value based on the imaging signal of the calibration pattern and the calibration pattern data.
[0040]
The reference value setting means includes
Preferably, the display area is divided into a plurality of areas, and a unique reference value is set for each of the divided areas.
[0041]
At this time, it is preferable that the calibration pattern image generating unit is configured to display a predetermined calibration pattern image having a calibration pattern for each of the divided areas on the display area.
[0042]
Further, it is preferable that the display area is divided into a corner area that is relatively dark and a central area that is relatively bright, and a unique reference value is set for each of the divided areas.
[0043]
In the information storage medium of the present invention,
The reference value setting information is
It is preferable that the display area is divided into a plurality of areas and includes information for automatically setting a unique reference value for each divided area.
[0044]
For example, when a projection projector is used as the image display means, if the display area is curved or the brightness of the black level of the screen is not uniform due to uneven brightness of the projected image, it is optimal for each area. It is necessary to set a proper reference value.
[0045]
According to the present invention, the display area is divided into a plurality of areas, and a unique reference value is set for each of the divided areas, so that even if there is uneven brightness on the display area, it is reliably indicated from the captured image. It is possible to extract the image area of the image and detect the pointing position.
[0046]
The calibration pattern image generation means includes
It is preferable that a predetermined calibration pattern image formed by combining a plurality of areas having different luminances is displayed on the display area.
[0047]
That is, at the time of setting the reference value, the calibration pattern displayed from the image display means is often set based on, for example, black having the lowest luminance level, so all the areas of the calibration pattern image are set to the black level. May be. However, even if the same black is displayed as an image in the display area, the imaging means picks up the same black when the surrounding area is all black and when there is a non-black area around it. However, the brightness level is slightly different.
[0048]
In the present invention, even in such a case, a calibration pattern image formed by combining a plurality of regions having different luminances is displayed so that the reference value can be reliably and optimally set. From this, the reference value is set. As a result, the image area of the instruction image can be more accurately extracted from the imaging signal, and the pointing position can be detected.
[0049]
In particular, it is possible to set a reference value for each divided region by configuring such a combination of a plurality of regions having different luminances to be displayed on the display region as a calibration pattern image set for each divided region. Can be set to a more appropriate value.
[0050]
DETAILED DESCRIPTION OF THE INVENTION
Next, a preferred embodiment of a presentation system to which the present invention is applied will be described in detail with reference to the drawings.
[0051]
(1) Overall description
FIG. 1 shows an example of a presentation system when a front projection device is used as an image display device. A predetermined presentation image is projected from a projector 10 provided substantially in front of the screen. The presenter 30 makes a presentation to a third party while pointing the desired position of the image on the screen, that is, the image in the display area 12 with the pointing bar 40.
[0052]
FIG. 4 shows an example in which the presenter 30 is giving a presentation.
[0053]
As shown in FIG. 4A, when the presenter 30 indicates the desired position of the display area 12 on the screen using the indicator bar 40, the display area 12 and the presenter 30 are shown in FIG. A part of the display bar 40 and the indicator bar 40 are provided substantially in front of the display area 12 and are picked up as a picked-up image 20 by the CCD camera 14 functioning as image pickup means.
[0054]
Here, a bar-like long shadow formed by the light emitted from the projector 10 being shielded by the pointer 40 is referred to as a shadow area 300 of the pointer image. In addition, a part of the presenter 30 and the actual image of the pointing bar 40 itself shown in the captured image 20 are referred to as a real image area 302 of the instruction image, and these pieces of information (objects) used for pointing position detection processing are used. These are collectively referred to as a detection target 304.
[0055]
The pointing position on the display area 12 indicated by the presenter 30 using the shadow area 300 of the pointing bar 40 is detected as the tip position 310 of the shadow area 300 of the instruction image displayed on the display area 12 in the captured image 20. The That is, the tip position 310 of the shadow area 300 of the bar-shaped instruction image is automatically detected as the pointing position, and predetermined data processing is performed.
[0056]
Similarly, the pointing position on the display area 12 indicated by the presenter 30 using the real image area 302 of the pointing bar 40 is detected as the tip position of the real image area 302 of the instruction image displayed on the display area 12 in the captured image 20. The
[0057]
According to the present embodiment, data processing is performed so that the cursor 200 projected from the projector 10 indicates the recognized pointing position. That is, in the display image projected on the display area 12, the cursor 200 included in the image moves following the pointing position of the pointer 40. Thereby, a third party who listens to the presentation of the presenter 30 can recognize the pointing position accurately and easily from the position of the cursor 200 displayed on the display area 12.
[0058]
As will be described later, the corner portion of the display area 12, particularly the four corner portions, has a relatively low luminance level compared to the central portion thereof. Even in such an area, the system of the embodiment is configured to use the light source 16 to illuminate the corner portion of the display area 12 so that the shadow and the real image of the pointer 40 can be accurately detected. Yes.
[0059]
FIG. 2A shows an example of a rear type presentation system. This presentation system projects an image of the projector 10 from the rear of the display area 12, and images the projected image using a CCD camera 14 provided behind the display area 12 and functioning as an imaging means. The projection image of the projector 10 is projected on the display area 12 via the reflectors 18-1 and 18-2.
[0060]
FIG. 2B shows another example of a rear type presentation system. This system captures an image of the display area 12 by a CCD camera 14 provided substantially outside the front thereof.
[0061]
Also in such a rear type system, when the presenter 30 indicates a desired position on the display area 12 with the pointing bar 40, a shadow or a real image of the pointing bar 40 is displayed in the captured image 20. The projected shadow can be captured by the CCD camera 14 as the shadow area 300 of the instruction image and the real image as the real image area 302 of the instruction image.
[0062]
Then, the shadow area 300 of the instruction image or the tip position 310 of the instruction image real image area 302 is automatically recognized as the pointing position, and the above-described data processing is performed.
[0063]
As the image display means of the rear type system to which the present invention is applied, in addition to the projector 10 which is a projection type display device as described above, for example, a direct view type CRT, a direct view type liquid crystal display panel, a PDP ( Plasma display panels) and other display devices may be used.
[0064]
FIG. 3 shows a specific example of the indicator bar 40. The pointing rod 40 used in the present embodiment is provided with a wide pointing portion 42 at the tip of the rod-shaped portion. The pointing unit 42 is made of a material having a low reflectance. By doing so, since no light is reflected at the tip of the pointing rod, a clear indication image detection target 304 can be formed in the imaging region. As a result, the pointing position can be more reliably determined from the imaging signal of the CCD camera 14. Can be detected.
[0065]
In the front type system shown in FIG. 1, the presenter 30 moves the indicator bar 40 on the optical path between the projector 10 and the display area 12 so that the indicator bar 40 does not directly contact the display area 12. A shadow area 300 of the instruction image can be formed in the captured image 20 of the camera 14. Similarly, by positioning the pointer 40 on the optical path between the display area 12 and the CCD camera 14, the real image area 302 of the pointer image is formed in the captured image 20 without the pointer 40 being in direct contact with the display area 12. can do. Therefore, the presenter 30 can make a presentation while pointing to a desired position on the display area 12 even from a position relatively distant from the display area 12.
[0066]
FIG. 5 schematically shows a functional block diagram of the system according to the embodiment.
[0067]
The system according to the present embodiment includes an input source 100 that supplies an image to the projector 10 and a processing unit 110 that automatically detects a pointing position from an image captured by the CCD camera 14. The detailed configuration of the processing unit 110 will be described later.
[0068]
In the present embodiment, the projector 10 and the CCD camera 14 are formed as an integral unit, but they may be arranged separately. When separated in this way, especially when performing presentations while enlarging a large screen in a dedicated large conference room or hall, it can be used in combination with existing equipment prepared in advance in the venue. Because it can be used while being transformed into a free shape according to the installation location and surrounding environment, versatility and usability are improved.
[0069]
The projector 10 is configured such that the projection size and direction can be arbitrarily set, and the CCD camera 14 is also configured so that the imaging region can be arbitrarily set.
[0070]
Further, the polarizing plate 10a is provided in each optical path of the projector 10 and the CCD camera 14 so that the CCD camera 14 can capture the detection target 304 of the instruction image formed by the instruction bar 40 of the presenter 30 as a clear image. , 14a are provided. These polarizing plates 10a and 14a are set so that the polarization directions thereof are different from each other. Accordingly, the CCD camera 14 can reliably capture the detection target 304 of the instruction image from the captured image information without being significantly affected by the light emitted from the projector 10.
[0071]
Further, the CCD camera 14 has a zoom function capable of enlarging and reducing the imaging area, and is configured so that an area wider than the display area 12 can be set as an imaging area, and an area narrower than this can be arbitrarily set as the imaging area. ing.
[0072]
As a result, the imaging area can be arbitrarily set according to the use environment and the imaging content, and the usability of the system is extremely improved. When the imaging area is set wider than the display area, alignment between the imaging area and the display area becomes unnecessary.
[0073]
In particular, by setting an imaging region that includes the presenter 30, it is possible to detect a more appropriate pointing position in consideration of the position of the presenter 30.
[0074]
Furthermore, since the resolution of the CCD camera 14 can be substantially increased by setting the imaging range narrow, a specific area displayed on the display area 12 is imaged at a high resolution, and desired image processing is performed. It is also possible to do this.
[0075]
(2) Pointing position detection
FIG. 6 shows a specific functional block diagram of the processing unit 110.
[0076]
In the present embodiment, the processing unit 110 includes a binarization processing unit 112, a pointing coordinate detection unit 116, and an arithmetic processing unit 118. Specifically, the processing unit 110 is realized using a CPU, a ROM that is an information storage medium that stores various programs, data, and the like, a RAM that functions as a work area, and the like.
[0077]
The image signal output from the CCD camera 14 is input to the binarization processing unit 112. In the present embodiment, it is assumed that the CCD camera 14 outputs a monochrome imaging signal.
[0078]
In the present embodiment, the binarization processing unit 112 compares the imaging signal with the reference value Vref, extracts a detection target 304 such as a shadow or a real image of the instruction image from the captured image, and the pointing coordinate detection unit performs the pointing operation. It functions as position detecting means for detecting the position.
[0079]
Then, the binarization processing unit 112 compares the luminance data of the imaging signal output from the CCD camera 14 with the reference value Vref for binarization, and designates an instruction image from the images captured by the CCD camera 14. The detection target 304 is extracted, and the processing data is output to the pointing coordinate detection unit 116 as binary image data.
[0080]
The pointing coordinate detection unit 116 extracts a mass of the detection target 304 from the binarized image data output from the binarization processing unit 112, and detects the tip of the detection target 304 as the pointing coordinate indicated by the pointing rod 40. The detection result is output to the arithmetic processing unit 118.
[0081]
In the present embodiment, the pointing coordinate detection unit 116 is configured to identify an instruction image based on the continuity of the image of the detection target 304 extending in a bar shape and detect the tip as a pointing coordinate. Therefore, the detection accuracy of the pointing position can be improved as compared with the case where the corner of the image of the detection target 304 is simply detected as the pointing coordinates.
[0082]
Details will be described below.
[0083]
FIG. 7A shows a flowchart for explaining the outline of the pointing coordinate detection process of the pointing coordinate detection unit 116. In the present embodiment, detection processing using the shadow area 300 of the instruction image as the instruction image detection target 304 will be described in particular. However, the actual image area 302 of the instruction image also has a low luminance in the captured image 20. A similar processing method can be applied by extracting the region.
[0084]
First, processing for specifying the shadow region 300 of the bar-shaped instruction image from the binarized image data output from the binarization processing unit 112 is performed. That is, as indicated by (1) in FIG. 7B, the binarized image data output from the binarization processing unit 112 includes the presenter's shadow 30a and the instruction image that is the shadow of the pointing bar 40. A shadow region 300 is included.
[0085]
In step S10, the pointing coordinate detection unit 116 performs a process of extracting only the shadow area 300 of the instruction image from the binarized image data, as indicated by (2) in FIG. This extraction process is performed by extracting a shadow image portion continuous in a bar shape from the binarized image shown in (1) in FIG.
[0086]
Next, the pointing coordinate detection unit 116 performs the pointing position detection process in step S12. That is, processing for specifying where in the shadow area 300 of the bar-shaped instruction image extracted in (2) of FIG. 7B is the pointing position is performed. That is, as shown by (3) in FIG. 7B, processing is performed to determine which of the points A and B at both ends of the shadow area 300 of the bar-shaped instruction image is the pointing position.
[0087]
Here, as indicated by (1) in FIG. 7B, the presenter's shadow 30a continuous with the B side of the shadow area 300 of the instruction image is added to the binarized image data input from the binarization processing unit 112. It is included. Therefore, the point A side that is not in contact with the shadow 30a of the presenter is specified as the pointing position. Therefore, as indicated by (4) in FIG. 7B, the tip of the shadow area 300 of the bar-shaped instruction image, that is, the coordinates of the point A is detected as the pointing position and output to the arithmetic processing unit 118. .
[0088]
As described above, in the present embodiment, when the binarization processing unit 112 obtains the binarized image data representing the shadow area as shown in FIG. 7B, first, in step S10, the bar-shaped shadow data is displayed. The image portion is extracted as the shadow area 300 of the instruction image, and then in step S12, which side of both ends of the shadow area 300 of the extracted instruction image is specified is the pointing position, and the coordinates of the pointing position are detected. Process. By doing so, the detection accuracy of the pointing position can be improved as compared with the case where the simple shadow corner portion is specified as the pointing position from the binarized image data output from the binarization processing unit 112. it can.
[0089]
Hereinafter, the processing of Step S10 and Step S12 will be described in more detail.
[0090]
(2-1) Details of Step S10
FIG. 8 shows a specific example of the processing in step S10, that is, a specific example of an algorithm for extracting the shadow area 300 of the instruction image from the binarized image data output from the binarization processing unit 112.
[0091]
Here, the extraction of the shadow region 300 of the bar-shaped instruction image is considered to be equivalent to the extraction of the elongated shape, and the processing is performed.
[0092]
First, an elongated shadow is detected, and then the area of the elongated shadow is detected. A region having a large area is recognized as having continuity, and is extracted as a shadow region 300 of the instruction image. Details will be described below.
[0093]
First, when the binarized image data 400 shown in FIG. 8B is input from the binarization processing unit 112, the pointing coordinate detection unit 116 starts an operation of extracting the shadow region 300 of the instruction image (step S20).
[0094]
As described above, the binarized image data 400 includes the presenter shadow 30a and the shadow area 300 of the instruction image which is the shadow of the instruction bar 40.
[0095]
The input image data 400 is subjected to HPF processing in steps S22 and S24 in the horizontal and vertical directions, respectively. Here, the HPF process refers to a process of removing a shadow portion having a certain length or more continuous in the horizontal and vertical line directions.
[0096]
Next, the images obtained by the processes in steps S22 and S24 are combined in step S26 to obtain binary image data 402 from which the presenter shadow 30a has been removed. That is, only a shadow image having a thin portion in the horizontal direction or the vertical direction remains, and other shadow images are removed.
[0097]
Then, the obtained binary image data 402 is subjected to LPF processing for removing shadows of a certain length or less in step 28 to obtain labeling target binary image data 404. Thereby, all the fine noises 440 are removed.
[0098]
Further, in steps S30, S32, and S34, detection noise 450 other than the shadow area 300 of the instruction image that could not be removed in step 28 is removed. The detection noise 450 is removed by first performing a labeling process in step S30, calculating an area for each label in step S32, and extracting a shadow having an area larger than the area in step S34.
[0099]
That is, each cluster of detected noises 450 included in the labeling process target binary image data 404 is divided into individual regions, and labeled with A, B... E, as shown in FIG.
[0100]
Next, the number of pixels for each label (area) is counted. The actual process may be performed simultaneously with the labeling process. Thereby, the area of the shadow area for each label A, B... E is calculated. Then, the label having a small area is determined as noise and removed from the labeling processing data 406 to obtain binary image data 408 including only the shadow region 300 of the instruction image. In step S34, a label having the maximum area may be extracted. When detecting a plurality of pointing positions as described later, a label having the maximum area and a label having the next largest area are extracted. You may do it.
[0101]
Further, as an extraction method other than the area calculation in S32, the shadow region 300 of the instruction image may be extracted from a continuous bar-shaped label. In this case, when the shape of the extracted labeling processing data 406 is rod-like and continuous, it can be determined as a shadow generated by the indicator rod 40, but in the case of a small and non-continuous detection noise 450, S22, It is determined as the detection noise of the binary image data that could not be removed in the filter processes of S24 and S28.
[0102]
Through the series of processes in steps S20 to S36, the presenter's shadow 30a and noise are removed from the binary image data 400 output from the binarization processing unit 112, and only the shadow area 300 of the elongated instruction image is extracted. Binary image data 408 can be obtained.
[0103]
If the noise included in the binary image data 402 subjected to HPF processing is small, the processing in step S28 need not be performed.
[0104]
(2-2) Details of Step S12
FIG. 9 shows a specific algorithm for determining the pointing position from the shadow area 300 of the instruction image included in the binary image data 408 obtained by the processing steps S20 to S30.
[0105]
In this process, the position of the corner of the shadow area 300 of the instruction image is obtained as pointing coordinates. Further, even when the tip of the shadow area 300 of the instruction image has a certain area, the coordinate of the corner is specified as the pointing coordinate.
[0106]
When this process is started (step S40), first, in step S42, the points A and B at both ends in the longitudinal direction of the shadow area 300 of the instruction image included in the binary image data 408 indicated by (1) in FIG. To detect. A specific example of this detection process will be described later with reference to FIG.
[0107]
Next, in step S44, it is determined whether or not the shadow image other than the shadow area 300 of the instruction image is in contact with or close to point B.
[0108]
If YES is determined here, as indicated by (2) in FIG. 9B, the binary image data 400 output from the binarization processing unit 112 (before the processing of steps S22 to S34 is performed). In the binary image data, point B is in contact with the shadow 30a of the presenter. Therefore, in this case, the point A on the opposite side is obtained as the pointing coordinate.
[0109]
If NO is determined in step S44, then in step S46, the point B is at or near the edge of the screen as in the binary image data 410 shown in (3) of FIG. 9B. It is determined whether or not it exists. 9B assumes that the presenter 30 points from the outside of the screen 410 to a desired position on the display area with a long pointer. In such a case, the point B of the shadow area 300 of the instruction image exists at the edge of the screen 410, and therefore the point A on the opposite side is obtained as the pointing coordinate A.
[0110]
If NO is determined in step S46, it is then determined in step S48 whether a shadow other than the shadow area 300 of the instruction image is in contact with or close to point A. If YES is determined here, the point B is obtained as the pointing position.
[0111]
If NO is determined in step S48, it is determined in next step S50 whether the point A is a point on the edge of the screen or a close point. If YES is determined here, the point B is obtained as the pointing position, and if NO is determined, a detection error is output.
[0112]
As described above, in the present embodiment, the processing of steps S44 and S46 can accurately detect the pointing position even when the presenter 30 stands on the left side toward the display area 12, and the processing of S48 and S50 can also be performed. Even when the presenter 30 stands on the right side toward the display area 12, the pointing position can be accurately detected. In this way, it is determined which point A or point B at both ends of the shadow area 300 of the instruction image is the pointing position, and this pointing position is detected as a two-dimensional coordinate (x, y).
[0113]
(2-3) Details of step S42
FIG. 10 shows a specific example of the process of step S42 of FIG. Here, three binary image data including the shadow area 300 of the instruction image are shown. FIG. 10A shows a case where the shadow region 300 is oriented in the lateral direction, FIG. 10B is oriented in the longitudinal direction, and FIG. 10C is oriented in the oblique direction. The shape and size of the shadow area 300 of the instruction image varies depending on the form of the instruction bar 40 used. For example, the shape of the shadow area 300 of the instruction image may be any of a round shape, a quadrangular shape, and a polygonal shape. In the present embodiment, the case of an elongated elliptical shape will be described as an example.
[0114]
In order to obtain the coordinates of both ends of the shadow area 300 of such an instruction image, as shown in FIG. 10, the upper left coordinate of the display area 12 is set to the origin (X, Y) = (0, 0). X, y of the shadow area 300 of the instruction image in the inside are the minimum and maximum four points (the end point on the coordinates, that is, the point where no detected coordinate exists in any direction thereafter) a, Obtained as b, c, d.
a (Xmin, Ya)
b (Xmax, Yb)
c (Xc, Ymin)
d (Xd, Ymax)
And a, b, c, d
(Xmax-Xmin)> (Ymax-Ymin)
If the above condition is satisfied, the points a and b are specified as the points A and B at both ends of the shadow area 300 of the instruction image in step S42, and a, b, c and d are
(Xmax−Xmin) <(Xmax−Ymin)
When the above condition is satisfied, the points c and d are specified as the points A and B at both ends of the shadow area 300 of the instruction image.
[0115]
Therefore, taking the shadow area 300 of the instruction image shown in FIG. 10 as an example, FIGS. 10A and 10C show the a and b coordinates as longitudinal end points, and FIG. 10B shows the c and d. Coordinates are detected as end points in the longitudinal direction.
[0116]
Note that the pointing coordinate detection unit 116 of the present embodiment always detects the position pointed later as the pointing position based on the time-series data of the instruction using the pointing bar 40, and outputs the detected position data. It is configured.
[0117]
In this way, the pointing coordinate detection unit 116 according to the present embodiment detects the coordinates of the tip of the shadow area 300 of the instruction image as the pointing position, and outputs the detected position data to the arithmetic processing unit 118.
[0118]
In the present embodiment, the part formed by the pointer 40 shielding the captured image 20 on the display area 12 is extracted as the shadow area 300 of the instruction image and the pointing position is detected. A shadow formed by the finger of the instruction unit, for example, the presenter 30 may be recognized as the shadow region 300 of the instruction image, and the tip position of the finger may be automatically recognized as the pointing position.
[0119]
In the present embodiment, a black and white image signal is output from the CCD camera 14, and the binarization processing unit 112 and the pointing coordinate detection unit 116 are used to separate the shadow area 300 of the instruction image from the image signal, and the pointing is performed. Although the position is detected, even when a color imaging signal is output from the CCD camera 14, the same method as in the above embodiment is used to compare the luminance level of the imaging signal with the reference value Vref and specify A shadow of the image may be extracted to detect the pointing position.
[0120]
(2-4) Shading measures
Depending on the usage environment of the presentation system, for example, as shown in FIG. 13A, a part of an image projected on the display area 12, particularly a corner portion, has a low luminance area 320a whose luminance level is lower than other areas. May occur. In such a case, when this is imaged by the CCD camera 14 and binarized by the binarization processing unit 112, the low luminance region 320b is extracted as a shadow as shown in FIG. 13B. In particular, when the shadow area 300 of the instruction image overlaps the low luminance area 320b, it is difficult to accurately detect the pointing position.
[0121]
In order to solve such a problem, in the system according to the present embodiment, the illumination means 16 such as a light or an LED array is used to concentrate and irradiate the low-luminance area 320a having a low luminance level on the display area 12. By doing so, the binarized image processed by the binarization processing unit 112 does not have a portion corresponding to the low luminance region 320a. Therefore, the shadow region 300 of the instruction image is removed from the binarized image data. It is possible to extract well and detect the pointing position.
[0122]
(3) Data processing based on detected position data
The arithmetic processing unit 118 in FIG. 6 described above performs various types of data processing and image processing based on the pointing position detection data input in this way.
[0123]
In the present embodiment, the arithmetic processing unit 118 functions as the camera control unit 122, the cursor control unit 120, and the reference value automatic setting unit 114.
[0124]
(3-1) Details of the camera control unit 122
The camera control unit 122 performs various types of optical control of the CCD camera 14, for example, focus control, based on information input from the operator operation unit 130 or the PJ (projector) optical control unit 150 of the projector 10. Here, in particular, optical control based on information input from the PJ optical control unit 150 of the latter projector 10 will be described.
[0125]
FIG. 22 is a diagram illustrating a method for automatically performing optical adjustment of the imaging lens of the CCD camera 14 in conjunction with various adjustment operations such as focus and zoom adjustment on the projection lens of the projector 10.
[0126]
FIG. 22A is a block diagram for explaining the entire configuration.
[0127]
In general, the presenter 30 manually adjusts the PJ lens adjustment mechanism 152 that is an optical adjustment mechanism of the projection lens of the projector 10 or automatically adjusts the focus of the display image projected from the projector 10. Or automatically using an adjustment mechanism. The adjustment amount of the PJ lens adjustment mechanism 152 is detected by the sensor 154 and input to the camera control unit 122.
[0128]
The camera control unit 122 executes a lens control program 126 incorporated in advance to control the optical adjustment amount of the CCD lens based on the detection data of the sensor 154. The execution result is supplied to the CCD optical control unit 22 in the CCD camera 14, and the CCD optical control unit 22 controls a CCD lens adjustment mechanism 24 that performs optical adjustment of the imaging lens. In this way, the imaging lens of the CCD camera 14 can be adjusted in conjunction with the control of the projection lens of the projector 10.
[0129]
Thus, conventionally, it has been necessary to individually adjust the lenses of the projector 10 and the CCD camera 14, but according to the present embodiment, the presenter 30 manually adjusts the projection lens 152 of the projector 10, or Since the fine adjustment of the CCD camera 14 is automatically performed by the automatic adjustment operation, the troublesome and troublesome fine adjustment of the CCD camera 14 can be omitted, and the operability and usability can be improved.
[0130]
FIG. 22B shows an algorithm of the processing of FIG.
First, in step S202, the PJ lens adjustment mechanism 152 is operated by the user, and the adjustment (variable) amount is detected by the sensor 154 (step S204). Next, based on the variable amount detected by the sensor 154, the control amount in the CCD lens adjustment mechanism 24 is obtained by the lens control program 126 (step S206), and the CCD lens adjustment mechanism is automatically controlled in step S208. .
[0131]
The cursor control unit 120 controls the position of the cursor 200 so as to indicate the detected pointing position. That is, the input source 100 is controlled so that the cursor 200 included in the image projected from the projector 10 moves following the pointing position of the pointing bar 40.
[0132]
The reference value automatic setting unit 114 performs processing for automatically setting the reference value Vref for binarization used in the binarization processing unit 112 to an optimum value.
[0133]
Next, details of the cursor control unit 120 will be described.
[0134]
(3-2) Details of cursor control unit 120
The cursor control unit 120 controls the position of the cursor 200 so as to point to the detected pointing position at a position that does not overlap with the shadow area 300 of the instruction image.
[0135]
FIG. 11A shows an example of position control of the cursor 200. In this example, the display position of the cursor 200 is controlled at a position separated by a certain distance on the extension line of the shadow area 300 of the instruction image. Specifically, the continuity of the shadow area 300 of the instruction image, which is the shadow of the instruction bar 40, is recognized, and position control is performed so that the cursor 200 is positioned on the extension line.
[0136]
At this time, the cursor 200 is displayed at a position where the pointer 40 and the cursor 200 do not overlap, for example, one cursor away from the tip of the pointer 40.
[0137]
Thereby, the cursor 200 is not hidden by the shadow of the pointing bar 40, and the pointing position of the presenter 30 can be displayed reliably and easily by the cursor 200.
[0138]
The cursor control unit 120 may be configured to control the size, shape, color, and the like of the cursor 200 to be displayed according to the thickness of the tip of the shadow area 300 of the instruction image. For example, when the shadow area 300 of the instruction image is thin, the accurate position of the tip position is difficult to identify. Therefore, the cursor 200 is displayed in a large size so that the viewer who is looking at the display area 12 can view the pointing position. Becomes easier to understand.
[0139]
Here, as shown in FIG. 11A, when the cursor 200 is displayed on the extension line of the pointing bar 40, there is a possibility that the cursor 200 may be hidden or an area that cannot be displayed appears at the periphery of the display area 12.
[0140]
For this reason, as shown in FIG. 11B, the pointing position approaches the peripheral edge of the display screen ((2) in the figure), and the constant distance = d cannot be maintained on the extension line of the shadow area 300 of the instruction image. Only in such a case, the cursor 200 may be displayed near the pointing position ((3) in the figure) or superimposed ((4) in the figure). Further, the size, color, or shape (the direction of the arrow) of the cursor 200 may be controlled in accordance with the distance between the tip of the shadow region 300 of the instruction image and the cursor 200 or the overlapping area. In the figure, (3) is an example in which the cursor size and shape are deformed, and (4) indicates that the color of the cursor has changed. By doing so, the cursor 200 does not become a shadow of the pointing bar 40 even at the peripheral edge of the display screen and is not visible to the viewer, and the pointing position becomes easier to understand.
[0141]
FIG. 21 is a diagram for explaining an algorithm used when the cursor size is changed.
[0142]
First, in step S162, length = d and width = t are calculated as the area calculation processing in step S32 of FIG. Here, in the present embodiment, the case of having a tip-shaped portion different from the shape of the branch portion of the indicator rod 40 is shown, but the tip portion having the same shape as the branch portion of the indicator rod 40 is provided. May be. In this case, only the length = d ′ of the tip portion of the indicator rod 40 may be cut out and the tip shape may be considered as length = d ′ and width = t.
[0143]
Next, the length of the detected indication image 304 in the longitudinal direction = d is compared with a length L that can be arbitrarily set in advance (step S164). Since it can be determined that the shape is smaller than the setting, the size of the cursor 200 is greatly changed. On the other hand, when d> L, it can be determined as a sufficiently large tip.
[0144]
Next, in step S166, the width of the detection target 304 of the detected instruction image = t is compared with a width value = k that can be arbitrarily set in advance. If t <k, it is determined that the tip shape is smaller than the setting, as in the length comparison, and the size of the cursor 200 is changed greatly. On the other hand, when t> k, it is determined as a sufficiently large tip.
[0145]
By performing such processing every time step S34 is performed, it is possible to control the cursor in real time.
[0146]
Further, the cursor control unit 120 may control the display position of the cursor 200 as shown in FIG.
[0147]
For example, when the presenter 30 makes a presentation while standing on the left side toward the display area 12, the display area 12 is divided into a narrow area 220 at the right end and an area 230 other than that, and the right end area 220 is further divided. The upper region 220a is divided into the lower region 220b. When the pointing position is in the area 230 other than the right end, the cursor 200 is displayed horizontally to the left next to the pointing position. When the pointing position is in the upper end area 220a at the right end, the cursor 200 is displayed below the pointing position. When the cursor 200 is displayed upward and the pointing position is in the lower area 220b other than the upper right end, the cursor 200 is displayed downward above the pointing position. Accordingly, when the presenter makes a presentation while standing on the left side of the display area 12, it is possible to prevent a situation in which the cursor 200 is hidden at the right end of the screen.
[0148]
When the presenter makes a presentation while standing at the right end of the display area 12, the areas 220a and 220b are provided at the left end of the projection area, and the cursor 200 may be displayed in the same manner. In this case, on the main area 230, it is preferable to display the cursor 200 horizontally rightward next to the pointing position.
[0149]
When the display control of the cursor 200 is performed as shown in FIG. 12A, the cursor control unit 120 causes the presenter 30 to stand on the right side of the display area 12 by the input operation from the operator operation unit 130 as described above. You may comprise so that the display direction of a cursor may be switched by the case where a presentation is performed and the case where a presentation is performed while standing on the left side.
[0150]
By doing so, it is possible to realize a presentation system that is easier to use for the user.
[0151]
FIG. 12B shows a specific example of an algorithm for displaying a cursor at a position that is a certain distance in the horizontal direction from the tip coordinates of the shadow area 300 of the instruction image, as indicated by (1) in FIG. It is shown.
[0152]
In step S100, an algorithm for displaying a cursor is started. In step S102, the tip coordinates (x, y) of the pointing position are detected.
[0153]
Next, in step S104, the tip coordinates (x ′, y ′) of the cursor are calculated based on the detected coordinates (x, y), and the cursor coordinates (x ′, y ′) are determined (S106). ).
[0154]
Next, in step S108, a process for controlling and displaying the cursor position at the cursor coordinates (x ', y') is performed, and the cursor can be displayed at a desired position on the display screen.
[0155]
FIG. 19 is a diagram illustrating a method for independently controlling display of a plurality of cursors corresponding to a plurality of pointing positions when a plurality of pointing positions are detected.
[0156]
First, in step S122, the instruction image detection target 304 in step S34 of FIG. 8 is extracted, and in step S124, the number of extracted labels is checked. Here, when the number of extracted labels is one, it is sufficient to display one cursor so that normal processing is performed. When the detection targets 304 of a plurality of instruction images are extracted in step S124, the number is calculated in step S126, and a cursor corresponding to the number is displayed in step S128.
[0157]
Such a series of processing is repeated to display a cursor at the designated image position.
[0158]
FIG. 20 shows a specific algorithm in the case where a plurality of cursors are displayed by the above-described method, in which the display position of one cursor is fixed and only another cursor is freely controlled. .
[0159]
As shown in (1) of FIG. 20, the presenter 30 indicates a desired position on the display area 12 using the pointing bar 40, and displays the first cursor 202 at the pointing position (steps S142, 144). ).
[0160]
The presentation system of the present embodiment is provided with a function switch 140 to which a predetermined function to be described later is assigned. Then, by operating this function switch 140, a command for temporarily stopping the movement of the first cursor 202 is sent (step S146), and as shown in (2) of FIG. The display position is fixed (step S148).
[0161]
Next, as indicated by (3) in FIG. 20, the second cursor 204 is displayed at a new pointing position by pointing on the display area 12 other than the above (steps S150 and 152).
[0162]
Thereby, the old and new pointing positions can be pointed simultaneously and separately. In particular, the second cursor 204 displayed later is subjected to processing for making it different from the first cursor 202 displayed earlier, such as making the color and shape different from those of the first cursor 202 or blinking display. Is preferred.
[0163]
It should be noted that the fixedly displayed first cursor 202 may be formed so that the display position is fixed until the presenter 30 operates the function switch 140 again in step S154, or the fixed display is released after a certain period of time. You may form so that. Since the first cursor 202 released from the fixed display in this way is not detected as an instruction image on the display area 12, it disappears from the screen as indicated by (4) in FIG. . On the other hand, the second cursor 204 that has been instructed continues to be displayed as it follows the instruction on the screen, and thereafter, the second cursor 204 is treated as if it were the first cursor 202.
[0164]
In addition to the functions described above, the function switch 140 is assigned a function capable of performing the same operation as a left click or right click of the mouse.
[0165]
The function switch 140 may be provided integrally with the indicator rod 40, or may be formed separately so that the presenter 30 can carry it.
[0166]
With the above configuration, the presenter 30 guides the cursor 200 to a predetermined position using the pointing bar 40 and operates the function switch 140 here, so that the cursor control unit 120 can be referred to as a so-called left mouse click or The same processing as when the right click is performed can be performed.
[0167]
Further, by assigning an underline function or handwritten character input to the function switch 140, an underline is drawn at a predetermined position of the image displayed on the display area 12, or a handwritten character input is performed by moving the pointing position. You can also.
[0168]
Further, by assigning a function for temporarily stopping the movement of the cursor 200 to the function switch 140, the function switch 140 is operated in a state where the presenter 30 has moved the cursor to a predetermined position by using the pointing bar 40. Then, the display position of the cursor 200 can be fixed until the function switch 140 is operated next time.
[0169]
(4) Automatic setting of the reference value Vref
Next, details of the reference value Vref set by the reference value automatic setting unit 114 will be described.
[0170]
In order to reliably extract the shadow area 300 of the instruction image from the imaging signal output from the CCD camera 14, it is necessary to set the binarization threshold reference value Vref to an optimum value.
[0171]
This reference value needs to be set to a value equal to or lower than the lowest luminance level included in the image signal when the image projected on the display area 12 is imaged by the CCD camera 14. For example, when a monochrome imaging signal is output from the CCD camera 14 as in the present embodiment, it is necessary to set the reference value to a value equal to or lower than the black luminance level. This is because the lowest luminance level on the screen is black, whereas the luminance level of the shadow area 300 of the instruction image is equal to or lower than the black luminance level.
[0172]
FIG. 14A shows the brightness level of each region obtained by capturing an image with the CCD camera 14.
[0173]
Here, a pure white image (display white area, (1) in the figure) and a black image (display black area, (2) in the figure) are projected on the display area 12 from the projector. Part of the display area 12 was shielded and a projection image was not displayed (real black area, (3) in the figure). FIG. 14A shows the luminance level of each region obtained by capturing this image with the CCD camera 14. The shadow (detected black area, (4) in the figure) formed by the indicator bar 40 is a luminance level (300a) between the black luminance level (VLC) and the true black luminance level (VB) of the image. In order to reliably extract the shadow region 300 of the instruction image, the reference value Vref for binarization is smaller than the black luminance level (VLC) of the video and larger than the true black luminance level (VB). It needs to be set to a value. Note that the luminance level in the figure represents the luminance level of one horizontal scanning line of the CCD camera 14.
[0174]
FIG. 14A shows an ideal display area environment, and shows the luminance level of each area obtained when an image is projected on the display area 12. However, FIG. The brightness level when the image projected onto the camera is picked up by the CCD camera 14 is, for example, as shown in FIG. In FIG. 14B, the brightness level of the image projected on the display area 12 is slightly higher in the center of the display area ((5) in the figure), and as it goes to the end of the display area ((6) in the figure). This is the brightness level when it is small. Even if there is a window near the display area 12 or the lighting environment in the room is biased, the same thing occurs although the state is different.
[0175]
Therefore, if the level difference ΔV (hereinafter referred to as margin) between the black level 300a of the shadow area 300 of the instruction image and the reference value Vref is set too small, the shadow area 300 of the center instruction image cannot be reliably detected. Will occur. That is, as shown in the binarization processing output of FIG. 14B, when the variation in the luminance level 300a of the shadow area 300 ((5) in the figure) of the central instruction image is large, the luminance level of the shadow area 300 is increased. 300a may not fall below the reference value Vref. In this case, this shadow region 300 cannot be extracted reliably.
[0176]
For this reason, the reference value automatic setting unit 114 of the present embodiment is configured to automatically set the reference value Vref to an optimum value.
[0177]
Details will be described below.
[0178]
FIG. 15A shows a state where an image projected on the display area 12 is divided into a plurality of areas, and here, the image is divided into nine areas.
[0179]
The reference value automatic setting unit 114 of the embodiment can set a unique reference value Vref for each divided area.
[0180]
For example, in the divided areas 1, 2, and 3, the reference values are set to Vref1, Vref2, and Vref3, respectively, as shown in FIG.
[0181]
As a result, the shadow area 300 of the instruction image can be reliably detected even in the central portion of the screen having a high luminance level (the portion (5) in FIG. 14B).
[0182]
In order to set a unique reference value Vref for each divided area, the calibration pattern image generation unit 124 of the present embodiment projects a predetermined calibration pattern image from the projector 10 onto the display area 12 on the screen. At this time, the reference value automatic setting unit 114 collates the data of the projected calibration pattern image with the image pickup signal obtained from the CCD camera 14, and sets the optimum reference value Vref for each divided area.
[0183]
Here, the calibration pattern image shown in FIG. 15B is configured to be projected onto the display area 12. The calibration pattern image 500 includes nine areas 500-1, 500-2,..., 500-9 corresponding to the divided areas, and each area has a black region 510 at the center and a white region around it. 512 is arranged. As described above, by arranging a pattern in which the areas 510 and 512 having different luminance levels of white and black are combined in each of the areas 500-1, 500-2,... Thus, the reference level for each divided area can be set to an optimum value.
[0184]
Note that when setting the reference value, the calibration pattern image projected from the projector may be formed with reference to black having the lowest luminance level. For example, a calibration pattern image in which the entire area of the calibration pattern image is entirely black level may be projected.
[0185]
However, even if the same black image is projected onto the display area 12, the brightness level of black captured by the CCD camera 14 is slightly different depending on whether the surrounding area is all black or a white area is present. It will be different.
[0186]
In the present embodiment, even in such a case, a calibration pattern combining a black region 510 and a white region 512 having different luminance levels is used for each region 500-1 so that the reference value can be reliably set to each optimum value. , 500-2,..., 500-9, and this combined image is projected onto the display area 12 as a calibration pattern image 500. Thereby, the reference value for binarization can be automatically set to an optimum value for each divided area.
[0187]
FIG. 16 shows a flowchart of the reference value automatic setting operation when starting up the presentation system.
[0188]
The reference value automatic setting unit 114 starts automatic setting of the reference value when the system is started up (step S60). First, it is detected that the video projection switch of the projector 10 is turned on (step S62), and the calibration pattern image shown in FIG. 15B is projected from the projector 10 onto the display area 12 (step S64).
[0189]
Then, the reference value automatic setting unit 114 collates the brightness level data of the calibration pattern image that has been determined in advance with the brightness level at the corresponding position of the imaging signal output from the CCD camera 14, and for each divided area. Is set to the optimum reference value Vref (step S66).
[0190]
Here, the luminance level of the signal corresponding to the black area 510 of the calibration pattern image is read from the imaging signal for each area, and the reference value Vref is set for each divided area with a predetermined margin with respect to this luminance level. .
[0191]
After completion of such setting, the reference value automatic setting unit 114 ends the projection of the calibration pattern image, switches the image input of the projector 10 to the normal image input mode (step S68), and ends the reference value automatic setting operation. (Step S70).
[0192]
Thus, according to the present embodiment, the threshold value Vref for binarization processing can be automatically set to an optimum value for each divided area when the system is started up.
[0193]
FIG. 17 shows a flowchart when the reference value Vref is reset after the system is started up.
[0194]
When the user operates the reference value reset switch 132, the reference value automatic setting unit 114 detects the operation of the reset switch 132 (step S82), and starts the reference value reset operation in steps S84 to S90.
[0195]
Note that steps S84 to S88 are the same processes as S64 to 68 shown in FIG. 16, and therefore detailed description thereof is omitted here.
[0196]
As described above, according to the present embodiment, the reference value can be automatically set to the optimum value even after the system is started up. For example, when the usage environment changes during the presentation (for example, When the brightness level on the display area 12 changes as the outside of the window becomes brighter or darker), the user operates the reference value resetting switch 132 to change the reference for binarization processing. The value Vref can be automatically set to an optimum value according to the use environment at that time.
[0197]
FIG. 18 shows a specific algorithm for explaining detailed processing of steps S64 and S66 of FIG. 16 and steps S84 and S86 of FIG.
[0198]
First, in step S182, it is selected whether to preset the value of the reference value Vref or whether to use the value of the reference value Vref that has been used before.
[0199]
When the environment is exactly the same as or equivalent to the previous use environment, No may be selected and the reference value Vref used last time may be used. This is convenient because the setting of the reference value can be omitted.
[0200]
On the other hand, when Yes is selected in step S182 and the value of the reference value Vref is preset, the number of areas where the reference value is set is specified (step S184) and the size is specified (step S186). Thereby, an area as shown in FIG. 15A can be determined. Next, in step S188, the type of calibration pattern as shown in FIG. 15B is selected.
[0201]
Next, luminance data for each divided area is detected (step S190), and the detected luminance data is compared with previously stored luminance data in step S192. Based on the comparison result, the reference value Vref is controlled to a desired value (step S194), and the value is stored (step S196). Next, in step 198, the processes in steps S64 and S66 in FIG. 16 and steps S84 and S86 in FIG. 17 are terminated.
[0202]
In this way, according to the present embodiment, it is possible to automatically set the reference value Vref to an optimum value and accurately extract the shadow region 300 of the instruction image from the imaging signal.
[0203]
In the present embodiment, the case where different reference values are automatically set for each region has been described as an example. However, if necessary, a single reference value is automatically set for all regions. Also good.
[0204]
(5) Hardware configuration of the processing unit
Next, an example of a hardware configuration capable of realizing the processing unit 110 will be described with reference to FIG. In the apparatus shown in the figure, a CPU 1000, ROM 1002, RAM 1004, information storage medium 1006, image generation IC 1010, I / O ports 1020-1, 1020-2,... . Are connected to the CCD camera 14, the function switch 140, the operator operation unit 130, the projector 150, and other devices via the I / O ports 1020-1, 1020-2.
[0205]
The information storage medium 1006 mainly stores programs and image data.
[0206]
In accordance with a program stored in the information storage medium 1006, a program stored in the ROM 1002, and the like, the CPU 1000 controls the entire apparatus and performs various data processing. The RAM 1004 is a storage means used as a work area of the CPU 1000 and stores the given contents of the information storage medium 1006 and the ROM 1002 or the calculation result of the CPU 1000. A data structure having a logical configuration for realizing the present embodiment is constructed on the RAM or the information storage medium.
[0207]
The various processes described with reference to FIGS. 1 to 7, 10, 11, and 13 to 15 are programs that perform the processes shown in the flowcharts of FIGS. 8, 9, 12, and 16 to 22. The information storage medium 1006 is stored, the CPU 1000 that operates according to the program, the image generation IC 1010, and the like. Note that the processing performed by the image generation IC 1010 or the like may be performed by software using the CPU 1000 or a general-purpose DSP.
[0208]
The present invention is not limited to the embodiment described above, and various modifications can be made within the scope of the gist of the present invention.
[0209]
For example, a CCD camera or the like that captures an image as a color image may be used as the imaging unit as necessary.
[0210]
The method for extracting the real image area 302 of the instruction image may be detected from the movement of the arm of the presenter 30 or the instruction bar 40 in addition to detecting the luminance level based on the imaging signal as described above.
[Brief description of the drawings]
FIG. 1 is a schematic explanatory diagram of a front type presentation system to which the present invention is applied.
FIG. 2 is a schematic explanatory diagram of a rear type presentation system to which the present invention is applied. FIG. 2 (A) is an explanatory diagram when a CCD camera is built in, and FIG. 2 (B) is externally attached. It is explanatory drawing in the case of having.
FIG. 3 is an explanatory diagram showing an example of an indicator bar used in the present embodiment.
FIG. 4A is an explanatory diagram of a presentation operation, and FIG. 4B is an explanatory diagram of a presenter, a shadow of a pointer, and a real image of a pointer in a captured image at this time.
FIG. 5 is a functional block diagram of a system according to the present embodiment.
FIG. 6 is a detailed functional block diagram of the system according to the present embodiment.
FIG. 7 is an explanatory diagram of processing for extracting a shadow of an instruction point from an imaging signal of a CCD camera and detecting a pointing position.
FIG. 8 is an explanatory diagram of specific processing for extracting a shadow of an instruction point from an imaging signal of a CCD camera.
FIG. 9 is an explanatory diagram of a specific process for detecting a pointing position from the shadow of an instruction point extracted.
FIG. 10 is an explanatory diagram of processing for detecting the coordinates of both ends of the shadow of the indication point.
11A and 11B are explanatory diagrams showing an example of a cursor display based on a pointing position. FIG. 11A is an explanatory diagram when the cursor is on an extension line of a pointing bar, and FIG. It is explanatory drawing for demonstrating the method of cursor display.
12A and 12B are explanatory diagrams showing another example of cursor display based on the pointing position. FIG. 12A is an explanatory diagram when changing the direction of the cursor, and FIG. 12B is a horizontal sideways cursor. It is explanatory drawing of the algorithm for displaying.
FIG. 13 is an explanatory diagram showing an example of a shading countermeasure in the present embodiment.
14A and 14B are explanatory diagrams of a setting process of a binarized threshold value Vref for extracting a shadow of an instruction point from an imaging signal. FIG. 14A is an explanatory diagram in an ideal state, and FIG. An explanatory diagram in the case where there is a luminance difference in the display area, FIG. 10C is an explanatory diagram in a case where the luminance difference is improved by changing the reference value.
FIG. 15A is an explanatory diagram of processing for dividing a display region into a plurality of regions and setting a unique reference value for each region; FIG. It is explanatory drawing which shows an example of the calibration pattern image used when setting a value.
FIG. 16 is a flowchart of a reference value automatic setting operation when the system is started up;
FIG. 17 is a flowchart of a reference value resetting operation after system startup.
FIG. 18 is a flowchart for explaining the details of the reference value automatic setting operation in the present embodiment;
FIG. 19 is a flowchart illustrating details of a cursor display operation corresponding to a plurality of instruction points in the present embodiment.
FIG. 20 is a flowchart illustrating details of a cursor position fixed display operation according to the present embodiment.
FIG. 21 is a flowchart illustrating a cursor size changing operation in the present embodiment.
FIGS. 22A and 22B are a block diagram and a flowchart for explaining an automatic setting operation of the optical control unit of the CCD camera according to the present embodiment. FIGS.
FIG. 23 is an explanatory diagram of a hardware configuration of a processing unit in the present embodiment.
[Explanation of symbols]
10 Projector
10a, 14a Polarizing plate
12 Display area
14 CCD camera
20 Captured image
22 CCD lens
24 CCD lens adjustment mechanism
40 indicator
42 Pointing part
110 processor
112 Binarization processing unit
114 Reference value automatic setting section
116 Pointing coordinate detector
118 Arithmetic control unit
120 Cursor control unit
124 Calibration pattern image generator
126 Lens control program
132 Reference value reset switch
140 Function switch
150 PJ optical controller
152 PJ lens
154 PJ lens adjustment mechanism
156 sensor
200 cursors
202 First cursor
204 Second cursor
230 Area other than right edge of display area
220 Below the right edge of the display area
300 Shadow of instruction image
302 Real image of instruction image
304 Detection target
320 Low brightness area
400 Binary image data
440 noise
450 detection noise
500 Calibration pattern image
510 black area
512 white area

Claims (8)

Imaging means for imaging a display area in which an image is displayed;
A reference value setting means for setting a reference value for separating the image area of the instruction image from the picked-up image based on the image pickup signal of the image pickup means;
Position detection means for comparing the imaging signal and the reference value to extract an image area of the instruction image and detecting a pointing position from the image area of the instruction image;
Including
The reference value setting means includes
A pointing position detection apparatus configured to divide the display area into a plurality of areas and to change a unique reference value for each of the divided areas. Image display means for displaying an image;
Imaging means for imaging a display area in which an image is displayed;
A reference value setting means for setting a reference value for separating the image area of the instruction image from the picked-up image based on the image pickup signal of the image pickup means;
Position detection means for comparing the imaging signal and the reference value to extract an image area of the instruction image and detecting a pointing position from the image area of the instruction image;
Cursor control means for controlling the position of the cursor included in the image displayed by the image display means based on the detected pointing position;
Including
The reference value setting means includes
A presentation system, wherein the display area is divided into a plurality of areas, and a unique reference value is set to be changeable for each divided area. In claim 2,
Calibration pattern image generating means for displaying a predetermined calibration pattern image from the image display means for each divided area of the display area,
The reference value setting means includes
A presentation system that automatically sets a reference value for extracting an image area of the instruction image based on the data of the calibration pattern image and the imaging signal. In claim 3,
The calibration pattern image generating means includes
A presentation system, wherein a predetermined calibration pattern image formed by combining a plurality of areas having different luminances is displayed for each divided area of the display area. In any one of Claim 3 and Claim 4,
Including instruction means for setting a reference value,
When a command for setting a reference value is input by the instruction means,
The calibration pattern image generating means includes
A calibration pattern image is displayed on the display area from the image display means;
The reference value setting means includes
A presentation system that automatically sets a reference value for extracting an image area of the instruction image based on the data of the calibration pattern image and the imaging signal. A computer-readable information storage medium storing a program for controlling a presentation system,
The program is
A reference value setting means for setting a reference value for separating the image area of the instruction image from the captured image from the imaging signal of the imaging means;
A position detection means for comparing the imaging signal and the reference value to extract an image area of the instruction image and detecting a pointing position from the image area of the instruction image;
Based on the detected pointing position, the computer functions as a cursor control means for controlling the position of the cursor included in the image displayed from the image display means,
The reference value setting means includes
An information storage medium, wherein the display area is divided into a plurality of areas, and a unique reference value is set to be changeable for each divided area. An imaging process for imaging a display area in which an image is displayed;
A reference value for separating an image area of the instruction image and an imaging signal to extract an image area of the instruction image, and a position detection step of detecting a pointing position from the image area of the instruction image;
Including
The position detection step includes
Changing the reference value for separating the image area of the instruction image from the captured image based on the imaging signal of the imaging means;
In the step of changing the reference value,
A pointing position detecting method, wherein the display area is divided into a plurality of areas, and a unique reference value is set to be changeable for each divided area. In a method for detecting a pointing position in a presentation,
An imaging process for imaging a display area in which an image is displayed;
A reference value for separating an image area of the instruction image and an imaging signal to extract an image area of the instruction image, and a position detection step of detecting a pointing position from the image area of the instruction image;
A cursor control step for controlling the position of the cursor included in the image displayed by the image display means based on the detected pointing position;
Including
The position detection step includes
Changing the reference value for separating the image area of the instruction image from the captured image based on the imaging signal of the imaging means;
In the step of changing the reference value,
A method for detecting a pointing position in a presentation, wherein the display area is divided into a plurality of areas and a unique reference value is set to be changeable for each divided area.

Images