JPH08123612A - Position input method and conference support system using same - Google Patents

Abstract

PURPOSE: To provide the position input method which prevents the fluctuation and deviation of an indication position that a user does not intends and stably displays the indication position at the position that the user intends. CONSTITUTION: When the user 2 makes a laser pointer 4 illuminate toward the screen 6 of a display device, a coordinate detecting device 12 detects the position of the bright point 8 of the laser light beam on the screen by using the image signal from a camera 10 for screen photography. A processor 18 decides whether or not the position is effective as an input and makes a display showing that the user indicates the position at the place on the screen that corresponds to the position of the bright point. Therefore, the effectiveness of the input is decided to prevent the fluctuation or deviation of the indication position that the user does not intends.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a position input method for a large-screen display used in a situation where a speaker holds a meeting in front of a large-screen display while pointing to a figure, table, or sentence displayed on the screen. A conference support system using the same.

[0002]

2. Description of the Related Art Various methods for designating a specific position on a display in order to select a menu or the like in a situation where a user gives a presentation or a conference in front of a large-screen display while pointing to a material displayed on the screen. There is.

As a conventional device for designating a position on a display, for example, a rear projection type display device described in JP-A-4-84191 is known. In this device, infrared light projection means for sequentially scanning and projecting infrared light horizontally and vertically on the transmissive screen and an infrared sensor for detecting the infrared light are provided behind the transmissive screen. And inputs a signal indicating the irradiation status of the infrared light projection means and the detection status of the infrared sensor,
When the infrared light is detected by the infrared sensor, the infrared light projection means determines that the infrared light projection device has given an instruction input to a predetermined position on the transmissive screen where the infrared light is currently projected. Have.

Further, as one method for designating a position on a display without requiring a special position detecting device to be attached to a hand, for example, an information input method described in Japanese Patent Laid-Open No. 4-271433 is known. ing. In this device, a plurality of image pickup signals of a pointing motion object such as a hand or an indicator are processed to obtain coordinates, and the position of the pointing motion object closest to the pointing target object such as a screen is set as a tip point, and a reference point is set. The pointed position coordinates on the pointed target object are obtained from the coordinates of the point and the tip point, and the cursor is displayed.

[0005]

When a user performs an instruction operation on a screen of a display in a presentation or a conference, the user does not always concentrate on the instruction operation. Often, attention is paid to other tasks such as explanation. In such a situation, it is difficult for the user to stably indicate the intended position. For this reason, the pointing position is deviated and is not stable, which makes it difficult for the audience to view the screen. However, the rear projection display device and the information input device described above do not describe a means or method for invalidating the input when the user's attention is deviated from the instruction operation.

A first object of the present invention is to prevent the user from intending to shake or shift the pointing position when the user's attention is directed to something other than pointing operation, and to display a stable pointing position. It is to provide a position input method that can be performed.

Further, when a presentation or a conference is held using a large screen, neither the user giving an instruction nor the audience side is always gazing at the instructed position, so that the cursor may lose sight. In the case of a large screen, it is difficult to search for a lost cursor due to the large viewing angle of the screen and the audience viewing from a distance.
The rear projection display device and the information input device described above do not describe means or methods for assisting the search of the cursor position when the cursor is lost.

A second object of the present invention is to provide a position input method which provides a display for assisting a user or an audience in searching for a cursor.

Further, when performing an instruction operation using a hand or an indicator, the instruction operation is easier when the menu display position is at the lower portion of the screen than at the upper portion of the screen. When the user who gives an instruction is not on the front of the screen but on either the left or right side, the instruction operation is easier if the menu is displayed closer to the user. Further, when the distance between the user and the screen is long, the operation for instructing the menu item becomes difficult due to camera shake unless the display size of the menu is sufficient. However, the above-mentioned rear projection display device and information input device do not describe a means or method for optimizing the display of the menu depending on the position of the user.

A third object of the present invention is to provide a conference support system capable of displaying a menu of a necessary and sufficient size at an optimum position according to the position of a user who gives an instruction.

[0011]

To achieve the above object, in the present invention, after detecting the position of the bright spot on the screen of the electromagnetic wave emitted by the indicator, the validity of the input is determined.
The user's gaze position is calculated, and if the user is not looking near the pointed position, the input is invalidated. Alternatively, it is determined whether or not the same position is being instructed for a certain period of time, and if it is not instructed, the input is invalidated.

In order to achieve the above object, the present invention is designed to display the indicated position by changing the brightness of the screen concentrically around the indicated position.

Further, for the above purpose, the present invention comprises means for detecting the position of the user who performs the pointing operation, and changes the display position and size of the menu to the optimum value according to the detected position of the user. I did it.

[0014]

[Function] In order to determine the validity of the input, the user's gaze point is determined, and the input is invalidated when the user is not looking near the pointed position, so the user's attention is directed to other than the pointing action. Even if the user does not see the pointed position, the pointed position display remains at the position where the pointed position was being used while the pointing operation was being performed, preventing the user from intending to shake or shift the pointed position. be able to.

Further, since it is determined whether or not the substantially same position is being instructed for a certain period of time and the input is invalidated if the instructing is not continued, the user pays attention to the instructing operation and intentionally instructs. Since the input is valid only when the user tries to do so, it is possible to prevent the blurring or shift of the pointing position which is not intended by the user.

As the indication position display, the brightness of the screen is changed concentrically around the indication position.
It becomes possible to know which direction on the screen the pointing position is, regardless of the position on the screen, and it is possible to assist the user or the audience in searching the cursor.

A means for detecting the position of the user who performs the instruction operation is provided, and the menu display position and size are changed to optimum values according to the detected position of the user.
Even if the position of the user who gives an instruction changes, a menu of a necessary and sufficient size can always be displayed at the optimum position.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings. The conference support system of the present embodiment detects the position of the bright spot of the laser pointer used as an indicator with a video camera installed behind the screen of the projection type large screen display and determines whether or not the position is effective as an input. Is determined, and when it is valid, a cursor is displayed at a place on the screen corresponding to the position of the bright spot as a display indicating that the position is designated.

FIG. 1 is a flow chart of an embodiment of the present invention, FIG. 2 is a diagram showing an example of the configuration of a conference support system for carrying out the present invention, and FIG. 3, FIG. 4, FIG. 5, FIG. ,
FIG. 7 is a flowchart of input validity determination processing, user position detection processing, and menu processing, which will be described below. FIG. 8 is a diagram showing an example of indication position display, and FIG. 9 is a diagram showing an example of user position division into regions performed during menu processing.

In FIG. 2, reference numeral 2 is a user who operates this conference support system, 4 is a laser pointer for instructing the user 2, 6 is a screen, and 8 is a screen 6. It is a bright spot of the laser pointer 4. Reference numeral 10 is a video camera for photographing the screen 6, and 12 is a coordinate detection device for obtaining the coordinates of the bright spot 8 from the image signal input from the screen photographing camera 10. Reference numeral 14 is a video camera for shooting the user 2, 16 is an image processing apparatus for extracting a user image from an image signal input from the user shooting camera 14, and 17 is an input from the image processing apparatus 16. The user state detecting device detects the position and the line-of-sight direction of the user 2 based on the size of the user image, the position in the image, the eyes, and the orientation of the head. A processing device 18 performs the following processing by using the coordinates input from the coordinate detection device 12 and the information of the line-of-sight direction and position of the user 2 input from the user state detection device 17 and outputs an image signal. Is. Reference numeral 20 is a document camera that captures characters and figures written on paper and converts them into video signals, and 22 is a document camera 20.
It is an image synthesizing device for synthesizing the image signal input from the image processing device and the image signal input from the processing device 18, and displays the output image on the screen 6. Reference numeral 24 is a cursor displayed on the screen 6 by the following processing, and 26 is a menu displayed on the screen 6 by the following processing.

The operation of the embodiment will be described below with reference to the flowchart of FIG.

When the processing is started (step 100), the processing device 18 outputs the image signal of the menu area at the position of the initial setting value (step 102), and the document camera 20 photographs the figure or character written on the paper. , And the image synthesizing device 22 synthesizes the image signals input from the processing device 18 and the document camera 20 and displays the image on the screen 6 (step 104).

Next, the screen photographing camera 10 inputs an image of the entire screen 6 and outputs an image signal. The coordinate detection device 12 searches the image signal input from the screen photographing camera 10 for a point having a brightness higher than a predetermined brightness, and if there is a corresponding point, the coordinate value thereof is used as the coordinate of the bright point 8 on the screen 6. The value is set (step 106). Then, the process proceeds to step 108 for determining whether or not this coordinate value is valid as an input.

In the following, input validity judgment processing (step 1
08) will be described with reference to FIG.

When the input validity determination process is started (step 300), it is determined whether or not the position data of the bright spot 8 has been detected (step 302). If the position data has not been detected, the input is invalidated (step 310), and the processing is performed. Is completed (step 314). If detected, the processing device 18 obtains the coordinate value of the gazing point of the user 2 on the screen 6 using the data of the line-of-sight direction of the user 2 detected by the user state detection device 17 (step 304). . The distance between the coordinate value of the bright spot and the coordinate value of the gazing point is obtained (step 306), and it is determined whether the distance is smaller than a preset constant value 1 (step 308). The value of the constant value 1 is, for example, 1/10 of the length in the vertical direction of the screen as a reference. Alternatively, the value of the constant value 1 may be given as a viewing angle of 3 ° with reference to the viewing angle of the user 2.
In this case, the constant value given by the viewing angle is converted into the distance on the screen 6 by using the input from the user state detecting device 17 which calculates the distance from the screen 6 of the user 2, and the value is used. Then, it is determined whether the distance is smaller than a preset constant value 1 (step 308). When the distance is larger than the constant value 1, the input is invalid (step 310), and when the distance is smaller than the constant value 1, the input is valid (step 312), and the process is terminated (step 314).

Input validity determination processing (step 108)
Another example of will be described with reference to FIG.

When the input validity judgment processing is started (step 400), it is judged whether or not the position data of the bright spot has been detected (step 402), and if not detected, the input is invalidated (step 408), and the processing is executed. The process ends (step 412). If it has been detected, the distances of all combinations are calculated for the coordinate value data of the bright points for the past fixed time, and the distance having the largest value is set as the displacement of the bright points during that time (step 404). It is determined whether this displacement is smaller than a preset constant value 2 (step 40).
6). Like the constant value 1, the constant value 2 is given based on the length in the vertical direction of the screen, the viewing angle of the user 2, or the like. If the displacement is larger than the constant value 2, the input is invalidated (step 408), and the process ends (step 41).
2). If the displacement is smaller than the constant value, the input is validated (step 410), and the process ends (step 41).
2).

Further, another example of the input validity determination process (step 108) will be described with reference to FIG.

When the input validity judging process is started (step 500), it is judged whether or not the position data of the bright spot could be detected (step 502), and if it is detected, the input is invalidated (step 508), and the process is executed. Finish (Step 5)
12). If it cannot be detected, the distances of all combinations are calculated for the coordinate value data of the bright points for the past fixed time up to the previous coordinate value, and the distance with the largest value is calculated as the bright point displacement during that time. (Step 504). It is determined whether this displacement is smaller than a predetermined constant value 3 (step 506). Similar to the constant value 1, the constant value 3 is given based on the vertical length of the screen or the viewing angle of the user 2. If the displacement is larger than the constant value, the input is invalidated (step 508), and the process ends (step 512). When the displacement is smaller than the constant value, the input is validated (step 510), and the process ends (step 512).

Input validity determination processing (step 108)
Is followed by a determination of whether the input is valid (step 11
0), and if invalid, return to step 102. If the position is valid, the position of the user is detected (step 112).

User position detection processing (step 11)
2) will be described with reference to FIG.

When the processing for detecting the position of the user is started (step 600), the camera 14 for photographing the user moves the user 2
The landscape of the conference room including the image is photographed and converted into an image signal, and the image processing device 16 extracts the image of the user from the image signal input from the camera 14 for photographing the user and displays the image of the user on the image. The size and position are obtained (step 604). The user state detection device 17 calculates the distance and direction from the screen 6 of the user 2 from the size and position data of the user image on the image input from the image processing device 16 (step 604),
The process ends (step 606).

Subsequently, menu processing (step 114) is performed.

The menu process (step 114) will be described with reference to FIG.

When the menu processing is started (step 70)
0), the processing device 18 uses the data of the distance and direction from the screen 6 of the user 2 input from the user state detection device 17 to set the display position of the menu (step 70).
2).

An example of how to set the menu display position in step 702 will be described with reference to FIG. The area in which the user 2 stands is previously set to the left area 902, the central area 904,
The right area 906 is divided. As a method of division,
For example, as shown in FIG. 9, a reference point 908 is set behind the screen 6, and a boundary is set so that the angle seen from that point becomes the reference angle 910. When the position of the user is the left area 902, the display position of the menu is set to the left position. When the position of the user is the central area 904, the display position of the menu is set to the lower position. The position of the user is the right area 90.
When it is 6, the display position of the menu is set to the right position.

Next, the processing unit 18 sets the display size of the menu by using the data of the distance and the direction from the screen 6 of the user 2 input from the user state detecting unit 17 (step 704).

An example of how to set the menu display size in step 704 will be described with reference to FIG. The distance boundary value 900 is set in advance. If the position of the user 2 is closer to the screen 6 than the distance boundary value 900, the menu size is set to a preset small size. If the position of the user 2 is farther from the screen 6 than the distance boundary value 900, the menu size is set to the preset large size.

After setting the display size of the menu as described above, the process is terminated (step 706).

Subsequently, a designated position display process (step 11)
Perform 6).

An example of the designated position display process (step 116) is to display the cursor 24, which is a graphic indicating that the position is designated, at the designated position.

Another example of the designated position display process (step 116) will be described with reference to FIG. Cursor 2 at the indicated position
4 is displayed, and the display brightness of the screen 6 is changed concentrically with the designated position as the center point as shown in FIG. 8 so that the center becomes brightest.

When the designated position display process (step 116) is completed, the process returns to step 102.

[0044]

According to the present invention, after the position of the bright spot on the screen of the laser pointer is detected, it is determined whether or not the bright spot position is effective as an input. As a method of determining effectiveness, the image signal of the user is processed to obtain the direction of the user's line of sight, the gaze point on the screen is obtained, and the input is invalid if the gaze point is not near the designated position. I decided to do it. When the user intentionally performs the pointing operation, he should be looking around the pointing position, so this determination processing should reflect the blurring or deviation of the pointing position not intended by the user in the pointing position display. It is possible to eliminate and display a stable pointing position.

Further, as another method of judging the validity of the input, it is judged whether or not the substantially same position is instructed for a certain period of time, and the input is invalidated if the instructing is not continued. Since the input is valid only when the user turns his or her attention to the pointing operation and intentionally gives an instruction, it is possible to prevent the pointing position from being unintended by the user.

Further, as another method for judging the validity of the input, when the same position is designated for a certain period of time and then the bright spot disappears, the position immediately before the bright spot disappears is validated as the input. . In this method, the user's intentional operation of turning off the bright spots is incorporated in the effectiveness determination, which has the advantage that the set value for a certain period of time can be reduced,
After all, it is possible to prevent the user from intending to shake or shift the designated position.

Further, in the present invention, the designated position is displayed in which the brightness of the screen is changed concentrically around the designated position. No matter where you look on the screen, the direction in which the display brightness changes can be obtained as visual information, so you can know which direction the pointing position is and assist the user or the audience in searching the cursor. You can

Further, according to the present invention, the user performing the instruction operation is photographed by the camera, the image signal thereof is processed to extract the user image, and the image of the user is extracted from the size and position of the image.
A means for detecting the position where the user is standing is provided, and the display position and size of the menu are changed to optimum values according to the detected position of the user. The operability of selecting a menu changes depending on the display position and display size of the menu. The optimum display position and size depend on the position where the user stands. According to the present invention, the operability can be always maintained in a good state by dynamically changing the menu display to an appropriate setting depending on the position of the user.

[Brief description of drawings]

FIG. 1 is a flowchart of an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration example of an apparatus that executes the present invention.

FIG. 3 is a flowchart of an input validity determination process of FIG.

FIG. 4 is a flowchart of an input validity determination process of FIG.

5 is a flowchart of the input validity determination process of FIG.

6 is a flowchart of a user position detection process of FIG.

FIG. 7 is a flowchart of the menu process of FIG.

FIG. 8 is a diagram showing an example of a screen on which the designated position display processing of FIG. 1 has been performed.

FIG. 9 is a diagram showing an example of area division of a user's position when performing the processing of FIG. 7.

[Explanation of symbols]

 2 ... User, 4 ... Laser pointer, 6 ... Screen, 8 ... Bright spot, 10 ... Screen shooting camera, 12 ... Coordinate detection device, 14 ... User shooting camera, 16 ... Image processing device, 17 ... User State detecting device, 18 ... Processing device, 20 ... Document camera, 22 ... Image synthesizing device, 24 ... Cursor, 26 ... Menu.

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor Keiyo Tokuda 3-529 Kitahama, Chuo-ku, Osaka City Hitachi West Software Co., Ltd.

Claims (9)

[Claims] 1. A display device displays a material composed of figures, characters, etc., and when a user emits a directional electromagnetic wave toward the screen of the display device, the position of the electromagnetic wave on the screen is detected. Position input characterized by performing processing to determine whether the obtained position is valid as an input, and displaying the user's instruction at the detected position only when the position is valid Method. 2. The position inputting method according to claim 1, wherein, as a process for determining whether the obtained position is valid as an input, the user's gaze position on the screen is detected, and the position of the electromagnetic wave and the gaze position are A position input method characterized by performing processing for determining that the input is valid only when the distance is less than a preset distance. 3. The position input method according to claim 1, wherein a process of determining whether or not the obtained position is effective as an input includes a displacement in a time series of the obtained position over a predetermined distance over a predetermined time. A position inputting method, characterized in that if there is no input, processing for determining that the input is valid is performed. 4. The position input method according to claim 1, wherein as a process of determining whether the obtained position is valid as an input, a displacement in a time series of the obtained position is equal to or more than a preset distance over a certain period of time. A position input method characterized in that when the time series ends without any input, the last position is used as an input to determine that the position is valid. 5. The position input method according to any one of claims 1 to 4, wherein a cursor, which is a graphic indicating the position, is displayed as a display indicating that the detected position is instructed by the user. And the position input method. 6. The position input method according to claim 1, wherein the display indicates that the detected position is instructed by the user, and the brightness of the screen is concentrically formed around the detected position. A position input method characterized by changing the. 7. An image input means for inputting an image, and a means for displaying the image obtained from the image input means by superimposing the display by the position input method according to any one of claims 1 to 6. A featured conference support system. 8. The conference support system according to claim 7, further comprising: a photographing device for photographing the user, and means for processing an image obtained from the photographing device to detect the position of the user. The conference support system is characterized in that the display position of the menu is selected and displayed from the left part, the lower part, and the right part of the screen according to the position of the user. 9. The conference support system according to claim 7, further comprising means for processing an image obtained from a photographing device for photographing the user and detecting a distance from the user's screen. A conference support system characterized in that the menu size is changed and displayed according to the distance of the user.