JP3517482B2 - Coordinate detection device and method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a coordinate detecting device and method,
More specifically, the present invention relates to a coordinate detection device and method for detecting a coordinate position designated by an operator.

[0002]

2. Description of the Related Art Conventionally, as a device of this type, a device of a type in which a dedicated tablet is indicated by a dedicated pen (called a digitizer), a rotary ball and a rotary encoder for detecting the rotating direction of the ball and the amount thereof are known. Built-in device (eg mouse or trackball)
Etc.

In many of these cases, the person who looks at the display screen is the person who operates the pointing device.

By the way, recently, it has become possible to digitize display information in advance and to give various presentations to a large number of people on the seat. Generally, in such a presentation, it is desired that the information is displayed on a relatively large screen, and it is often performed by connecting a projector or a dedicated OHP device to an information processing device (for example, a personal computer).

With such a system, attention has been paid to the fact that presentation materials can be created as if they were word processors.

[0006]

However, for example, even if some menus are displayed in the displayed (or projected) image, a dedicated operator is the one who ultimately selects the menu. A person can participate only as a recipient.

As a method for solving this, there are, for example, JP-A-4-18859 and JP-A-4-18860. These proposals disclose a technique including means for irradiating a pointing light beam and means for detecting the irradiated position provided on the screen.

Although it is possible to specify a desired position on the screen remotely with such a structure, the screen has a complicated structure, which is difficult to manufacture, and Very expensive. Also, since the screen is mandatory, the place to give a presentation is naturally decided.

[0009]

[Means for Solving the Problems] and

The present invention focuses on such a point, and provides a coordinate detection device and method with a simple configuration, which makes it possible to remotely designate a desired position while utilizing existing resources. Is what you are trying to do.

In order to solve this problem, for example, the coordinate detecting device of the present invention has the following configuration. I.e. remote
A coordinate detecting device for detecting a et indicated coordinate positions,
A pointing device that includes an imaging unit and is capable of freely changing the imaging direction , and a detection unit that detects at least two markers that specify a coordinate input space in an image supplied from the pointing device. If, based on the detected position of the marker, said against the coordinate input space
And deriving means for deriving a coordinate position pointing device is designated, designated by the pointing device
Judgment to determine whether the coordinate position is within the display screen area
The pointing device by the determining means and the determining means.
The coordinate position specified by the device is outside the display screen area.
If it is determined that the pointing device is present,
The coordinate position specified by the chair is outside the display screen area
And means for making it known .

[0011]

The marker in this case is preferably an identification marker image displayed at a predetermined position on the display screen. As a result, the display control side has only to perform the process of displaying the marker, and the image display side can use a general-purpose one.

The marker may be a removable physical marker. In this case, the display control side does not need to display a marker or the like, and the coordinates can be detected by installing the marker.

Further, it is desirable that the pointing device and the detecting means and the deriving means are at least separate bodies. As a result, the pointing device can be made very compact so that coordinate input can be performed remotely at intervals such as holding a microphone.

Further, it is preferable that the deriving means derives the coordinate position based on the detected two markers and predetermined shape information for specifying the coordinate input space. This allows coordinate input to be performed regardless of the distance between the operator and the place where the coordinate input space is placed.

The shape information is preferably an aspect ratio. As a result, the coordinate detection becomes extremely simple, and the detection speed can be made high.

Further, it is preferable that a general-purpose interface for transmitting the coordinate data derived by the deriving means is provided. As a result, the coordinates can be input only by connecting to a general-purpose information processing device.

It is preferable that the pointing device includes at least one button switch, and state information of the button switch is output together with a video signal. As a result, not only the input of coordinates but also various instructions can be performed.

Further, it is desirable to further comprise means for adjusting the relationship between the position of the marker and the coordinate input space.
As a result, normal coordinate input can be performed even in different coordinate input spaces.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment according to the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 1 shows a system configuration in the embodiment. In the figure, 1 is a CPU that controls the entire apparatus, 2 is a ROM that stores a boot program, and 3
Is a RAM used as an OS or application processed by the CPU 1 or a work area. Reference numeral 4 is an external storage device (for example, a hard disk device, a floppy disk device, a magneto-optical disk device, etc.) that stores an OS, application programs, and various kinds of information. A keyboard 5 is used to input various commands and characters. Reference numeral 6 is a VRAM which expands an image displayed on the display screen, and 7 is a display device which displays the image expanded in the VRAM 6. When various image data is expanded in the VRAM 6, the expanded image is displayed on the display device 7. Therefore, writing data in the VRAM and displaying the data are equivalent.

Reference numeral 9 denotes a coordinate input section operated by an operator, which is provided with buttons similar to those of an image pickup element and a general mouse device or a coordinate input pen. The imaged image data and the pressing information of this button are loaded into the apparatus via the interface 8 shown in the figure.

In the above configuration, when the power of the present system is turned on, the CPU 1 operates from the external storage device 4 according to the boot program stored in the ROM 2.
The program is read into the RAM 3 and the OS is operated.
Then, the operator operates the keyboard 5 and the coordinate input section 9 of the embodiment.
Start various applications based on the instructions from
The processing will be carried out. At this time, when the coordinate input unit 9 of the embodiment is used, a corresponding cursor is displayed on the display device 7, and the same work as a general mouse or digitizer can be performed. Although the cursor display control itself is well known and is not directly related to the present invention, the description thereof is omitted, but at least the OS receives coordinate data and button press information via the interface 8 Will be controlled.

FIG. 2 shows the system and coordinate input unit 9 of the embodiment.
An operation example of is shown.

As shown, the coordinate input unit 9 in the embodiment
Is a size that you can hold in your hand and allows you to remotely control the position of the cursor on the display screen.
In the figure, 9a indicates a button provided on the coordinate input section 9, and 101 is a cursor which is interlocked with the operation of the coordinate input section 9.

The principle for realizing the operation shown in FIG. 2 will be described in detail below.

The coordinate input section 9 has a built-in solid-state image pickup device (CCD) as described above. Then, the extended display screen position pointed by the coordinate input unit 9 is detected based on the captured image.

Generally, a very complicated process is required to recognize a picked-up image, but the inventor of the present invention realized such a process very easily.

Specifically, predetermined positions (at least two places) in the display area (hereinafter, screen area) of the display device 7
A marker distinguishable from the displayed image is displayed on the screen. Reference numerals 102 and 103 in FIG. 2 are markers. Here, it is conceivable that the marker has a special color, or the blinking period has a predetermined value. In any case, as long as the marker can be recognized, the function described below can be realized, and the method is not limited. Here, in order to simplify the description, the marker 102 is red and the marker 10 is
An example in which 3 is green will be described. However, in this case, the color used in the marker is not displayed in the screen area.

FIG. 3 shows an example of imaging by the coordinate input section 9. FIG. 13A shows an example in which the center position of the image pickup is outside the screen region where the image was picked up, and FIG.
Two markers (red marker and green marker) are used to determine whether the center position of imaging is within the display screen.
Can be detected, and the size and position of the screen area in the captured image that is uniquely determined from the position of the imaged marker can be calculated, and the determination can be made accordingly.

For example, if the aspect ratio (generally called the aspect ratio) of the screen area of the display device 7 is determined, as long as two markers are detected, the position of the marker and the aspect ratio of the display screen, The coordinates of the center position of imaging with respect to the display screen can be obtained. The advantage of using the aspect ratio is that the distance of the operator of the coordinate input unit 9 from the display device 7 can be ignored (the distance between the operator and the display screen can be ignored).

Therefore, in the state of FIG. 9A, the detected coordinate data may be processed as being out of the range, and in the state of FIG. 9B, the image pickup for the screen area determined by the marker is performed. The coordinates of the center position may be calculated.

The point to be noted here is that it suffices to know at which position in the screen area the directivity direction of the coordinate input section 9 is located. Therefore, how the coordinate input section 9 is held by the operator. It is not necessary to consider whether or not it is present. That is, it is not obligatory to hold the button 9a so that it is always on top.

The method of obtaining the coordinate position will be briefly described as follows.

FIG. 4A shows a screen area imaged by the coordinate input section 9 (area surrounded by points A, B, A ', B').

Here, it is assumed that the point A is a red marker and the point B is a green marker. Of course, the positions of the points A ′ and B ′ have not been determined at this point, but they can be easily obtained. That is, based on the distance between the points A and B and the point B, the angle θ1 from the horizontal axis to the point A can be easily detected (calculated).
it can. Further, since ∠ABA '(= θ2) is fixed, the angle θ3 from the horizontal line of the point A'is θ3 = θ1-θ2. That is, in this case, the image displayed on the screen can be erected by rotating the points A and P clockwise by θ3 with reference to the point B.

As a result, the points A'and B'can be simply obtained. That is, the point A ′ and the point B ′ are obtained by multiplying the length of the side AB by cos θ2 and sin θ2, respectively. More specifically, x of the point A ′ after the rotation processing
The coordinates are the same as the x-coordinate of the point A after the rotation processing, and the y-coordinate of the point B ′ after the rotation processing is the same as the y-coordinate of the point A after the rotation processing. After that, the coordinates x and y of the point P and the points A and B,
Whether or not the point P is within the screen area can be determined only by comparing the magnitudes of A ′ and B ′.

That is, the coordinates of each point after rotation are the coordinates of the point P (XP, YP), the coordinates of the point A (XA, YA),
When the point B is (XB, YB), the point P can be determined to be within the screen area when the following conditions are satisfied.

XB.ltoreq.XP.ltoreq.XA YB.ltoreq.YP.ltoreq.YA However, here, the lower left of the screen is the origin, the right direction is the positive direction of the X axis, and the upward direction is the positive direction of the Y axis. Of course, the positive direction of the axis is not limited to this.

The maximum coordinate value XMAX in the horizontal direction and the maximum coordinate value YMAX in the vertical direction of the screen area of the coordinate space used inside the information processing apparatus (the number of dots or the virtual absolute coordinate value adopted by the information processing apparatus). ) Is fixed, the coordinates X and Y utilized by the information processing of the point P are X = XMAX ・ (XP-XB) / (XA-XB) Y = YMAX ・ (YP-YB) / (YA-YB) Can be found at.

Now, when the CPU 1 in the apparatus (system) of the embodiment receives an instruction to acquire the coordinate position from the OS or application, it may detect the coordinate by the procedure shown in FIG. The processing procedure shown in the figure is initially stored in the external storage device 4, is read into the RAM 3 when the system is in an operating state, and is called at appropriate intervals.

First, in step S1, a picked-up image is input from the coordinate input section 9. Then, in step S2, it is determined whether or not two markers are present therein. If neither is detected, it means that the coordinate input unit 9 is not in the direction of the display screen.
Proceed to step 5 to perform error processing. For error handling,
For example, the caller may be notified of an error, or the latest coordinate data determined to be within the screen area may be notified as a detection result. In the latter case, the cursor could be left visible in the screen area, thus providing a natural environment for the operator.

When two markers are detected, the center position of the entire captured image is included in the captured screen area based on the position of each marker and the preset aspect ratio of the screen area. Is determined. If it is determined that it is outside the screen area, the error process of step S5 is performed.

When it is determined that the image is within the screen area, the coordinates of the center position of the entire captured image with respect to the screen area are calculated, and the calculation result and information indicating the pressed state of the button 9a are recorded. Return to the caller of the process.

As a result of the above, according to the present embodiment, the desired position of the display image can be designated remotely and without using a special display device.

In the above embodiment, the example in which the CPU 1 which controls the whole of the information processing apparatus detects the coordinate position is shown. However, for example, a circuit for detecting the coordinate may be independent. In general, an information processing device (for example, a personal computer) is provided with an interface called an expansion slot for expanding various functions, so that a card or board for performing the above coordinate detection may be mounted. By doing so, the functions of the above-described embodiments can be realized by a general-purpose device, and there is no need for a special information processing device.

<Description of Second Embodiment> In the above embodiment,
Although it has been described that the aspect ratio of the display screen of the display device 7 is constant, it is not unusual for a device that can finely adjust the aspect ratio. Further, when the same presentation data is displayed on another display device of another device, there is no guarantee that the aspect ratio is always constant between the display devices.

Therefore, in the second embodiment, a process for determining the aspect ratio is added before the actual operation with the coordinate input section 9.

The apparatus configuration is the same as that in FIG. 1, and in the following, the processing to be started when a start instruction for fine adjustment processing of coordinate input (for example, a start instruction of the corresponding program from the keyboard) is issued will be described with reference to FIG. explain. Before issuing a processing start instruction, the operator should fix the coordinate input unit 9 so that it faces the display screen appropriately (not touch it).

First, in step S11, a marker is displayed at the upper right corner position on the display device 7. Step S1
At 2, it is determined whether or not this marker is within the imaging range of the coordinate input unit 9. If not, the direction of the coordinate input unit 9 is adjusted and the determination in step S12 is yes.
Adjust until s. If the coordinate input unit 9 has a zoom function, the determination result of step S12 is NO if the magnification is set to a small value.
Is rare.

When the marker at the upper right corner of the screen area is detected, the marker position in the entire captured image is stored and held. Next, in step S14,
Now display the marker in the upper left corner of the screen area,
The same process is performed (steps S15 and S16).

Then, a marker is also displayed in the lower right corner of the screen area to detect the marker (steps S17, S).
18).

In this way, when all the markers at the positions that specify the shape of the screen are detected, the aspect ratio of the screen is determined accordingly, and thereafter, the coordinate detection may be performed according to the determined aspect ratio.

Generally, the aspect ratio of the display screen
It is desirable that the ratio of the numbers of dots in the vertical and horizontal directions of the display image be the same, but the aspect ratio of the display screen can be adjusted freely with the adjustment knob provided on the display device, so there is a possibility that they do not exactly match. high. That is, inside the information processing apparatus, for example, when processing is performed in the dot position or the coordinate space corresponding thereto, it is necessary to correct the center position of the detected captured image.

For this purpose, for example, the distance in the coordinate system of the upper right corner marker and the lower right corner marker (absolute coordinate value used by the device) and the distance in the coordinate system of the upper right corner and the upper left corner (similar) are input, It may be better to set it as a correction parameter.

As described above, according to the second embodiment, since the fine adjustment processing can be performed before the coordinate input section 9 is operated, even if the size or shape of the screen area is slightly different, it is correct. You will be able to enter coordinates.

<Description of Third Embodiment> In the first and second embodiments described above, the information processing apparatus itself has been described as also performing coordinate detection processing. However, the processing portion related to coordinate detection is performed by the information processing apparatus. An example provided externally will be described as a third embodiment.

FIG. 7 shows a schematic diagram of the usage pattern in the third embodiment. In the figure, 100 is an information processing apparatus main body, 200 is a video signal from the information processing apparatus,
It is a projector that projects on a screen 500. Thirty
Reference numeral 0 is a coordinate detection device, and 400 is a coordinate input unit.

In the third embodiment, the coordinate detecting device 300 is used.
And the information processing apparatus 100 are connected via a serial interface (for example, RS-232C or the like). Since the personal computer is provided with this kind of interface, the information processing device to be connected may be a general-purpose one with the above configuration.

FIG. 8 shows a coordinate detecting device 300 according to the third embodiment.
The block block diagram of is shown.

In the figure, reference numeral 301 is C for controlling the conduction of the apparatus.
PU, 302 is an RO that stores the operation processing procedure
M and 303 are RAMs used as work areas.
Reference numeral 304 is a serial interface for bidirectional communication with the information processing apparatus, and 305 is an interface incorporating a converter for converting a video signal from the coordinate input unit 400 into a digital signal. A push-down signal of a button of the coordinate input section 400 is also supplied to this interface. Thirty
Reference numeral 6 denotes an operation switch for setting the environment of this apparatus, which is used for setting the baud rate, parity, etc. when communicating with the information processing apparatus, for example.

In the above structure, the coordinate detection processing contents of the CPU 301 are the same as those in the first embodiment. However, the detected coordinate data is output to the information processing device side via the interface 304. Further, the adjustment processing described in the second embodiment may be executed when an instruction command for adjustment is received from the information processing device. Or, send it when the interface becomes ready.

As a result, the existing device can be used as it is, and the coordinates can be input remotely.

<Description of Fourth Embodiment> In each of the first to third embodiments, a specific marker is displayed on the displayable area. That is, it is necessary for the information processing apparatus to display the marker at a predetermined position on the display image.

However, the present invention is not limited to this. For example, as shown in FIG. 9, a physical marker is set on the outside of the projection surface (or display area) (it can be detached and can be used for an existing screen or display device only by carrying the marker). This is because it can be realized by detecting

By doing so, although it is necessary to install a marker or a display device or a screen with a marker, no special processing is required on the side of the information processing device. However, when the marker is installed, the fine adjustment process is required first.

<Description of Fifth Embodiment> In the first embodiment, when the center position of the picked-up image is not within the screen area, error processing is performed even if the marker can be detected. In this case, the screen area can be specified, and if the coordinates are outside the screen area, the coordinates may be output to the information processing device. When the obtained coordinates exceed the display area, the information processing device may indicate which direction the coordinate input unit 9 is facing, as shown in FIG.

<Explanation of Sixth Embodiment> In the first embodiment, the marker is displayed under the control of the information processing apparatus side. However, this portion is displayed without intervention of the CPU of the information processing apparatus. Is also good.

FIG. 11 shows the main block configuration of the sixth embodiment. The synthesizing circuit shown in the figure inputs a video signal to be displayed, synthesizes a marker image at the upper right corner and the lower left corner (not limited to this but may be the upper right corner and the lower right corner) of the input video, and displays the image on the display device ( It is output to a CRT display device, a projector, a dedicated OHP device). Also. Further, the coordinate data and button press information from the coordinate input unit are input and output to the information processing body.

With such a configuration, the CPU on the information processing device side does not need to perform special marker display control, and the load on the CPU can be reduced.

<Description of Seventh Embodiment> In the above embodiment,
The description has been given assuming that the display screen is rectangular. However, when the operator is relatively close to the size of the display screen and the operator is not in front of the display screen but in a biased position, the image captured by the coordinate input unit 9 The screen area of is not necessarily rectangular.

Therefore, the coordinate input section 9 is separately provided with a fine adjustment start instruction button, and when the depression of the button is detected, the markers are sequentially displayed at the four corners of the screen to form a trapezoidal trapezoid as a screen area in the captured image. Other shapes may be acceptable.

Further, in the embodiment, the pointing device for interlocking the cursor on the display screen has been described. That is, the target for coordinate input is the display screen or the screen, but the target is not limited to this. However,
When giving a presentation or the like, it is desired that the operation result be reflected remotely on the display screen.

[0074] <Description of Eighth Embodiment> In addition, the coordinate input section 9, an LED is mounted, whether mosquito could be detected two markers, toward was direction whether in the screen area The LED may be used to notify the operator so as to be user-friendly.

Further, in the present embodiment, as described above, it may be applied to one device or separate devices. Further, of course, it may be realized by supplying a program from the outside. Therefore, the present invention is limited to the above-described embodiments, and various improvements or applications are possible within the scope of the idea.

[0076]

As described above, according to the present invention, it is possible to specify a desired position on a display screen remotely while utilizing existing resources with a simple structure.

[0077]

[Brief description of drawings]

FIG. 1 is a block configuration diagram of an apparatus according to an embodiment.

FIG. 2 is a diagram illustrating an operation example of a coordinate input unit according to the embodiment.

FIG. 3 is a diagram for explaining the principle of coordinate detection in the embodiment.

FIG. 4 is a diagram for explaining coordinate determination processing and coordinate detection processing in the embodiment.

FIG. 5 is a flowchart showing a processing procedure of coordinate detection in the embodiment.

FIG. 6 is a flowchart showing a fine adjustment processing procedure in the second embodiment.

FIG. 7 is a diagram showing an operation example in the third embodiment.

FIG. 8 is a block diagram of a coordinate detecting device according to a third embodiment.

FIG. 9 is a diagram showing a screen according to a fourth embodiment.

FIG. 10 is a diagram showing a display example of a screen in the fifth embodiment.

FIG. 11 is a diagram showing a configuration of a sixth exemplary embodiment.

[Explanation of symbols]

1 CPU 2 ROM 3 RAM 4 External storage device 5 keyboard 6 VRAM 7 Display 8 interfaces 9 Coordinate input section

Continuation of the front page (56) Reference JP-A-7-121293 (JP, A) JP-A-60-138628 (JP, A) JP-A-6-308879 (JP, A) JP-B 4-72250 (JP , B2) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06F 3/033-3/037

Claims (6)

(57) [Claims] 1. A coordinate detection device for detecting a coordinate position remotely instructed, comprising: a pointing device having image pickup means , the image pickup direction of which can be freely changed ; and a pointing device supplied from the pointing device. a detecting means for detecting at least two markers to identify the in the image, a coordinate input space is, based on the detected position of the marker, the coordinate input in pairs to the space coordinates the pointing device is designated <br /> Derivation means for deriving the position, and the coordinate position designated by the pointing device,
Determining means for determining whether or not it is within the display screen area, and the determining means for operating the pointing device with the finger.
It is determined that the specified coordinate position is outside the display screen area.
If you do, specify it to the operator with the pointing device
Know that the coordinate position to be displayed is outside the display screen area.
A coordinate detection device comprising: a means for tightening . 2. The informing means is the derivation means.
It is determined that the derived coordinate position is outside the display screen area.
When the display screen area is
The display according to claim 1, wherein the direction is displayed.
On-board coordinate detection device. 3. The deriving means is the informing means.
Whether the coordinate position derived in step 3 is outside the display screen area
Is characterized by an LED.
The coordinate detection device according to claim 1. 4. A coordinate detection method for detecting a coordinate position designated from a remote location, the image detection means comprising an image pickup means , wherein the image pickup direction can be freely changed . , deriving a detection step of detecting at least two markers to identify the coordinate input space, based on the detected position of the marker coordinate <br/> position where the pointing device against the coordinate input space is designated a deriving step of, coordinate position specified by the pointing device,
A determination step of determining whether or not the display device is within the display screen area, and the determination step indicates the pointing device.
It is determined that the coordinate position is outside the display screen area.
The pointing device is displayed to the operator.
That the coordinate position is outside the display screen area.
A coordinate detection method comprising the steps of : 5. The derivation step according to the determination step
It is determined that the derived coordinate position is outside the display screen area.
In the case of the above, the informing step is conducted in the display screen area.
Claim to display the direction of the issued coordinate position
Item 4. The coordinate detection method according to item 4. 6. The deriving means in the informing step
Whether the coordinate position derived in step 3 is outside the display screen area
Is indicated by an LED.
4. The coordinate detection method described in 4.