JPH08331667A - Pointing system - Google Patents

Abstract

PURPOSE: To indicate the certain location of the video displayed or projected on a screen, etc., by a pointing device, to easily detect the location on the instructed screen and to use detected coordinate data as the control information on a pointing system. CONSTITUTION: A controller 100 performs photoirradiation for the certain part of the video of a display screen part 150 or A or B, for instance, from a light emitting part 101 and images the screen including an irradiation spot by a video camera 103. The photoirradiation location is determined by a coordinate calculation part 108 from the imaged signal, the coordinate location information is transmitted from a transmission/reception part 110 to an adapter 201a, the adapter 201a receives the coordinate location information, and pointing information is imparted as the man-machine interface signal to the control part 201 of a personal computer.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pointing system, and more particularly to detection of a pointing position pointed by a pointing device or an pointing object.

[0002]

2. Description of the Related Art In recent years, a mouse, a trackball or the like has been mainly used as a pointing device on a computer display screen, and a touch panel or a light pen has been used when it is required to immediately point to a position on the display. became.

A recent document: JP-A-6-15328.
In the "remote control" of the invention proposed in Japanese Patent Publication No. 0, a marker such as a cursor is displayed on the display screen based on the angle formed by the direction of the transmission signal of the remote control and the direction of the sensitivity of the receiving unit that receives the transmission signal. Is displayed and the direction of the transmission signal of the remote controller is changed, the marker displayed on the screen is moved in a corresponding direction accordingly.

[0004]

However, in the conventional pointing system technology, when it is necessary to directly point a point on the display, the marker on the display screen is moved up and down and left and right in view of the pointing position accuracy. It could not be said that a sufficient function could be realized when it was not possible to use it in terms of pointing operability, that is, to reach the position of. Further, there is a drawback in that the display side needs a reception signal processing means having a special sensor mechanism, and there is a limitation in the display and the place to be used.

Further, there has been a demand for a device which does not require a special device on the display device side and can in particular indicate on a large screen obtained by projecting a simple screen with a video projector.

From the above, the pointing device or the pointing object is used to point at the target portion of the image displayed or projected on the screen or the screen, and the designated position on the screen or the screen is easily set. It has been demanded to provide a pointing system having a simple structure that can detect the position coordinates and use the detected position coordinates as control information for interactive operation.

[0007]

Therefore, the first invention is
What is claimed is: 1. A pointing system, comprising: a display device for receiving a light spot irradiation for position indication on a display screen; and a pointing device for giving a light irradiation instruction for a position on the screen of the display device, wherein the pointing device emits a light spot. Light irradiating means, detecting means for picking up an image of the screen of the display device, and detecting an instructed position on the screen illuminated by the light irradiating means, and information on the instructed irradiation position obtained by the detecting means The above-mentioned problem is solved by further comprising: a transmitting unit that transmits the signal in a wireless or wired manner, and a control device that uses the irradiation instruction position information provided by the transmitting unit of the pointing device as pointing control information. It is a thing.

A second aspect of the present invention is a pointing system which is instructed by a pointing object for pointing a position on a display screen or a screen of a display device, including the pointing object. By including an image pickup device that picks up the image on the display screen or the screen that includes the pointing object, and a pointing position detection device that detects the pointing position of the pointing object from the image pickup signal obtained by the imaging device, To solve the problem of.

Further, a third invention is a pointing system having a display device or screen which receives light irradiation for position indication, and a pointing device which gives light irradiation instruction to a screen or a position on the screen of the display device. The pointing device includes a light irradiating means for irradiating a light spot, and an image of a display screen or a screen illuminated by the light spot by the light irradiating means is imaged, and the light spot is radiated from the image pickup signal obtained by this imaging. The problem described above is solved by providing a detecting unit for detecting the indicated position separately from the pointing device.

[0010]

According to the configuration of the above-mentioned first invention, the pointing device irradiates a certain position on the display screen with a light spot, and the display screen including the point where the light spot is irradiated is imaged and irradiated with the light spot. The position of the existing part is detected and this irradiation position information is transmitted to the control device, and this irradiation position information is replaced with the control information for interactive operation, that is, the control information by the conventional mouse operation. It can be used as control information for selection.

According to the configuration of the above-mentioned second invention,
By pointing a certain part of the display screen or screen with a pointing object (for example, a simple pointing stick), capturing an image of the designated display screen or screen, and detecting the pointed position by the pointing object from the obtained imaging signal. The pointed position on the screen can be easily detected with a simple configuration.

Further, according to the configuration of the above-mentioned third invention,
Since a means for irradiating a simple light spot is provided as a pointing device, for example, a simple pointing device using laser light can be used, and by providing the above-mentioned detecting means separately from this pointing device, The pointing position where the pointing device emits light can be detected with a simple configuration.

With the various configurations as described above, with a simple configuration, an instruction is given with a pointing device or an instruction object while observing a target portion of an image displayed or projected on the screen or the screen, and the instructed screen is displayed. Alternatively, the position on the screen can be easily detected and the detected position coordinates can be used as control information for interactive operation.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described with reference to the drawings. First Embodiment: FIG. 1 is a functional block diagram of the pointing system of the first embodiment. In FIG. 1, the pointing system mainly includes a pointing controller 100, a display screen section 150, and a control section 600. Pointing controller 100
Is a laser pointer light emitting unit 101, an operation switch unit 102, a video camera unit 103, a video signal processing unit 104, a display screen recognition unit 105, a video frame memory 106, an image processing unit 107, and coordinate calculation. Part 108
And a transmission processing / transmission / reception unit 109 and an antenna 110.

When the lighting control signal a is input from the operation switch section 102, the laser pointer light emitting section 101 outputs a laser beam spot and irradiates it on the display screen section 150.
The operation switch unit 102 sends the blinking signal 1 to the coordinate calculation unit 10
8, the interlock signal i is fetched from the display screen recognition unit 105, the lighting control signal a is generated, and the control signals b and c are generated and given to the coordinate calculation unit 108 for control.

The display screen recognition unit 105 receives the video data h from the video signal processing unit 104, generates the interlock signal i, and supplies it to the operation switch unit 102. Upon receiving the video signal from the video camera unit 103, the video signal processing unit 104 provides the video data h to the display screen recognition unit 105, the video data w to the video frame memory 106 and the image processing unit 107, and synchronizes them. The signal j is given to the coordinate calculation unit 108.

The coordinate calculation unit 108 receives control signals b and c from the operation switch unit 102, image processing data x from the image processing unit 107, and the video signal processing unit 10.
4, the synchronization signal j is given, and the coordinate data y is given to the transmission processing / transmission / reception unit 109. Video frame memory 10
6 receives the video data w from the video signal processing unit 104, outputs video data w ′, and the image processing unit 10
Give to 7. When receiving the video data w ′ from the video frame memory 106, the image processing unit 107 outputs the image processing data x and supplies it to the coordinate calculation unit 108. The transmission processing / transmission / reception unit 109 gives the transmission signal to the antenna 110, and the reception signal captured by the antenna 110 is transmitted / processed.
It is given to the transmitting / receiving unit 109.

(Operation): Hold the pointing controller 100 in hand, orient the tip side in the direction of the display screen, and indicate the point to be pointed to by the laser pointer light emitting section 10.
Illuminate with laser spot from 1. Turning on / off of the laser beam spot is performed by operating the operation switch unit 102. The operation switch unit 102 is of a momentary type and has a two-step operation function, an interlock signal i, and a blinking signal l.
The control signals a to c are output according to the presence or absence of. That is, the control signal a is i = H (high level) and l = H, and is high level only while "lightly or strongly pushing", and the control signal b is i = high level and "while lightly or strongly pushing". And the control signal c outputs a high level only "while strongly pressed".

The video camera unit 103 outputs a video image of a region wider than the display screen unit 150 as a video signal with a relatively wide-angle lens. This video signal is provided to the video signal processing unit 104. This video signal processing unit 104
From the input video signal to the sync signal j, the video data h,
w is separated and given to the coordinate calculation unit 108, the display screen recognition unit 105, and the video frame memory 106, respectively. The video data w is given to the image processing unit 107 at the same time.

The display screen recognizing unit 105 controls the video data h
Therefore, the raster display of the display screen unit 150 is identified within the captured image area, and the interlock signal i is controlled to a high level indicating permission of laser emission only when the existence of the raster display is confirmed. The video frame memory 106 updates the data with the video data w at a specific timing. The specific timing means that the blinking signal 1 selects a frame at a low level.

In the image processing section 107, the video data w
Then, the video data w ′ stored in the video frame memory 106 is given. Temporally old video data w '
Is video data taken on the screen at the moment when the mark is turned off by the blinking laser light, and video data w that is new in time is the screen at the moment when the mark is turned on by the blinking laser light spot.

FIG. 2 shows the relationship between the blinking timing of the laser light spot and the frame timing of the video camera unit 103. In FIG. 2, when the switch of the operation switch unit 102 is lightly pressed, the laser pointer continuously emits light and functions as a pointer for visually displaying a mark at a specific location. When the switch is pushed harder with the mark of the laser beam spot being set at a specific position, the control signal c becomes high level during the period of strong push, and during the period when the control signal c is high level,
Since the generation of the laser beam spot is stopped only during the low level period of the blinking signal l, the blinking is performed as shown in "Flashing emission of laser pointer". By generating the blinking signal l in synchronization with the frame timing of the video camera unit 103, the frames before and after the change of the blinking signal l,
That is, "the video data including the mark of the laser beam which irradiates a specific portion of the screen on only one side can be obtained in the frames before and after the video frame timing which are shown in black in FIG. 2" in FIG.

The video data w and w'correspond to both frame data in which f (n) and f (n-1) are written as an example of the frame. The video data w and w ′ are transferred to the image processing unit 10.
7, the video data w ′ and the data obtained by the image processing operation including the inter-frame difference operation from the image processing unit 107, that is, the video data in which the mark by the laser light spot is extracted are supplied to the coordinate calculation unit 108. is there. The synchronization signal j separated from the video signal is further supplied to the coordinate calculation unit 108, and the coordinates of the raster area of the display imaged inside the video data w'at the timing of the video data and the laser light spot are used. The coordinates of the mark are calculated, and further, at which position in the display the mark due to the laser light spot exists on the basis of the raster region of the imaged display as a reference.

An example of this calculation will be described in detail with reference to the explanatory diagram of FIG. In this example, the video signal system is NT
This is a case where a standard video camera such as an SC system is used. In FIG. 3, an example of the waveform of the luminance level of the video data of the scanning line having two frames continuous to Dw ′ and Dw is shown. The horizontal axis t represents the time axis of the horizontal scanning period. The vertical axis represents the video data level, but Th1,
Dw, Dw 'and Th2, Dw-Dw' are for explanation of the timing relationship on the time axis, and are expressed by ignoring the relative relationship of the levels between them.

Start timing (left end) Htfs and end timing (right end) Ht of horizontal effective pixel of video data
Since fe has a fixed timing relationship with the synchronization signal, it is a reference timing. In order to identify a display screen area having a shape close to a rectangle existing in the entire captured video area, a certain threshold value Th1 is set, and the leftmost timing Htds and the rightmost timing Htde at a level exceeding this threshold value are detected.

On the other hand, for the waveform of Dw-Dw ', the timings Htms to Ht identified by a certain threshold Th2.
me is the photographing position of the mark by the laser light. Since these timings are obtained as digital values from the horizontal counter output of the video timing generation circuit in the video signal processing unit 104, for example, the horizontal coordinates of the mark position on the display screen can be accurately calculated. .

The same applies to the detection of the vertical coordinate, D
The w'detects the uppermost line number and the lowermost line number that exceed the threshold Th1, and Dw-Dw 'detects the line number that estimates the existence of the mark position from the line number that exceeds the threshold Th2. Similarly, since the line number is obtained as a digital value from the output of the vertical counter of the video timing generation circuit in the video signal processing unit 104, the line number can be accurately calculated at the vertical coordinate of the mark position on the display screen. .

In the above description of FIG. 2, the interlock signal i is described as being on the high level period, but it operates so that the laser emission is prohibited while the interlock signal i is on the low level. In the display screen recognition unit 105, the interlock signal i has a shape close to a rectangle existing in the entire captured video area from the leftmost timing Htds, the rightmost timing Htde, the uppermost line number and the lowermost line number. Only when it is confirmed that the display screen area exists in the substantially central portion of a certain defined range, the display screen area is controlled to the high level.

The thus obtained "coordinates of the mark position" and the information indicating that the operation switch section 102 is "strongly pressed" depending on the intention of selection are transmitted / received by the transmission / reception section 10.
9 and the transmission / reception signal of the transmission processing / transmission / reception unit 109 is wirelessly transmitted from the antenna 110. The receiving function is not always necessary, but a bidirectional communication function is required depending on the protocol of coordinate data transmission.

The transmitted signal is received by the adapter 201a of the control unit 600, and the control information including the decoded pointing instruction coordinates is sent to the control unit 201 of the personal computer instead of the control signal from the conventional keyboard or mouse. Is provided as a man-machine interface signal, and pointing control can be performed from a position away from the display screen unit 150.

(Effect of the first embodiment): According to the first embodiment described above, since the mark is displayed on the display with the laser emission spot to give an instruction, the pointing point can be set in the same manner as the conventional pointer. The operability that can be visually viewed and immediately indicates the point to be instructed with an intuitive sensitivity is preferable because it does not require training in operation techniques and can be freely used by an unspecified person.

Further, since the operation of selecting a point can be performed by pushing the operation switch with one hand, and the spot irradiation position or the cursor position automatically coincides with the point mark position, the cursor position can be adjusted with the conventional mouse. It is possible to obtain excellent effects such as ease of use compared to the operation of clicking a button at the position you want to move while moving, and especially to realize very preferable operability even in applications where you give a presentation while looking at the display screen. You can

Further, the pointing controller 10
0 has a function of detecting that the front end faces the direction of the display screen unit 150, and the interlock system that allows the laser emission spot only when the front end faces the direction of the display screen unit 150 functions. It does not emit light even when it is operated directly toward the human eye, and human-friendly operability and safety can be realized.

Furthermore, in the calculation of the pointing coordinates, the horizontal coordinates are detected line by line in the video data, and the calculation is performed based on the relative relationship with the display area in the video data. Even if the display screen section 150 has a distorted shape with different magnifications, it is possible to obtain an effect such as not adversely affecting the coordinate detection. Further, by displaying the light irradiation instruction position as a symbol on the display screen, the visibility and operability can be further improved.

From the above, the pointing controller 100 gives an instruction while looking at the target portion of the image displayed on the screen, the position on the screen that is instructed is easily detected, and the detected position coordinates are operated. It is possible to realize a pointing system having a simple structure that can be used for controlling modes.

[Second Embodiment]: FIG. 4 is a functional block diagram of the pointing system of the second embodiment. In FIG. 4, the pointing system includes a pointing controller 500, a television receiver 550, and a set top unit 560. The pointing controller 500 includes a laser pointer light emitting section 501, an operation switch section 502, and a video camera section 5.
03, a control signal multiplex / video modulator 504, and an antenna 505. The set top unit 560 includes an antenna 561, a reception demodulation unit 562, a coordinate calculation unit 563, and a control unit 564.

The light emitting control section 501 for the laser pointer receives the lighting control signal a from the operation switch section 502 and irradiates the television receiver 550 with a laser beam spot. The control signal multiplex / video modulator 504 is supplied with the video signal v from the video camera unit 503, the control signal c from the operation switch unit 502, the blinking signal 1 to the operation switch unit 502, and the transmission signal to the antenna 505. Give to.

The antenna 561 also receives a signal from the pointing controller 500 and gives a received signal to the reception demodulation section 562. The reception demodulation unit 562 demodulates the reception signal and separates it into a video signal and a control signal, and the coordinate calculation unit 5
Give to 63. The coordinate calculation unit 563 calculates the coordinate of the mark and gives it to the control unit 564. The control unit 564 controls the cursor of the television receiver 550 from the coordinates of the mark.

(Operation): As shown in FIG. 4, the pointing controller 500 is held in the hand, the front end side is directed toward the screen of the television receiver 550, and the point to be instructed for the laser pointer is the same as in the first embodiment. Light emitting unit 501
Illuminate with the laser light spot. Turning on / off of the laser beam spot is performed by operating the operation switch unit 502. Since the operation switch unit 502 is of a momentary type and has a function of two-step operation, the selection is controlled by the mere pointing and the intention by the pushing strength similarly to the first embodiment.

In the second embodiment, unlike the first embodiment, coordinates are not calculated inside the pointing controller 500, the video signal captured by the video camera unit 503 is directly modulated, and the set top unit 5 is used.
Wirelessly to 60. Since the selection signal information by the operation switch unit 502 and the video frame identification information corresponding to the blinking of the laser emission spot have a very small amount of information, they can be multiplexed by a simple method such as overlapping in the blanking period of the video.

The blinking control signal 1 is preferably synchronized with the synchronization signal from the video camera unit 503 as in the first embodiment. Furthermore, it is preferable to have a permission control function of laser emission by an interlock signal by screen detection.

The signal from the pointing controller 500 is received by the antenna 561, and the receiving / demodulating section 56.
Demodulated / separated by 2 to obtain a baseband video signal and a control signal. These signals are given to the coordinate calculation unit 563, and the coordinates of the mark by the pointer are calculated by the same processing as in the first embodiment. The calculated coordinate information and control information are sent to the control unit 56 of the set top unit 560.
4 to be a cursor control signal for the television receiver 550 and other man-machine interface signals to realize an interactive operation function.

(Effects of the Second Embodiment): According to the second embodiment described above, since the mark is displayed on the display by the laser emission pointer to give an instruction as in the case of the first embodiment described above, the conventional method is used. As with the pointer, the pointing point can be visually seen, and the excellent pointing function and the operability capable of pointing the point to be pointed with intuitive sensitivity can be realized.

In particular, when a VOD (video on demand) terminal at home is assumed, unlike a conventional environment in which a user sits in front of a display of a personal computer or a workstation and operates, it exists at a relatively distant position. Interactive control by pointing and selection control is possible as if operating the remote control toward the screen of the receiver.

The set top unit 560 is generally composed of a plurality of functional blocks and realizes a function as a CATV reception or a video-on-demand terminal. For the present pointing system, the receiving / demodulating section 562 and the coordinate detecting section 563 are provided. By adding / integrating, it is possible to realize a pointing system for interactive control in which the hardware allocation of the pointing controller 500 is reduced.
Can be small and lightweight.

Further, unlike the first embodiment, since the coordinate calculation operation may be performed in the set top unit 560, the power consumption and the limitation of the hardware scale are relaxed, so that more complicated image processing can be performed and the threshold value is increased. Th1, T
Adaptive control can be applied such that h2 is determined in correlation with the average value of the brightness level of the entire video screen or the surrounding area. The conditions regarding synchronization of laser emission flashing with video may also be relaxed. Further, by displaying the light irradiation position as a symbol on the display screen, the visibility and operability can be further improved.

From the above, the pointing controller 500 gives an instruction while looking at the intended portion of the image displayed on the screen, the position on the screen that is instructed is easily detected, and the detected position coordinates are interactive. It can be used as control information for operation.

[Third Embodiment]: In the following third and fourth embodiments, no special device or device is required on the display screen side, and it can be obtained by projecting onto a simple screen with a video projector. It is possible to digitize the information of the pointing point while pointing the large screen with a light emitting irradiation pointing pointer using a simple pointing stick or laser light, etc. The pointing system constitutes a man-machine interface such as cursor movement and selection control, which had to be performed, simply by operating a pointing rod or a light emission irradiation pointing pointer.

FIG. 5 is a schematic configuration diagram of the pointing system of the third embodiment. In FIG. 5, the pointing system is mainly composed of a pointing system main unit 120, a personal computer control unit 200, a screen 300, and a pointing rod 400. The main body unit 120 for pointing system is mainly provided with a video projector 121, a pointing detection camera 122, and the like.

FIG. 6 is a functional block diagram of the main body unit 120 for the pointing system. In FIG. 6, the pointing system main unit 120 includes a video projector 121 and a pointing detection camera 12.
2, a digitizing circuit 123, a frame memory 124 for the previous frame, a motion detecting circuit 125, a background memory circuit 126, a pointing coordinate detecting circuit 127, a coordinate variation judging circuit 128, and a timing generating circuit 129. Has been done.

In FIG. 6, the video projector 1
21 between the lens 21 and the lens of the pointing detection camera 122 is the projector unit 1.
The interlocking | linkage of the optical mechanism, such as a focus and a sum, when it is made into the integrated structure of 30 and the pointing detection part 131 is shown.

The video projector 121 is supplied with electric power PS for a light source and the like, and is supplied with the video signal VS1 from the video input connector J1. The video signal VS2 captured by the pointing detection camera 122 is supplied to the digitizing circuit 123. Digitizing circuit 123
Digitizes the input video signal VS2 and supplies the video data d to the frame memory 124 for the previous frame, the motion detection circuit 125, and the pointing coordinate detection circuit 127 to generate and output the synchronization signal s. The previous frame frame memory 124 stores the video data d, outputs the previous frame data d ′, and supplies the same to the motion detection circuit 125 and the background memory circuit 126. The motion detection circuit 125
Given the video data d and the previous frame data d ′, a motion detection output m is generated and given to the background memory circuit 126.

The pointing coordinate detection circuit 127 gives the coordinates of the tip portion of the pointing rod 400 from the video data d and the background memory output bg from the background memory circuit 126 to the coordinate variation determination circuit 128 as the detected coordinates xy, and the pointing coordinates. It is output to the output connector CN1. The coordinate variation determination circuit 128 obtains the determination outputs c1 and c2 from which the coordinate variation determination result is still, from the detected coordinates xy. The determination output c1 is provided to the background memory circuit 126, and the determination output c2 is provided to the selection control signal output connector CN2. Background memory circuit 12
6 outputs the background memory data bg from the previous frame data d ′, the motion detection output m, and the determination output c1 and supplies it to the pointing coordinate detection circuit 127.

(Operation): The video output from the control unit 200 of the personal computer is input to a general liquid crystal video projector 121 to project an image on the screen 300. If you point a certain point on the screen 300 with the tip of the indicator rod 400, the indicator rod 400
The coordinates of the pointing pointer are detected from the image data captured by the video camera 122 of the screen 300 including the.

This "pointer pointer detection operation" will be described in detail with reference to FIG. The pointing detection camera 122 is a relatively wide-angle lens and is a screen 300.
The video signal VS2
Output as The video signal VS2 is input to the digitizing circuit 123, and the sync signal s and the digital video data d are separated and output. The digital video data d is input to the frame memory 124 for the previous frame, the motion detection circuit 125, and the coordinate detection circuit 127, and the video data d ′ delayed by one frame from the frame memory 124 for the previous frame is also input to the motion detection circuit 125. To be done.

The motion detection circuit 125 compares the absolute value of the inter-frame difference (d-d ') with a certain threshold to estimate the coordinates of the pixel that has changed, and only in the area excluding the coordinates that have been determined to have changed. , The background memory circuit 1 so that the pixel data of the background memory circuit 126 is updated by the video data d ′.
Control 26. In this way, the background memory circuit 126
Always stores the background image data bg, and supplies the background memory data bg to the pointing coordinate detection circuit 127. The pointing coordinate detection circuit 127 detects an inter-frame difference (d−d) between the video data and the background memory data.
bg) The image of the pointing stick is extracted by image processing including calculation, and the coordinates of the pixel corresponding to the pointing stick are estimated.
On the condition that the image of the pointing rod is rod-shaped, coordinately discontinuous isolated points and isolated regions are removed from the detection result.

Further, a pointing coordinate detection circuit 127
Then, the tip coordinates of the pointing stick 400 are estimated from the pointing stick image and the background video detected in this way. The timing generation circuit 129 outputs the video timing signal generated in synchronization with the synchronization signal s to the frame memory 124 for the previous frame.
And to give to the pointing coordinate detection circuit 127,
The coordinate information xy corresponding to the spatial position of the video data d captured from the video timing signal can be obtained as digital data.

[Method for estimating the tip coordinates of the pointing rod 400 from the pointing rod image and the background image] thus obtained
This will be described with reference to FIGS. 7 and 8. FIG. 7 shows the relationship between the video imaged by the pointing detection camera 122 and the video data d. FIG. 8 shows pattern classification of the positional relationship between the raster area of the display image and the indicator rod 400.

First, the raster area of the display image is identified from the background video data bg. In FIG. 7, a frame indicated by 4001 is an effective photographing region of the pointing camera 122, and a frame indicated by 4002 is a raster region of an image projected by the video projector 121. 4006
Indicates a scan line with the background video data bg, and an example of the waveform of the brightness level of this scan line is shown in a region 4007. The horizontal axis t of this area 4007 represents the time axis of the horizontal scanning period, and the vertical axis represents the luminance level of the video data. Start timing of the horizontal effective pixel of the video data d (left end)
Htfs and end timing (right end) Htfe are reference timings because the timing relationship with the synchronization signal s is fixed.

In order to identify an area of the screen 300 having a shape close to a rectangle existing in the entire area of the captured video,
A certain threshold Th1 is set, and the leftmost end timing Htds and the rightmost end timing Htde of the level exceeding the threshold value are set.
To detect.

With respect to the detection in the vertical direction, the scan line corresponding to the area of the screen 300 can be detected by the same method, and the line number can be associated. Since the lens of the pointing detection camera 122 is selected or adjusted after presetting an approximate positional relationship in which position of the captured video data the raster area of the display image exists, the brightness is determined by such a method. It is possible to detect an area near a rectangle whose level is higher than its surroundings without difficulty.

On the other hand, when the relationship between the pointer image and the raster area is patterned, there are eight patterns P1 to P8. As shown in FIG. 8, the raster area of the display image is included in the coordinate set representing the pointer 400. Patterns P1 and P2 having coordinates common to the lowermost edge B, patterns P3 and P4 having coordinates common to the rightmost edge R, patterns P5 and P6 having coordinates common to the leftmost edge L, and a lowermost edge T. The tip coordinates of the pointing rod 400 are specified by identifying four types such as patterns P7 and P8 having common coordinates.

That is, when the patterns P1 and P2 are identified according to the identification results of these patterns, the uppermost point in the two-dimensional coordinate set of the bar images, the leftmost point in the patterns P3 and P4 is the pattern P5, In P6, the rightmost point is the pattern P
In 7 and P8, the coordinates of the tip can be detected by determining that the lowest point is the tip.

Generally, it is often displayed at a position higher than that of the person giving the presentation so that it is easy to see, and there are often four patterns P1 to P5. In this way, the coordinates of the pointing point by the pointing rod 400 are obtained as the horizontal and vertical coordinates of the video frame, and this two-dimensional coordinate data is converted into an expression of a relative positional relationship with the coordinates of the display image raster area. From the pointing coordinate output connector CN1 to the control unit 200 of the personal computer as pointing coordinate data xy.

The pointing coordinate data xy is also given to the coordinate variation judging circuit 128, where the type of temporal variation of the pointing coordinates is recognized. As an example of the type of temporal fluctuation, it is assumed that "a fixed period of time", "move in small steps", "a specific gesture" such as drawing a V shape or O, etc. It has a function to register a specific movement of the tip.

Next, "temporal change detection of pointing coordinates" will be described. As soon as the stillness of the tip of the pointer 400 is detected, the background memory circuit 126 is controlled to prohibit the pointer memory image area from being updated to the background memory circuit 126. It is intended to prevent it from being stored in the circuit 126.

Further, when the tip of the pointing rod 400 is made to stand still at a specific point for a certain period of time, the information "stationary point rest" is output from the selection control signal output connector CN2 to the control unit 200 of the personal computer to control. Instead of the conventional control signal from the keyboard or mouse, the unit 200 is equipped with software that similarly realizes a control unit function of a control signal corresponding to a mouse button click. It is possible to realize interactive operations such as selection control.

Further, by registering a specific movement of the tip of the pointing rod 400, it becomes possible to perform a selection operation according to the user's preference. For example, a plurality of buttons can be operated corresponding to the button operation of a familiar mouse. It is possible to selectively use and double-click by simply moving the tip of the pointer.

(Effect of the third embodiment): According to the third embodiment described above, the pointing is performed by a simple method in which the point on the image projected and projected on the screen is simply pointed by the pointing bar 400. Since information is digitized and can be used as an interface to a computer, the pointing point looks like a conventional pointing stick, and the operability of immediately pointing at the point you want to point is intuitive. It is preferable because it can be used freely by unspecified persons without the need to train operating techniques.

Further, it is possible to realize an excellent pointing system which can be operated naturally and with one hand such that the operation of selecting a point is stopped at a specific point of selecting the tip of the pointing rod. Particularly in an application in which a presentation is performed while standing and looking at a large display screen, a very preferable operation can be performed.

In the calculation of pointing coordinates,
By providing the background memory circuit 126 dedicated to the background image, not only by the image processing between the frames, the extraction of the pointer image is ensured, and the updating of the background memory circuit 126 when the pointer is stationary is prohibited. By
Since the background memory circuit 126 always stores an image that does not include the pointer 400, the reliability of extraction of the pointer 400 can be improved.

Furthermore, in the calculation of pointing coordinates,
Since the horizontal coordinate detection is performed for each line of the video data and is calculated based on the relative relationship with the raster area of the display image in the video data, the magnification rate of the upper portion and the lower portion of the projected image by the video projector 121 is different and distorted. Even under the elliptical display condition, the coordinate calculation is not adversely affected.

Since the function of recognizing and registering the variation pattern of the tip coordinates of the indicator rod 440 is provided, it is possible to realize an operation method and a plurality of control functions according to the user's preference. Project unit 130 and pointing detection unit 1
In the case where 31 is an integrated structure, the operability can be improved by linking the focus and zoom adjusting operation of the project unit 130 and the focus and zoom adjusting operation of the pointing detection camera 122.

From the above, the pointing bar 40 can be seen while looking at the target portion of the image projected on the screen 300.
0 (pointing device) can be used to easily detect the designated position on the screen, and the detected position coordinates can be used as control information for interactive operation.

Regarding the pointer extraction in the above description,
Although the example of extracting the image of the indicator rod itself has been explained, when the image is projected from the front side of the display screen by the projector, the shadow of the indicator rod occurs on the screen, so the image of the shadow of the indicator rod moving on the screen. May be extracted. It should be noted that when extracting the image of the indicator rod itself, it is possible to easily remove the adverse effect of the shadow image by distinguishing it by using the property that the shadow image is darker than the brightness level of the peripheral image. is there.

[Fourth Embodiment]: In the fourth embodiment, illumination-type pointing devices 401 and 402 using a laser beam or the like are used for pointing. In addition, 302
The feature is that the coordinate detection is not adversely affected even if the moving image window as shown in FIG. The other structure is the same as that of the third embodiment.

FIG. 9 is a block diagram of the pointing system of the fourth embodiment. In FIG. 9, the laser light output from the pointing device 401 is the screen 3
A certain point of 00A is irradiated, and a visual display mark 403 is displayed by this irradiation. Further, by simultaneously digitizing the coordinates of the designated portion, the same instruction and selection as the mouse operation can be performed.

FIG. 10 is a functional block diagram of the pointing system main unit 120A. This functional configuration is also almost the same as that of the third embodiment, except that the coordinate memory circuit 133 is provided, and the coordinate variation determination circuit 128 stores information from the control unit 200 of the personal computer in the coordinate memory circuit 133. Coordinate memory circuit 133
The control signal c3 corresponding to the coordinate information from is supplied to the background memory circuit 126.

The pointing coordinate detection circuit 107 performs image processing including an inter-frame difference (d-bg) calculation between the video data d and the background memory video data bg.
The optical mark image by the laser light spot is extracted, and the coordinates of the optical mark image are estimated. It is possible to obtain the pointing coordinates by converting the center coordinates of the mark image into the coordinate data of the relative relationship with the raster area 301 of the display image.

As shown in FIG. 9, a moving image window 302 is displayed.
When displaying, the information corresponding to the moving image window area is output from the personal computer, the coordinate data is stored in the coordinate memory circuit 133, and the movement of the corresponding area is detected, the background memory is updated, or the optical mark image is extracted. Is prohibited or the operation parameter is changed to eliminate the bad influence on the pointing coordinate detection by the moving image window 302.

The pointing device 401 shown in FIG.
If it is possible to use a general-purpose one instead of a special one and to change the blinking or the emission color by operating the operation switch 402, the pointing coordinate detection circuit 107
It is also possible to perform the selection control by recognizing blinking of the laser light spot or by identifying the characteristic of the color component of the light mark image.

(Effect of Fourth Embodiment): According to the above fourth embodiment, since the laser emission spot is irradiated and the mark is displayed on the displayed image to indicate the instruction, it is the same as the conventional pointer. You can see the pointer visually,
It is possible to improve the functionality and operability such that the point to be instructed can be instructed intuitively, and the pointing coordinates are digitized in the same manner as in the second embodiment to enable interactive control. Can be planned.

Further, even if there is a region such as a moving image window in which a difference between frames occurs in the displayed image, the pointing detection operation corresponding to the region information is performed, so that the detection performance does not deteriorate.

From the above, pointing with the pointing device 401 while looking at the target portion of the image projected on the screen 300A, the position on the screen which is pointed to can be easily detected, and the detected position coordinates can be interactively displayed. It can be used as control information for operation.

(Other Embodiments) (1) In the above-mentioned embodiment, the mouse operation is difficult and the display screen is free, as in the case where a display with a relatively large screen is presented before. A very useful pointing system can be realized by using it in a case where precise pointing is required.

(2) Further, the above-described embodiment can realize an excellent human interface for interactive communication such as a video-on-demand terminal.

(3) Furthermore, in the above-mentioned embodiment, an example in which a luminance signal is used for video data of a video camera has been described, but by using color information, more reliable coordinate detection can be performed. Especially when performing a demonstration or presentation by displaying the screen of a personal computer or workstation on a large display or projector, the area of the display screen taken from the video signal or the area of the window is matched with the color of the window frame to be displayed. It is possible to preset the threshold value for detecting the mark, and the color of the laser light is also known, which is also advantageous for mark coordinate detection. Furthermore,
It also improves stability and reliability.

(4) Furthermore, in the first embodiment, the data including the detected coordinates and in the second embodiment the data including the video signal are wirelessly transmitted from the antenna, but they may be wired or wireless. Alternatively, a similar system can be realized by wired optical transmission.

(5) Also, an example has been described in which a two-step operation according to the strength of pressing the operation switch is used to distinguish between simple pointing and selection. However, even with a one-step operation momentary switch, By repeating continuous on / off like so-called double click,
The same function can be realized by electronically distinguishing and providing a plurality of switches.

(6) Further, in a video-on-demand system or the like, it is also preferable to integrally mount a pointing system rather than providing it separately from a conventional remote control unit.

(7) Furthermore, in the above-described first embodiment, the display output of the control unit 201 of the personal computer is connected to a large screen display device such as a video projector, and an image is displayed on the display screen unit 150. However, the personal computer that receives the pointing information from the adapter 201a may display the information on another terminal connected via the network.

Alternatively, the display screen section 1 of the first embodiment described above
The display device 50 may include the functions of the adapter 201a and the control unit 201 of the personal computer described above.

(8) Further, the pointing system main body units 120 and 1 of the third and fourth embodiments described above.
Although 20A has been described as an example of a unit having a structure integrated with a video projector, it is also possible to configure the video project unit and the pointing detection unit as separate units, or by adding the pointing detection unit as an adapter to a conventional projector. A similar system can be realized.

(9) Furthermore, in each of the above-mentioned third and fourth embodiments, the display is explained by using the image projected by the video projector as an example. However, no matter what type of display, the area including the display is included. A similar system can be realized by installing the pointing detection unit so that the can be photographed.

(10) Furthermore, although the effect in the form of giving a presentation in front of a large screen has been described, in the case of providing a display screen at a position out of reach like a home theater, especially video-on-demand. It is also very effective in applications in which it is preferable to perform interactive operations by pointing on the screen like a terminal.

(11) In addition, in the above-described third and fourth embodiments, an example in which a luminance signal is mainly used for video data of a pointing detection camera has been described, but color information (RGB component or luminance component By using a combination of color components, etc.), more reliable coordinate detection becomes possible.

In particular, when a plurality of optical mark images are extracted in the case of extracting the optical mark image by the laser light, the characteristic of the color component of the optical mark image is utilized to detect the mark extraction coordinates for detection. Can be removed from.

As a selection control method, the pointing rod is made to stand still for a certain period at the pointing point. Finely move the indicator rod at the indicator point. Make a predetermined gesture while pointing the pointing point with the tip of the pointing stick (for example, draw O). Blink the illuminated pointer. Change the luminescent color of the illuminated pointer,
Although the method described above has been described, a gesture (for example, V is drawn) defined by the pointing point is performed even with an illumination type pointer, and a switch is provided on the pointing stick, although not a simple pointing stick. An LED lighting function is provided at the tip of the indicator rod. The same function can be realized by using a method of providing a sensor for detecting vibration or the like at the tip of the pointing rod.

(12) Further, in the third and fourth embodiments, the main body unit 120 projects the selection image for switching the image on the screen 300, and the pointing rod 40.
0 indicates the target portion of the selection image, and the main body unit 120 detects the target portion of the selection image indicated by the indicator rod 400, and detects the target portion of the detected selection image. Image according to the selection function of the control unit 20
By switching with 0, a more functional pointing system can be realized.

(13) Furthermore, in the above-mentioned third and fourth embodiments, for example, a storage means for storing in advance specific coordinates for designating a moving image window or the like is provided, and on the display screen or on the screen. In the area corresponding to the specific coordinates, the motion detection means detects the movement of the pointing object image (pointing rod image), the background image storage means stores the background image of the pointing object image, the display screen or the screen. Of the screen / screen detection means for detecting the presence of the object, or the detection of the pointing object image detecting means for detecting the pointing object image, the operation of changing the operation, for example, to affect the image of the video window area The pointing position can be accurately detected without being disturbed.

(14) Although the expression "irradiation of a light spot" is used in the above description, the expression "irradiation of a light beam" may be used.

[0102]

As described above, the first invention is a display device that receives a light spot irradiation for position indication on a display screen, and a pointing device that gives a light spot irradiation instruction to a certain position on the screen of the display device. A pointing system having a light emitting means for irradiating a light spot and an image of a screen of a display device,
The light irradiating means is provided with a detecting means for detecting a pointed position on the screen illuminated by the light spot, and a transmitting means for transmitting the information on the irradiation instructing position obtained by the detecting means by a wireless system or a wired system. By providing the control device that uses the irradiation instruction position information given by the transmission means of the pointing device as the control information of the pointing device, the pointing device gives an instruction while watching the image displayed on the screen, and It is possible to realize a pointing system having a simple configuration in which a position can be easily detected and the detected position coordinates can be used as control information for interactive operation.

The second invention is a pointing system which is instructed by an indicator for position indication for indicating a position on the display screen of the display device or on the screen. By providing an image pickup device for picking up an image on the display screen or the screen including the pointing object, and a pointing position detecting device for detecting the pointing position of the pointing object from an image pickup signal obtained by the image pickup device, the screen Or, a pointing system with a simple structure that can be pointed at with a pointing object while looking at the screen, easily detect the pointed screen or position on the screen, and use the detected position coordinates as control information for interactive operation. Can be realized.

Further, the third invention is a pointing system having a display device or screen which receives light irradiation for position indication, and a pointing device which gives a light spot irradiation instruction to a screen or a position on the screen of the display device. Therefore, the pointing device includes a light irradiation unit that irradiates a light spot, and images the display screen or the screen which is irradiated with the light spot by the light irradiation unit.
Since the detection means for detecting the pointing position where the light spot is irradiated from the image pickup signal obtained by this image pickup is provided separately from the pointing device, the configuration of the pointing device can be made very simple, and further, A pointing system with a simple structure that allows pointing with a pointing device while looking at the screen or screen, easily detecting the designated screen or position on the screen, and using the detected position coordinates as control information for interactive operation. Can be realized.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a pointing system according to a first embodiment of the present invention.

FIG. 2 is a timing chart of laser blinking in the first embodiment.

FIG. 3 is an explanatory diagram of mark coordinate calculation according to the first embodiment.

FIG. 4 is a configuration diagram of a pointing system according to a second embodiment.

FIG. 5 is a configuration diagram of a pointing system according to a third embodiment.

FIG. 6 is a functional configuration diagram of a pointing system main unit according to a third embodiment.

FIG. 7 is a diagram showing a relationship between video and video data according to a third embodiment.

FIG. 8 is an explanatory diagram of pattern classification of a positional relationship between a raster area and a pointer according to the third embodiment.

FIG. 9 is a configuration diagram of a pointing system according to a fourth embodiment.

FIG. 10 is a functional configuration diagram of a pointing system main unit according to a fourth embodiment.

[Explanation of symbols]

100 ... Pointing controller, 101 ... Laser pointer light emitting section, 102 ... Operation switch section, 103 ...
Video camera unit, 104 ... Video signal processing unit, 105 ...
Display screen recognition unit, 106 ... Video frame memory, 10
Reference numeral 7 ... Image processing unit, 108 ... Coordinate calculation unit, 109 ... Transmission processing / transmission / reception unit, 110 ... Antenna, 150 ... Display screen unit, 600 ... Control unit, 201a ... Adapter.

Claims (9)

[Claims] 1. A pointing system comprising a display device for receiving a light spot irradiation for position indication on a display screen, and a pointing device for giving a light spot irradiation instruction for a certain position on the screen of the display device, said pointing device Is obtained by the light irradiation means for irradiating the light spot, the detection means for imaging the screen of the display device, and detecting the indicated position on the screen where the light irradiation means irradiates the light spot. And a transmission unit for transmitting the information on the irradiation instruction position by a wireless method or a wired method, and further comprising a control device that uses the irradiation instruction position information provided by the transmission unit of the pointing device as pointing control information. Pointing system. 2. A pointing system having a display device for receiving a light spot irradiation for position indication on a display screen, and a pointing device for giving a light spot irradiation instruction for a certain position on the screen of the display device, said pointing device Is provided with a light irradiating means for irradiating a light spot, and a transmitting means for imaging the screen of the display device illuminated by the light irradiating means and transmitting the obtained imaging signal by a wireless system or a wired system. Further comprising: a detection unit that is provided by the transmission unit of the pointing device, detects a light spot irradiation instructed position indicated by the light irradiation unit from the imaging signal, and outputs irradiation instructed position information, Control using the irradiation instruction position information from the detecting means as pointing control information Pointing system comprising: a location. 3. The pointing device includes a control means for blinking the light spot irradiation of the light irradiation means, and has a synchronous relationship between the blinking timing of the light spot irradiation and the frame timing in the image capturing of the screen. The pointing system according to claim 1 or 2, wherein: 4. The pointing device further comprises: identification means for identifying the presence of a display screen included in the image from an image pickup signal obtained by picking up the screen; and the identification information does not indicate the presence of the display screen. 4. The pointing system according to claim 1, further comprising: a light emission prohibiting unit that prohibits light emission of the light irradiation unit. 5. The pointing device further comprises an operation switch for outputting at least two kinds of information according to an operation pressure difference, and controls the light irradiation means according to the kind of operation switch output information, or an operation switch therefor. 5. A structure in which information is transmitted to the control device.
The pointing system according to any one of 1. 6. A pointing system which is instructed by a pointing object for pointing a position on the display screen of the display device or on the screen, and an image on the display screen including the pointing object. Alternatively, a pointing system comprising: an image pickup device for picking up an image of the screen including the pointing object; and a pointing position detecting device for detecting a pointing position of the pointing object from an image pickup signal obtained by the image pickup device. 7. The pointing position detection device, a luminance level relationship between the image of the pointing object and an image around the pointing object or a relationship based on color information, the pointing object image, 7. The pointing system according to claim 6, wherein the pointing position is detected based on a positional relationship between the image of the pointing object and a peripheral image. 8. A storage unit for storing specific coordinates in advance, wherein the operation for detecting the indicated position is changed in an area corresponding to the specific coordinates. The pointing system described in Crab. 9. A pointing system comprising a display device or a screen for receiving a light spot irradiation for position pointing, and a pointing device for pointing a light irradiation on a screen of the display device or a position on the screen, the pointing system comprising: The device includes a light irradiation unit that irradiates a light spot, images the display screen or screen that is irradiated by the light spot by the light irradiation unit, and determines the indicated position where the light spot is irradiated from the image pickup signal obtained by this image pickup. A pointing system comprising a detecting means for detecting separately from the pointing device.