JP3121418B2 - Pointing system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the use of demonstrative pronouns used during a conversation when communicating information or making a decision at a remote place, such as in plant monitoring or teleconference. The present invention relates to a remote pointing device capable of smoothly giving an instruction of an object.

[0002]

2. Description of the Related Art Conventionally, IS used for remote conferences and the like
In a conference system using a DN, information is shared electronically by conference participants. In this method, documents and photos to be shared are digitized by a word processor or a scanner in advance and stored in a terminal, and are communicated with each other by indicating them from both sides with an electronic pointing device. .

[0003] However, in such a form, it is necessary to take the information into the terminal. In addition, an instruction such as "same color" could not be given by pointing to an object on the desk that cannot be taken in advance as information.

Further, in order to share materials newly obtained on the spot with each other, information can be shared by projecting the materials using a document television camera. However, in this case, there is no way for the person who does not have the material to point to a part of the material, and he had to explain it in words.

On the other hand, in a plant system,
As disclosed in Japanese Unexamined Patent Publication No. Hei 3-1297, a television camera installed at the site of a plant allows a supervisor in the central operation room to monitor the site with images. In such plant monitoring, there is a case where a supervisor is in a central operation room and a maintenance worker working at a plant site is contacted by a transceiver or the like to proceed with the work. At this time, when specifying a part or location of the equipment from the observer, communication can be achieved unless the name of the equipment is unified beforehand and how to call the location and direction is not determined. Did not.

[0006] Furthermore, although there is a monitor for monitoring in the central operation room, the scene that the observer is looking at is not visible to the maintenance personnel at the site, so that the observer directly points to the monitor and designates the device. I can't use an intuitive method of specifying.

[0007]

As described above, conventionally,
In order to instruct information between distant places, it has been indirectly shown or instructed using means such as video and audio. However, there is information that cannot be pointed out by these methods, it is not useful in the case of emergency because preparation is necessary, and special equipment is required both in remote places, However, there is a problem that it cannot be applied when the robot moves around.

In the present invention, in order to solve such a problem,
In situations where video information is exchanged using a TV camera, such as in plant monitoring or teleconferences, to more directly convey information about where the user is watching the TV camera to a remote party far away A remote pointing device is provided.

[0009]

SUMMARY OF THE INVENTION A pointy according to the present invention .
Pointing device for pointing the object
Wherein imaging means for capturing an image, and the imaging means
Image display means for displaying a captured image, and the image display
Receiving an input indicating a part of the image displayed by the means.
Input receiving means and a beam device for emitting a light beam
Based on the input received by the input receiving means.
The object displayed in a part of the designated image
The light emitted from the beam device so that the light
Control means for controlling the direction of the beam.
You.

[0010] Preferably, the imaging means is a television camera.
And the optical axis of the lens of the TV camera and the beeper.
The light beam emitted from the
Is defined.

Further, the information processing apparatus according to the present invention can
Imaging device for capturing images and image display device for displaying images
And an input receiving device that receives input, and emits a light beam
Information processing device connected to the beam device
Receiving information on an image captured by the imaging device.
And displaying the image on the image display device.
An input for designating a part of the image displayed by the display device,
Received via the input receiving device,
Based on a part of the designated image
The beam device emits light so that light hits the object
Generates and outputs information for controlling the direction of light
It is.

Further, the remote conference system according to the present invention includes two sets of the pointing system, and a point A and a
In a teleconferencing system for holding a meeting with point B
At point A, the image of the first pointing system
Means and beam device, and second pointing system
System display means and input receiving means are arranged, and at point B
And the imaging means of the second pointing system
Device and the first pointing system
And input receiving means are arranged.

Further, in the remote conference system according to the present invention, two sets of the remote pointing devices are used, and a remote instruction can be made from both sides of the remote conference.

[0014]

The operation of the representative configuration of the present invention will be described below.

There are a plurality of objects to be pointed at the point B, and an image of the point B is displayed on the screen of the display means at the point A by the image transfer means by an image pickup means such as a television camera. When a person at point A intends to designate an object at point B, the instruction means indicates the object on the screen. Based on the instruction information, control information generating means generates control information for controlling a change inducing means such as a beam device, and the control information transferring means transfers the control information to the point B. At the same time, the angle of the television camera is controlled so that the designated target is located at the center of the image. If the beam device is fixed to a television camera, only the television camera is controlled.

With this configuration, the spot of the light beam hits the object pointed by the person at the point A, and the person at the point B can immediately and reliably determine what the person at the point A is pointing. Can be recognized. It is not necessary to prepare in advance that an object to be instructed is digitized in advance and taken into the terminal, and it is possible to easily cope with a case where the other party moves around.

[0017]

FIG. 1 shows an embodiment in which the present invention is applied to plant monitoring. In the figure, 11 is a monitoring display, 12 is a touch panel, 13 is a monitoring camera, 14 is a monitoring camera remote control device, 15 is a remote control network, 16 is a video transmission network, 17 is a workstation, 18 is a plant device, 19 is a supervisor, 20 is a maintenance person, 21 is a transceiver, and 22 is a red pointing light device.

The surveillance camera remote control device 14 comprises a microcomputer, a stepping motor and a position sensor. The microcomputer can move the surveillance camera 13 at a specified angle using a stepping motor in response to a command from the workstation 17 received via the remote control network 15. Further, the current zoom ratio and angle of the surveillance camera 13 are checked by using a sensor, and the current zoom ratio and angle are checked.
Has a function of notifying the work station 17 via. The remote control network 15 is an interface between the microcomputer in the surveillance camera remote control device 14 and the workstation.
/ O bus. The video transmission network 16 carries the video from the monitoring camera 13 to the workstation 17 using multiplex transmission.

Now, let us consider a case where a supervisor 19 monitoring a plant in the central control room is trying to point a plant apparatus 18 to a maintenance person 20 at the site of the plant. The monitor 19 uses the transceiver 21 to instruct the maintenance staff 20 to “check this plant device”. At the same time, the monitor 19 indicates the position of the plant device 18 on the monitor display 11.

The touch panel 12 senses the position (x, y) indicated on the screen by the monitor 19 and notifies the work station 17 of the position. Here, (x, y) is x, y orthogonal coordinates when the origin is in the optical axis direction, that is, the center of the screen.

The workstation 17 inquires the current zoom ratio Z of the monitoring camera 13 via the remote control network 15. Then, the position (x, y) on the screen indicated by the monitor 19 notified from the touch panel 12
The angle at which the surveillance camera 13 is moved is calculated as follows.

FIG. 2 shows the relationship between the screen of the monitor display 11 and the frame actually viewed. When the shift angle of the point (x, y) on the screen with respect to the optical axis when the zoom ratio Z = 1 is (Rx, Ry) with respect to the x-axis direction and the y-axis direction, Rx = atan (x / T) ), Ry = atan (y / T). Here, T is the distance from the screen to the virtual vanishing point O, which is a camera-specific constant. When the zoom ratio changes, even if it appears at the position (x, y) on the screen, it is actually at the position (x / Z, y / Z) when the zoom ratio Z = 1, so that when the zoom ratio is considered, The deviation from the optical axis is as follows: Rx = atan (x / ZT), Ry = atan (y / ZT) (Equation 1)

Based on the calculation of Equation 1, the angles (Rx, Rx
y) The angle of the surveillance camera 13 is changed by the amount. That is, a command is sent to the surveillance camera remote control device 14 through the remote control network 15 so as to move by -Rx degree in the x direction and -Ry degree in the Y direction. Upon receiving the command, the surveillance camera remote control device 14 moves the surveillance camera 13. Thereby, the surveillance camera 13 takes an angle such that the position pointed to by the observer 19 comes to the center of the screen.

On the other hand, a red pointing light device 22 is attached to the surveillance camera 13, and the direction is set so as to coincide with the optical axis of the lens of the surveillance camera 13. For this reason, the light of the red pointing light device 22 shines on the plant device 18 which is seen at the center of the monitoring display 11.

For this reason, the monitoring camera 13 moves so that the plant device 18 pointed by the monitoring person becomes the center of the monitoring display, and the light of the red pointing light device 22 hits the object. Therefore, the maintenance person 20 at the site can specifically understand that the monitoring person 19 is looking at the plant device 18.

It should be noted that if the red pointing device 22 is fixed directly above the camera, it may not be possible to properly align it with the optical axis. At that time, a method of using a half mirror and a prism so that the optical axis of the light coincides with the optical axis of the camera can be considered.

FIG. 3 shows that the red pointing light device is not fixed to the surveillance camera
An example in which is directly controlled will be described. In this figure, the same elements as those of FIG. 1 are denoted by the same reference numerals. In this embodiment, a red pointing light device 22 and a pointing light remote control device 23 for controlling the red pointing light device 22 are arranged independently of the monitoring camera 13 and connected to a remote control network 15.

Also in the present embodiment, a case is considered where a monitor 19 monitoring the plant in the central control room attempts to point the plant equipment 18 to a maintenance person 20 at the site of the plant.

The supervisor 19 uses the transceiver 21 to instruct the maintenance staff 20 to "check this plant equipment". At the same time, the plant
Point to position 8.

The touch panel 12 senses the position (x, y) pointed on the screen by the monitor 19 and notifies the work station 17 of it. Here, (x, y) are coordinates when the origin is set in the optical axis direction, that is, the center of the screen.

The workstation 17 inquires the current zoom ratio Z of the monitoring camera 13 via the remote control network 15. Then, the position (x, y) on the screen indicated by the monitor 19 notified from the touch panel 12
Then, the angle at which the red pointing device 22 is moved is calculated using Equation (1).

Next, the surveillance camera remote control device 14 is inquired about the current angle (Bx, By). Using this, the red pointing light 22 is moved by the angle (Rx-Bx, Ry-By). That is,-
A command is sent to the pointing remote control device 23 through the remote control network 15 so as to move by Rx + Bx degrees and -Ry + By degrees in the Y direction. The pointing light remote control device 23 moves the red pointing light device 22 according to this value. As a result, the plant equipment 1
8 hits the light. Thereby, the maintenance person 20 can specifically understand the plant apparatus 18 pointed by the monitor 19.

Further, the surveillance camera 13 and the red pointing light device 22 may be moved independently (without interlocking). Using this method, the camera is usually moved so that the object pointed by the surveillance display 11 is centered, and at the same time the red pointing device 22 illuminates the object pointed to by the red pointing device 22. When an object not pointed by the restriction is pointed by the observer 19, the red pointing device 22
Only move it so that the light hits.

FIG. 4 shows an example in which a plurality of lamps are previously mounted on an object instead of a light as the object indicating means, and one of the lamps is selectively turned on to give an instruction. Reference numeral 401 denotes an instruction lamp attached to the plant device 18.

The workstation 17 has a table indicating what range of coordinates is located on the monitor screen and what devices are arranged. FIG. 5 is an example of this table. The first column of this table is the item number, the second column is the area on the screen occupied by the device, the third column is the device name, and the fourth column is the device number. In this manner, the table manages the names and numbers of the devices together with the data of the rectangular area occupying the screen. Incidentally, the data of the rectangular area is composed of the data of the upper left vertex and the lower right vertex.

This table is searched in order from the top, and it is determined whether or not the coordinates (x, y) from the touch panel 12 are included in the area of the second column. If not, go to the next step.

If the area including the coordinates is not found even after searching all the tables, nothing is done.

If the coordinates (x, y) are included,
The lamp 4 attached to the device from the device number in the fourth column
01 switch, and set it to workstation 1
7 outputs a command to turn on the switch. If the lamp 401 of another device has already been turned on, a command to turn off the switch is also output.

In order to turn on / off the switch at the workstation 17, there is a method of connecting a microcomputer to an output port of the workstation such as RS232c. "No. 1 ON" from the workstation side
A simple command such as “No. 2 OFF” is sent, and the microcomputer operates the relay circuit according to the number in response to the command to open and close the switch.

By performing such an operation, it becomes possible to indicate even a small device part that could not be instructed when the red pointing light device 22 was used.

Until now, the information of the touch panel has been used as it is to specify the indicated device. However, when the devices are complicated, the devices are inevitably distinguished. Hateful. In this case, another embodiment is provided in which the device is identified more accurately using image recognition. For the object indication, a method of fixing the red pointing light device to the surveillance camera is employed. The configuration of the device is the same as in FIG.

As in the previous examples, when the observer 19 indicates the touch panel 12 on the monitor display 11, the coordinates (x, y) on the screen are notified to the workstation 17.

The device is provided with an image recognition tag 601 as shown in FIG. The tag 601 describes, for example, a device number. Workstation 1
Numeral 7 extracts only this tag portion from the image information of the camera by image processing. This can be achieved by creating a filter that cuts out a white area of a certain size. If there are multiple tags in the image, cut out multiple tags.

From the images of the plurality of cut-out tag areas, the image closest to the coordinates (x, y) on the monitor display screen is extracted. If two or more areas correspond, select the one closest to the top left on the screen.

The number of the device written in the selected area is read by recognizing a numeral.

The workstation 17 stores a table in which the numbers of the devices and the angles of the corresponding red pointing light devices 22 are set. FIG. 7 shows an example of this table. The first column of this table indicates the device number, the second column indicates the device name, the third column indicates the x-direction angle of the red pointing light device, and the fourth column indicates the y-direction angle. The number of the recognized device is listed in the first of this table.
The values are compared in order with the columns, and when they match, the pointing device remote control device 14 is instructed via the remote control network 15 to the angle at which the monitoring camera 13 should move. The surveillance camera remote control device 14 moves the surveillance camera 13 according to this value. As a result, the surveillance camera 13 is at such an angle that the video of the tag closest to the position pointed by the observer 19 comes to the center of the screen.

At the same time, the red pointing light device 22
Lights illuminate the tag, and the maintenance person 20 can immediately understand which device is pointed out by the monitoring person 19.

FIG. 8 shows an example in which the monitor 19 uses a mouse and a menu to indicate a device, instead of using coordinate information on the screen by a touch panel. In the figure, reference numeral 801 denotes a mouse, 802 denotes a selection menu, 803 denotes a mouse pointer, and 804 denotes a keyboard. The configuration of the entire device is the same as in FIG. The menu may use icons.

The monitor operates the mouse pointer 803 by moving the mouse 801. On the monitoring display 11, the name of the device to be monitored is displayed in the selection menu 80.
2 is displayed. Pointer 803 selects the item of the device to be pointed in selection menu 802, and mouse 3
Press the 01 button to select a menu item.

The workstation 17 stores a table in which a menu item number and a corresponding angle of the red pointing light device 22 are set. FIG. 9 shows an example of this table. The first column of the table shows the menu item number, the second column shows the device name, the third column shows the x-direction angle of the red pointing light device, and the fourth column shows the y-direction angle.

The item number of the selected menu is sequentially compared with the number in the first column of this table, and if they match, the pointing device remote control device 14 is operated in accordance with the contents of the menu item number. An angle at which the surveillance camera 13 should move is commanded via the remote control network 15. In response, the surveillance camera remote control device 14 moves the surveillance camera 13. As a result, the surveillance camera 13 is at such an angle that the image of the device that the surveillant 19 wants to point to comes to the center of the screen.

At the same time, the red pointing light device 2
The two lights will illuminate the equipment, and the maintenance personnel 20 can immediately understand which equipment the monitor 19 has pointed to.

Similarly, a command input can be used instead of the coordinates on the screen to determine the object to be pointed.

In this case, the monitor inputs the name of the device to be pointed out from the keyboard 804. The workstation 17 stores a table in which the names of the devices and the angles of the corresponding red pointing light devices are set. The table shown in FIG. 7 can be used similarly.

The input device names are sequentially compared with the second column of this table. If the names match, the pointing device remote control device determines the angle at which the surveillance camera should move via the remote control network. Command. The surveillance camera remote control device moves the surveillance camera according to this value, and the angle of the surveillance camera is such that the image of the device desired by the observer is in the center of the screen. At the same time, the light of the red pointing light device 22 illuminates the equipment, and the maintenance staff 20 can understand which equipment is pointed.

Further, the same operation can be performed even if there are a plurality of such monitoring cameras 13 and a plurality of red pointing light devices 22. FIG. 10 shows the system configuration. A case where the red pointing light device 22 is fixed to the monitoring camera 13 will be described.

The monitor 19 works in cooperation with the maintenance person 20. When the maintenance staff 20 arrives at the place where the work is performed,
The monitor 19 is notified of the current position using the transceiver.

The supervisor 19 receiving the notification instructs the workstation 17 to display the image of the surveillance camera 13 at the place on the screen.

The work station 17 switches to the surveillance camera 13 where the maintenance person 20 is located and displays it on the surveillance display 11. At the same time, remote control network 1
5 is instructed to independently connect to the surveillance camera remote control device 14 attached to the surveillance camera 13 currently switched. Thereafter, the workstation 17 exchanges information with only one of the plurality of surveillance camera remote control devices 14.

The supervisor uses the transceiver to instruct the maintenance staff 20 to "check this plant equipment". At the same time, the position of the plant device is indicated on the monitoring display 11.

The touch panel senses the position (x, y) pointed on the screen by the monitor 19 and notifies the work station 17 of it.

The workstation 17 is connected to the currently selected surveillance camera 1 via the remote control network 15.
3 is queried for the current zoom factor Z. Then, the currently selected surveillance camera 13 is selected from the position (x, y) on the screen indicated by the surveillant 19 notified from the touch panel.
Calculate the angle to move. (Equation 1) Based on the calculation of Equation 1, the angle of the currently selected monitoring camera 13 is changed by the angle (Rx, Ry). -Rx degree in x direction, Y
A command is sent to the currently selected surveillance camera remote control device 14 through the remote control network 15 so as to move in the direction by -Ry degrees. The currently selected surveillance camera remote control device 14 moves the currently selected surveillance camera 13 according to this value, and the position of the currently selected surveillance camera 13 pointed by the observer 19 comes to the center of the screen. Angle.

The surveillance camera 15 has a red pointing light device 22 attached thereto.
The direction is set so as to coincide with the optical axis of the lens. Therefore, the light of the red pointing light device 22 shines on the plant device that is seen at the center on the monitoring display 11.

Therefore, the currently selected surveillance camera 13 moves so that the plant apparatus indicated by the observer becomes the center of the surveillance display, and the light of the red pointing light device 22 illuminates the object. Therefore, even when there are a plurality of surveillance cameras, the maintenance person 20 at the site can specifically understand that the monitor person 19 is looking at the plant device.

As described above, even when there are a plurality of monitoring cameras and a red pointing light device, as in the case of one monitoring person, the monitoring person can indicate the equipment to the maintenance person.

At this time, the number of monitoring cameras and the number of red pointing light devices may not match.

In such a case, the surveillance camera and the red pointing light device are separately controlled. Then, at least one red pointing light device corresponding to each monitoring camera is determined. If there is a corresponding red pointing light device when the surveillance camera is selected, an appropriate one is selected, and if not, the monitoring person is notified that pointing cannot be performed.

Further, such an apparatus can be used not only for monitoring a plant but also for a system in which a conference is remotely held. FIG. 11 shows an example in which the present invention is implemented in a remote conference. 1101 is a meeting member, 1102
Is a conference table with a built-in workstation with embedded display and touch panel.

Each of the conference rooms has a conference table 1102 with a built-in workstation for controlling the remote pointing device of the other party. A display is embedded in the upper part of the conference table 1102, and a touch panel is provided on the surface. Member 1 participating in the meeting
Reference numeral 101 indicates an image of the other party's venue shown on the conference table 1102, so that an object, a person, or the like at the other party's venue can be pointed by the red pointing light device 22.

By using such a device, information on the environment around each other can be shared and pointed, and more intimate communication becomes possible.

[0071]

According to the present invention, the means of communication at a remote place is increased, and better plant monitoring can be performed. Further, by applying the present invention to a remote conference system, it is not necessary to digitize a document when sharing information.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment in which a red pointing light device is fixed to a monitoring camera.

FIG. 2 is a conceptual diagram illustrating a frame of a surveillance camera.

FIG. 3 is a block diagram of an embodiment for controlling a red pointing light device independently of a surveillance camera;

FIG. 4 is an explanatory diagram of an embodiment in which a light emitting device is attached to a monitoring target.

FIG. 5 is an explanatory diagram of an example of a correspondence table of a relationship between an area on a monitoring display and a device;

FIG. 6 is an explanatory diagram of an example of a tag attached to a monitoring target when image processing is used.

FIG. 7 is an explanatory diagram of an example of a correspondence table between a device name and an angle of a red pointing light device;

FIG. 8 is an explanatory diagram of an embodiment for selecting a device indicated by a menu.

FIG. 9 is an explanatory diagram of an example of a correspondence table between a menu number and an angle of a red pointing light device;

FIG. 10 is an explanatory diagram of an embodiment using a number of surveillance cameras and red pointing light devices.

FIG. 11 is an explanatory diagram of an embodiment in which the present invention is applied to a remote conference system.

[Explanation of symbols]

11 monitoring display, 12 touch panel, 13
Surveillance camera, 14: Surveillance camera remote control device, 15: Remote control network, 16: Video transmission network, 17: Workstation, 18: Plant device,
19 ... Monitor, 20 ... Maintenance staff, 21 ... Transceiver, 2
2. Red pointing light device.

──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Masayasu Futagawa 4026 Kuji-cho, Hitachi City, Ibaraki Prefecture Within Hitachi, Ltd.Hitachi Research Laboratories (72) Inventor Shinya Yato 4026, Kuji-cho, Hitachi City, Ibaraki Prefecture Hitachi Research, Ltd. In-house (72) Inventor Atsuhiko Nishikawa 5-2-1 Omika-cho, Hitachi City, Ibaraki Prefecture Inside Hitachi, Ltd. Omika Plant (72) Inventor Atsuhiko Hirota 5-2-1 Omika-cho, Hitachi City, Ibaraki Hitachi, Ltd. In the Omika Factory (56) References Hikaru 1-149187 (JP, U) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06F 3/03 380 G06F 3/033 360 H04N 7/15

Claims (7)

(57) [Claims] A pointing system for pointing an object
A beam, and imaging means for capturing an image, image display means for displaying an image said imaging means has captured
If, instructs the part of the image displayed by said image display means
An input receiving unit that receives an input that a beam device for emitting a light beam, based on the input to the input receiving unit receives, before
The object displayed in a part of the indicated image
The light beam so that the light emitted from the beam device hits it.
Control means for controlling the direction of the camera.
Pointing system. 2. The pointing system according to claim 1,
A beam, wherein the imaging means is a television camera, and the optical axis of the television camera lens, the beam apparatus
The direction of the emitted light beam is set to match
Pointing system characterized by that 3. The pointing system according to claim 2,
A beam, the beam device is fixed to the television camera
A pointing device, characterized in that: 4. The pointing system according to claim 1,
A beam, wherein the imaging means is a television camera, the orientation of the beam device independently of the television camera
A pointing system characterized by being controlled. 5. The method according to claim 1, wherein
A pointing system, wherein the light beam emitted by the beam device is red.
A pointing system characterized by the following. 6. An image pickup device for picking up an image, and displaying the image.
Image display device and input receiving device for receiving input
When, information that a beam device for emitting a light beam, the Ru is connected
An information processing device, which receives information about an image captured by the imaging device,
The image is displayed on the image display device, and instructs the part of the image by the image display device has display input
Is received via the input receiving device, and based on the received input, the
In order to illuminate the object displayed in part,
Information for controlling the direction of light emitted by the beam device
An information processing apparatus characterized by generating and outputting. 7. The pointing system according to claim 1,
And a remote location for holding a meeting between point A and point B.
In a remote conference system, At point A, Imaging means and beam device of the first pointing system
And pointing means of the second pointing system
And input receiving means are arranged, At point B, Imaging means and beam of the second pointing system
Device and table of the first pointing system
Indicating means and input receiving means are arranged.
Remote conference system.