JP3413489B2 - Transmission method of remote control image - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of transmitting an image for remote operation, which is required when operating a remote-controlled robot from a remote place, and more particularly to a three-dimensional image for easily providing stereoscopic information in real time. It relates to the method of transmitting information.

[0002]

2. Description of the Related Art The conventional surveillance cameras for remote operation are roughly classified into the following types. 1) Three-dimensional modeling type In this type, three-dimensional information is obtained by modeling and transmitting a three-dimensional object and performing display according to the three-dimensional visual characteristics by computer processing. It is described in a document (Shintoku, Tanie, “Interference Force Generation Algorithm for Virtual Environment”, Transactions of the Society of Instrument and Control Engineers, 29-3, 347-355, 1992). 2) It gives a sense of realism by moving or rotating the surveillance camera according to the movement of the head or hand of the worker at a remote location, as seen from the surveillance camera that displays the work contents of the remote camera. Yes, for example, it is described in a document (Tachi Nissho “artificial reality”, Nikkan Kogyo Shimbun). 3) Off-line two-dimensional image presentation type It is possible to present a three-dimensional environment by arranging a plurality of synchronized cameras and recording the video in parallel, and selecting the recording screen to provide the video when reproducing. , For example, literature (Takeo Kanade, Hideo Saito, Sundar Vedul
a, "The 3D Room: Digitizing Time-Varying 3D Events
by Synchronized Multiple Video Streams "Carnegie Me
llon Univ. Robotics Institute Technical Report, CM
U-RI-TR-98-34).

Among the above-mentioned conventional techniques, the offline two-dimensional image presentation type is the closest to the present invention.
The difference is that three-dimensional information is presented in real time to a remote operator by selecting and switching the camera at high speed according to the movement of the hand, head, or eyes.

[0004]

In the conventional method of transmitting an image for remote control, when performing remote control, it is important to work while having a feeling similar to the actual environment. The present situation is that information is not presented with a sense of reality even when using presentation technology. Most of the causes are image transfer speed shortage, transfer delay, low image resolution, etc., but it can be said that the development of image transfer hardware and network infrastructure holds the key to all of these. . Furthermore, in order to give a sense of realism, there is often a contradiction that a two-dimensional image presents a three-dimensional environment at a remote place.

The present invention has been proposed in view of the above,
It is an object of the present invention to provide a remote operation image transmission method capable of presenting three-dimensional information to a remote operator in real time.

[0006]

In order to achieve the above object, the invention according to claim 1 relates to a method of transmitting an image for remote control, and three or more units displaying images of different sides of a common subject. an imaging unit that is arranged along the way of the predetermined sequence a plurality of imaging apparatus, from the image of the plurality of imaging devices, 2 in synchronization with the video frame signal following
In a system comprising means for selecting a lower image, means for transmitting the selected image, means for displaying the transmitted image, and means for detecting a reaction of an observer of the displayed image, The procedure of detecting the reaction of the observer, the procedure of transmitting a signal corresponding to the detected reaction to the means for selecting the above-mentioned image, and the signal transmitted to the means for selecting the image are two or less images. And a procedure for selecting.

The invention according to claim 2 relates to a method for transmitting an image for remote control, and a plurality of image pickup devices for displaying images of different sides of a common subject are arranged in a predetermined arrangement. Arranged imaging means, 3 or more
From images of a plurality of imaging apparatus, and means for selecting two images positioned contiguous in synchronism with the video frame signal, this
Means for interpolating the image between the two selected images, and means for transmitting the interpolated image, means for displaying the transmitted video, means for detecting the reaction of the viewer of the displayed image A system for detecting the reaction of the observer, transmitting a signal corresponding to the detected reaction to the means for selecting the image, and transmitting the signal to the means for selecting the image. And a procedure for selecting an image using a signal or changing an interpolation method.

Further, in the invention described in claim 3, in addition to the structure of the invention described in claim 1 or 2, the means for detecting the reaction of the observer detects the movement of the head of the observer. It is characterized by doing.

Further, in the invention described in claim 4, in addition to the configuration of the invention described in claim 1 or 2, the means for detecting the reaction of the observer detects the movement of the eyeball of the observer. It is characterized by that.

Further, in the invention described in claim 5, in addition to the configuration of the invention described in claim 1 or 2, the selected image includes an image for the right eye and an image for the left eye. It is characterized by that.

Further, in the invention described in claim 6, in addition to the configuration of the invention described in claim 5, at least one of the image for the right eye and the image for the left eye has the above configuration. It is characterized in that the image selection device is provided with means for switching independently of the other image.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to FIGS. FIG. 1 is a block diagram showing signal processing on a transmission side of image information, and is a diagram showing a configuration including a processing device controlled by remote control, an imaging device for monitoring the processing state, and peripheral devices. is there. Figure 1
In FIG. 1, the work piece is processed by the processing device equipped with the control device 1. The control device 1 is connected to the network 100 via the network connection device 10.
Further, the image pickup device for monitoring the processing state is composed of an image pickup camera group having a predetermined arrangement, and some images are selected by the image selection device 21 from the plurality of camera images, and are connected to the video server. It is transmitted to the network 100 via the network connection device 11.

The image selection device 21 is controlled by the control device 2 and can also be controlled via a network. Further, the control device 2 has a configuration for processing image data when necessary.

FIG. 2 is a block diagram showing signal processing on the receiving side of image information. The processing information transmitted through the network 100 is displayed on the display after being passed through the network connection device 12 and the control device 3 for controlling communication, and then after being subjected to the second selection by the image selection device with the control device 4. It The image selection device 2 enables selection on the side of receiving an image, and the number of images that can be selected can be determined by the transmission rate that can be used in the network. This display is a normal CRT type, but it is obvious that a liquid crystal type or plasma emission type can also be used. Further, the processing information that has passed through the image selection device is sent to the stereoscopic glasses as needed. The information sent to the stereoscopic glasses does not have a specific fixed positional relationship between the left-eye image and the right-eye image, but the information input by the console included in the control device 5
It can be changed to other arrangements. More specifically, the position of the imaging camera of FIG. 1 for transmitting the right-eye image is normally located next to the imaging camera for transmitting the left-eye image, but this positional relationship is changed via the console. For example, the position of the imaging camera for the right-eye image can be set to the position next to the imaging camera for the left-eye image. With this configuration,
The stereoscopic effect of the image viewed through the stereoscopic glasses can be emphasized.

In the image selection device 22, a signal for selecting an image is normally input through the console,
The position information detecting device is used to detect the position information of the worker so that the operator can input the position information. As a method of detecting the position information of the worker, a method of detecting the line of sight of the worker, which is already well known, or a magnetic field information created around the magnetic sensor temporarily fixed to the head of the worker is used. There is a method of detecting, or a method of temporarily fixing a magnet to the head of the operator and targeting this, and using an external magnetic sensor. All of these are already well known methods.

A case where a machining operation is performed by remote control using the above-mentioned structure will be described in practice. The worker should either sit in front of the display or wear stereoscopic glasses,
At the same time, attach a line-of-sight sensor or a magnetic position sensor. Hereinafter, a case where the stereoscopic glasses and the magnetic position sensor are attached will be described. Use the input device of the console to control the processing device at a remote place while watching the image of the work piece displayed on the stereoscopic glasses, but if you want a different image from the image that looks like a sham, display Turn your face to the right or left. When the magnetic sensor detects this movement of the head position and the image desired by the selection of the image selection device 22 can be provided, the image is provided, and when the desired image cannot be provided, the image transmission request signal to the image selection device 21 is sent. To provide to workers. With this method, the image viewed by the operator can see the side surface of almost any work piece, although it is in a limited range. In addition, although only the side faces in the direction of scattering are viewed only by selecting an image, adjacent images are captured by using the control device 2 or the control device 4 and by the morphing technique which is already well known in the field of computer graphics. By generating a composite image of the camera, it becomes possible to see the side surface of an arbitrary work piece in a pseudo manner within a limited range.

Further, the cameras for picking up the right-eye image and the left-eye image of the stereoscopic glasses are located adjacent to each other, but by arranging this positional relationship to the position next to one imaging camera via the operation console, The three-dimensional effect is increased and more careful processing work can be performed.

The image selecting device 21 and the image selecting device 2
In 2, the video signal preferably includes a video routing switcher for the following reasons. That is, the information to be transmitted is, for example, image information consisting of 33 ms for one frame, and it is possible to synchronize the frames by applying external synchronization of the camera, but the timing of switching the video data is Since it does not always match the video signal switching timing, if a screen switching signal is input during the one-screen display being displayed, it is necessary to hold the video signal until the display of the screen in the middle of display is completed. Because there is.

[0019]

Since the present invention has the above-mentioned structure,
The effects described below can be achieved.

According to the first aspect of the invention, since the remote operator can easily see the side image desired, it is possible to easily obtain the three-dimensional information of the subject. became.

According to the second aspect of the invention, the side image desired by the remote operator can easily see the image between the image pickup devices where the image pickup device is not installed. Therefore, it becomes possible to easily obtain the three-dimensional information of the subject from any direction, though it is in a limited range.

Further, according to the third aspect of the invention, since the camera for remote operation can be switched by the movement of the observer's head, both hands are opened and remote operation is facilitated.

In the invention according to claim 4, since both hands are opened and the camera for remote operation can be switched while facing the display surface, remote operation is facilitated.

Further, in the invention described in claim 5, since it is possible to give respective images to both eyes,
I was able to work by seeing more realistic images.

In the invention according to claim 6, since the distance between the eyes can be changed substantially,
It became possible to select the three-dimensional effect, and it became possible to work more carefully.

[Brief description of drawings]

FIG. 1 is a block diagram showing signal processing on a transmission side of image information.

FIG. 2 is a block diagram showing signal processing on the reception side of image information.

[Explanation of symbols]

1,2,3,4,5 control device 11, 12 Network connection device 21, 22 Image selection device 100 networks

Claims (6)

(57) [Claims] 1. Displaying images of different sides of a common subject
An image pickup means in which three or more image pickup devices are arranged in accordance with a predetermined arrangement method, and two or less images in synchronization with a video frame signal from the images of the plurality of image pickup devices described above. The means for selecting, the means for transmitting the image selected by the selecting means, the means for displaying the image transmitted by the transmitting means, and the displaying means.
In a system provided with a means for detecting the reaction of an observer of a displayed image, a procedure for detecting the reaction of the observer, and this detecting procedure
And a step of transmitting a signal corresponding to the reaction detected by the image selection means to the image selection means, and a step of selecting two or less images using the signal transmitted to the image selection means. A method of transmitting an image for remote control, which is characterized in that 2. Displaying images of different sides of a common subject.
An image pickup means in which three or more image pickup devices are arranged in accordance with a predetermined arrangement method, and images of the plurality of image pickup devices are positionally synchronized with a video frame signal.
A means to select two adjacent videos and this video
Means for interpolating the image between the two images selected by means of the means for transmitting the image interpolated by means of interpolation, and means for displaying an image transmitted by means of this transmission, the In a system provided with a means for detecting a reaction of an observer of an image displayed by a display means, a procedure for detecting the reaction of the observer, and a procedure for this detection
Procedures to modify the or interpolation method for selecting an image using the procedures for transmitting a signal corresponding to the more the detected reaction means for selecting an image of the to come, the signal transmitted to means for selecting the video And a method for transmitting an image for remote control, comprising: 3. The remote control image transmitting method according to claim 1, wherein the means for detecting the reaction of the observer detects the movement of the head of the observer. Transmission method. 4. The remote control image transmission method according to claim 1, wherein the means for detecting the reaction of the observer detects the movement of the eyeball of the observer. Method. 5. The remote control image transmission method according to claim 1 or 2, wherein the selected video includes a right eye video and a left eye video. Method. 6. The method of transmitting an image for remote control according to claim 5, wherein at least one of a right-eye image and a left-eye image is different from the other image by the image selection device. Is a method for transmitting an image for remote operation, characterized by comprising means for independently switching.