JPH03501903A - Remote vehicle control - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] Remote vehicle control The present invention relates to remote vehicle control. The present invention particularly provides that the connecting link is located at the front of the vehicle. A device for controlling remotely operated vehicles that transmits information for displaying images of the ground and obstacles. Involved.

One type of remotely operated vehicle utilizes a cable connection between the vehicle and a remotely located operator. are doing. Such cables can be used, for example, to transmit video from a television camera mounted on a vehicle. It transmits broadband information for operators, including video signals. However, the cable The necessary limited length limits the range of movement of the vehicle and also limits its maneuverability, further reducing the risk of damage. Make it more susceptible to damage. Another method is to use a high-bandwidth wireless link, This also limits the range of operation and the type of country in which it is driven. In addition, one or more remote control It is not suitable for applications where vehicles are operated together. Videos that may be useful ・While data compression technology exists, it is difficult to transmit the necessary sensor feedback information. However, they do not have adequate compression ratios to enable use of low-bandwidth wireless links.

The present invention provides a method of remote vehicle control that eliminates the need for a dedicated high-bandwidth wireless link. It is designed to do so.

According to the present invention, a remotely operated vehicle control device carries a television camera. Including remotely controlled vehicles, the camera and vehicle are wirelessly linked to another base station that has a wireless receiver. The video image for transmission is connected by Minutes are split into separate datasets displaying long-term variations, two datasets are transmitted at different video frame rates and the two transmissions are combined to Remotely operated vehicle control device including a receiver device that provides a composite image on a video display is provided.

The video frame rate includes a rate of 301-+2 along with a second slower rate. .

The image information for transmission is the overall image used to generate the composite scene on the video display. Processed to select static terrain. The selected image information has only edge and corner information. Processed for use in the aforementioned slow frame rate transmissions.

The received slow frame rate information is sent to the video frame for projection onto the image plane of the video display. frame speed.

- By way of example, specific embodiments of the invention are described below with reference to the accompanying drawings; among, Figure 1 shows a typical example with a base station housing the vehicle operator's III tllll console. 1 illustrates a remote-controlled vehicle in a typical scene.

Figure 2 is Lord II! FIG. 2 is a block diagram of a control device.

FIG. 3 illustrates the different data transformations required to provide the desired image display.

As shown in Figure 1, a remotely controlled vehicle 1 is traveling along a rural road. is shown, the vehicle being under the control of an operator located at base station 2.

The operator is located at the operation console 3, which is a visual representation of the scene in front of the vehicle. I am using video display. TV camera 4 placed on one base of the vehicle is in front of vehicle 1 The image of the ground surface on the other side is transmitted to the receiver of the console 3. Wireless link 6 has two directions. Therefore, signals from the console 3 can be used to guide the vehicle. Image information In addition to the information, signals from the sensors of the vehicle 1 can also be sent to the console 3. Information such as braking effects, steering movements, etc. can also be applied to the video display.

As mentioned above, transmitting the entire video image to the base station requires a large video bandwidth. shall be. In some applications this is acceptable, but if more than one vehicle is It is desirable to reduce this band if it is necessary to operate on In the present invention , this is done by transferring different parts of the video information at different data rates. be done. Basic vehicle movement information that changes relatively quickly can be captured at normal video frame rates. Sent in degrees. The underlying 3D scene structure changes slowly and has a slow frame rate. will be transferred. This uses some newly inferred structural information to send means to take video frames of

In order to infer the structural information of the observed scene, each image is processed by an information processing stage into an “image tome”. It is broken down into "kun". These tokens are placed in the corners of a particular video double-sided frame. and edge feature points, and identification of these features is a method of ``digital data processing''. ” in jointly filed Japanese Patent Application No. 881123.

123 Negative of International Symposium on Remote Control and Ti Control, July 1988 132 pages Preset R.G.A., Charnley D., Harris C. ・Explained in the paper “Towards robot mobility using passive monocular vision” by Gee. A suitable application of ``Structure from Movement'' is used. Then the current position The IA three-dimensional position and four-dimensional evaluation of the corresponding scene features with respect to the vehicle position and vehicle attitude is performed. Next Break down the scene you see into three-dimensional tokens. Please keep this information low! wireless link to the base station.

At the base station, the processing circuitry creates a skeletal display of scene information based on the received token data. Can be synthesized. The composite scene is created at normal video frame rate (>30Hz). The movement of the viewed scene (video speed f!) is displayed at low frequency, provided that it is reprojected to It can be generated on the basis of updates (eg 2Hz). To achieve this, vehicle relocation is required. transmit dynamic parameters to the base station at video speed. This method is based on vehicle dynamics. It faithfully reproduces short-term fluctuations and reduces the overall terrain and object development in the field of view of the TV camera. Ensures that real-time video display recording speed is available at the base station.

The dynamically updated scene is monitored by the operator and sensor feedback that allows the vehicle to be steered. Used to obtain return information. The need for data compression is amplified by television cameras. Partial lack of information in that the actual gray levels recorded are not reconstructible. However, the reconstruction of 3D information is related to the nature of the terrain in front of the vehicle and the presence of obstacles. Another valuable quantitative data can be obtained. This method is cumbersome and has many limitations in the central area. It offers the advantage of eliminating the need for dedicated high-bandwidth wireless links. Therefore , the usability of remotely controlled vehicles is thereby enhanced.

The advantages of the device application are as follows: Comparison between the data rate requirements of different video transmission formats It becomes clear.

Video format Data rate (K pits/sec) Full resolution digital 60000 Full resolution compression <10:1’) 6000 low resolution compression (10:1) 1500 full resolution edge images approx. 1500 2 of 240 scene tokens H2 update adds vehicle features Lameter 30H2 update 15. Approximately 1 main unit required to carry out this process The device-based W device is illustrated in FIG. This figure shows that the cable is connected to the front video ・TV camera 4 (non-input) that passes analog video data to processor 7 The data is digitized into cells using a tarlace scan line method and this data is stored in frame memory. Keep it in the section. The front processor converts each digitized frame into image tokens. It is designed to be decomposed into an extraction list and associated attributes. These tokens are Based on localized corners and/or edges. The design of processor 7 is based on the VMEbus architecture and follows the structure disclosed in the aforementioned patent application.

Data exiting the front processor 7 is processed at a greatly reduced data rate (approximately 1.8 per second). M bit). This data stream is transmitted by the auxiliary movement and attitude sensor mounted on the vehicle 1. These data are also passed to the three-dimensional geometry module 8 which receives them separately.

3D geometry module 8 aligns image tokens from frame to frame; Calculate or utilize vehicle movement parameters and use triangulation to determine corresponding scene tokens. Evaluate the three-dimensional position of These evaluations receive further data from subsequent frames. Continuously improves as you progress. The output from module 8 is an improved moving pattern. Relative 3 of the scene token currently visible to TV camera 4 along with the parameter evaluation This is a table of dimensional positions. These outputs are transmitted to the base by a transmitter 9 and a low band radio link 6. 82.

Movement data is sent every video frame (30) - 1z frame rate). , 3D data is transmitted at a low rate (e.g. >2Hz), 3D geometry module The Le 8 processor uses a parallel processing architecture. via link 6 The speed of data transmission is approximately 16 pits per second.

When the data is received by the receiver 11 of the base station 2, the three-dimensional scene token is used to Composes a three-dimensional surface display of the scene viewed. This is done by the three-dimensional surface generator 12. It will be done.

A method for obtaining a point-bounded set of scene tokens is described in the above-mentioned published paper. At first , the 3D token is placed on the current image plane using the latest vehicle movement and attitude data. Reproject. Delaunay triangulation is then implemented on the image plane via the projected token. go This triangulation reconstructs the corresponding three-dimensional surface of the unit price in the depth direction. We guarantee that.

The surfaces are then transformed into an orthogonal grid of contours (formed at the intersections of surfaces by equally spaced pairs of orthogonal vertical surfaces). Become more visible. Furthermore, potential obstacles and areas unsuitable for maneuvering can be estimated from a three-dimensional plane. discussed and emphasized where necessary.

This information is then provided as a dynamic display on the operator console 3. The operator is The vehicle can be operated using remote control and by observing the display console. Ru.

The required vehicle steering and braking commands are sent to vehicle 1.

Providing 3D information to the operator allows the nature of the terrain in front of the vehicle to be assessed and potential be able to locate and avoid obstacles.

FIG. 3 illustrates the different data transformations required to obtain the desired image display.

TV - The signal from the camera produces a series of 14 video frames at a rate of 30 per second. Occur. The frames are displayed sequentially at intervals along the time axis. leave an interval From each video frame captured, the front processor generates a set 17 of image tokens. It works like this. The set of image tokens is then passed to the 3D geometry module to create the 3D Directed to the production of the original scene token 18.

The vehicle 1 is also equipped with auxiliary sensors that provide information regarding, for example, vehicle speed and attitude. Ru. The information 19 from these auxiliary sensors changes along the time axis like l1j1. The vehicle movement vector 21 is processed to form a vehicle movement vector 21 having the following characteristics.

The information in the vehicle movement vector is transmitted over the radio link 6 at a rate of 30 frames per second. received vehicle movement vectors 22 at the base station.

One of the information from the auxiliary sensor is also passed to the 3D geometry module to generate 3D scene tones. Directed to the production of Kun-18.

These tokens 18 fluctuate at a rate of 30 frames per second, and the data generated is It is transmitted by radio link 6 to the base station.

Sampled 3D scene tokens 23 at a rate of 2 frames per second and 30 frames per second. The information received at the base station is composed of the received vehicle movement vector 22 of the frame. 2 to provide a series of projected image token frames 24. This is the latest vehicle transfer By reprojecting the 3D token onto the current image plane using motion and pose data. will be done.

Delaunay triangulation 26 is performed on the image plane with the projected tokens. this triangle The surveying ensures the reconstruction of a corresponding three-dimensional surface that is uniform in depth.

This surface is then visualized by an orthogonal grid of contours at a rate of 30 frames per second. Create a 3D surface contour and maneuverable area. This contour information 27 is displayed on the video image. Although a somewhat simplified topographical map, it provides a view from which the operator can maneuver the vehicle. I can do it.

The foregoing description of embodiments of the invention is given by way of example only and is as defined by the appended claims. Numerous modifications may be made without departing from the scope of the invention. For example, an image file The rate of change of frame data is the rate described in this example, that is, 30 frames per second. It is possible to easily change from 1 frame to 2 frames. related to vehicle position and attitude. Instead of using auxiliary sensors on the vehicle that provide data on It is also possible to obtain this information in other ways, such as through the use of It is Noh.

The present invention is not necessarily limited to the control of road vehicles; It can also be used for other purposes such as transmitting sensor return information for remote landing of unmanned aircraft or unmanned aircraft. Available for use.

5UBSTjTUTE8H! i, Er International search report. rT/n. 9°/nl, lGlmm*1m+lA+, i+, + wm, ePCT/CB89101018 international search report ? Cτ/CBe910:01! SA　3)075

Claims (6)

[Claims] 1. In a remote-controlled vehicle control device, a remote-controlled vehicle carrying a television camera. The camera and vehicle are connected by a wireless link to another base station that has a wireless receiver. The video image for transmission has some parts representing short-term fluctuations and other parts representing long-term fluctuations. The two datasets are divided into separate datasets that display different videos. the video display of the receiver by combining the two transmissions. A remotely operated vehicle control device including a receiver device providing a composite image above. 2. 2. The control device of claim 1, wherein the video frame rate is a second Control unit including a 30Hz speed along with a slower one. 3. The control device according to claim 1 or 2, having an entirely stationary part. The image portion is a control device that plays the role of generating a composite scene on the video display section. 4. 4. The control device according to claim 3, wherein only edge and corner information is transmitted to the slow frame. control device for processing selected image information for use in system speed transmission; 5. The control device according to claims 1 to 4, wherein the received slow frame rate The information is converted to video frame rate for projection onto the image plane of the video display. control device. 6. In a remote controlled vehicle control device, a vehicle substantially as described above with reference to the accompanying drawings. Vehicle control device.