JP2009118072A - Remote control device and remote control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow successful compression processing of video data imaged by a camera attached to a mobile unit, to facilitate detection, even when there is an obstacle located within a range, at which an operator is observing, to prevent the structure from turning complex and control, as well as, to allow the mobile unit to cope with the case of conducting revolving or high-speed running, in this remote controller which controls the mobile unit, in a remote manner. <P>SOLUTION: The remote controller is provided with a compression processing part 3, which compresses and transmits the video data output by the camera 2 attached to the mobile unit 101; a video data processing part 4, which receives the video data transmitted from the compression processing part 3 to display it on a monitor 5 of an operation part 1. The compression processing part 3 decomposes a video imaged by the camera 2 into a plurality of areas of a mesh form; calculates a predetermined area according to a moving state of the mobile unit 101; compresses video data on the area by predetermined compression ratio or preferentially transmits the video data to the video data processing part, compresses video data on areas other than this area at a higher compression ratio; or reduces the frame rate. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a remote operation device and a remote operation method for remotely operating a mobile device such as a self-propelled robot, and more particularly to display of an operation screen in remote operation.

  Conventionally, a mobile device such as a self-propelled robot that can be self-propelled by remote control has been proposed. Such a self-propelled robot can be remotely operated using a remote operation device having a wireless transmission / reception function. When such a self-propelled robot is remotely controlled, an operator may operate it in a place where the self-propelled robot cannot be seen. In such a case, the self-propelled robot is provided with a camera that captures a moving direction image, the image captured by this camera is sent to the remote control device, and the operator performs a remote operation based on the transmitted image. Will do.

  As described above, when an image captured by a camera installed in a self-propelled robot is sent to the remote control device, it is necessary to send an image with a resolution, a frame rate, and a field of view that allow the operator to perform a favorable operation. For this reason, it is necessary to use a sufficiently wide band of radio waves for transmitting this video. However, in practice, there is a limit to the band of radio waves that can be used, so it is necessary to reduce the video data to be transmitted.

  As a means for transmitting necessary images by radio waves in a limited band, in Patent Document 1, a plurality of cameras are installed, and images captured by the plurality of cameras are combined into one image, The direction of the operator's line of sight in the display image is detected, and only the image captured by the camera corresponding to the direction of the operator's line of sight on the display screen is maintained at a high resolution and a high frame rate, and is captured by another camera. For video, a remote control device is described in which the resolution and frame rate are reduced.

  Patent Document 2 discloses a remote control device in which an omnidirectional camera is installed, a high-resolution video is captured in the traveling direction of the self-propelled robot, and a low-resolution video is captured in the peripheral portion. Are listed.

  Further, Patent Document 3 describes a remote control device that widens the area of a displayed video by superimposing environmental information acquired by past imaging on a video currently being captured.

As a method for reducing video data, an image compression technique for compressing video data, such as a so-called JPEG system or MPEG system, is generally used.
JP 60-217781 A Japanese Patent No. 3523783 JP 2005-208857 A

  By the way, in the remote control device described in the above-mentioned Patent Document 1, since it is necessary to individually control a plurality of cameras, it is necessary to maintain high resolution and frame rate over a wider range than the range in which the operator is gazing. There was a limit to data compression. In addition, when there is an obstacle in a range where the operator is not gazing, it is difficult to detect the obstacle, and there is a possibility that the response is delayed and an accident is caused.

  In the remote control device described in Patent Document 2, since the direction in which a high-resolution video is captured is specified by the traveling direction of the self-propelled robot, the operator views a high-resolution video in another direction. If this is desired, a mechanism for moving the camera itself with respect to the self-propelled robot is required, which complicates the structure and control.

  In the remote control device described in Patent Document 3, when the self-propelled robot turns or runs at high speed, the environmental information acquired by past imaging is insufficient, and thus a video in a desired range is displayed. There is a problem that it is difficult to obtain.

  An image compression technique such as the so-called JPEG system or MPEG system is a general-purpose data compression technique, and it is not always possible to optimally compress an image obtained by a camera attached to a self-propelled robot.

  Therefore, the present invention has been made in view of the above circumstances, and an object thereof is to be attached to a mobile device in a remote operation device and a remote operation method for remotely operating a mobile device such as a self-propelled robot. The video data captured by the camera can be compressed well, and even if there is an obstacle in the range where the operator is not gazing, the obstacle can be easily detected. It is another object of the present invention to provide a remote control device and a remote control method that can cope with a case where the mobile device performs turning and high speed travel without complicating the structure and control of the above.

  In order to solve the above-described problems and achieve the object, the present invention has any one of the following configurations.

[Configuration 1]
An operation unit that remotely controls the moving operation of the mobile device, a camera that is attached to the mobile device and outputs captured video data, a compression processing unit that compresses and sends the video data output from the camera, and this compression A video data processing unit that receives video data sent from the processing unit and displays a video based on the video data on a monitor provided in the operation unit, and the compression processing unit meshes the video imaged by the camera Are divided into a plurality of areas, and based on the moving speed and turning speed of the mobile device, a predetermined area corresponding to these moving states (high speed running, low speed running or turning) is calculated. The video data in the area is compressed at a predetermined compression rate or preferentially sent to the video data processing unit, and the video data other than the predetermined area is compressed in the video data in the predetermined area. It is characterized in lowering the high by the amount of compression for compressing or frame rate, than.

[Configuration 2]
An operation unit that remotely controls the moving operation of the mobile device, a camera that is attached to the mobile device and outputs captured video data, a compression processing unit that compresses and sends the video data output from the camera, and this compression A video data processing unit that receives video data sent from the processing unit and displays a video based on the video data on a monitor provided in the operation unit, and a line-of-sight sensor that detects the direction of the line of sight of the operator on the monitor The compression processing unit disassembles the image captured by the camera into a plurality of areas in a mesh shape, calculates the gaze area that the operator is gazing based on the detection result of the gaze sensor, and the video of the gaze area The data is compressed at a predetermined compression rate or preferentially sent to the video data processing unit, and the video data other than the gaze area is converted to the pressure of the video data in the gaze area. Compressed by the high compression rate than the rate, or is characterized in lowering the frame rate. In addition, by using a sensor that measures the distance between the monitor and the operator, it is possible to accurately detect the region that the operator is gazing at.

[Configuration 3]
An operation unit that remotely controls the moving operation of the mobile device, a camera that is attached to the mobile device and outputs captured video data, a compression processing unit that compresses and sends the video data output from the camera, and this compression A video data processing unit that receives video data sent from the processing unit and displays a video based on the video data on a monitor provided in the operation unit, and whether or not there is an obstacle attached to the mobile device and the distance to the obstacle An obstacle sensor to detect, and the compression processing unit decomposes the image captured by the camera into a plurality of regions in a mesh shape, calculates a region where the obstacle exists based on the detection result by the obstacle sensor, Depending on the relationship between the moving direction of the mobile device and the direction in which the obstacle exists, and the distance from the mobile device to the obstacle, the compression rate of the video data in the area where the obstacle exists is selected by the distance. Then, the video data of the area at a close distance is preferentially sent to the video data processing unit, or the display color of the video data of the area where the obstacle exists is selected and the area other than the area where the obstacle exists is selected. The video data is compressed at a compression rate higher than the compression rate of the video data in the area where the obstacle exists, or the frame rate is lowered.

[Configuration 4]
A remote operation method for compressing and sending video data captured by a camera attached to a mobile device, displaying a video based on the video data on a monitor, and remotely controlling the moving operation of the mobile device based on the video. The image captured by the camera is decomposed into a plurality of areas in a mesh shape, and according to the moving state (high speed running, low speed running, or turning) based on the moving speed and turning speed of the moving device A predetermined area is calculated, and the video data in the predetermined area is compressed at a predetermined compression rate, or is preferentially sent to the video data processing unit, and video data other than the predetermined area is transmitted to the predetermined area. The video data is compressed at a compression rate higher than the compression rate of the video data, or the frame rate is lowered.

[Configuration 5]
A remote operation method for compressing and sending video data captured by a camera attached to a mobile device, displaying a video based on the video data on a monitor, and remotely controlling the moving operation of the mobile device based on the video. The image captured by the camera is decomposed into a plurality of areas in a mesh shape, the gaze sensor calculates the gaze area in which the operator is gazing, and the video data in the gaze area is compressed at a predetermined compression rate, or The video data is preferentially sent to the video data processing unit, and video data other than the gaze area is compressed at a compression rate higher than the compression rate of the video data in the gaze area, or the frame rate is reduced. It is. Note that by measuring the distance between the monitor and the operator, it is possible to accurately detect the region in which the operator is gazing.

[Configuration 6]
A remote operation method for compressing and sending video data captured by a camera attached to a mobile device, displaying a video based on the video data on a monitor, and remotely controlling the moving operation of the mobile device based on the video. The image captured by the camera is decomposed into a plurality of areas in a mesh shape, the area where the obstacle exists is calculated by the obstacle sensor attached to the moving apparatus, the moving direction of the moving apparatus and the direction where the obstacle exists The compression ratio of the video data in the area where the obstacle exists is selected according to the distance according to the relationship between the mobile device and the obstacle, and the video data in the area in the near distance is preferentially processed. The display color of the video data in the area where the obstacle exists, and the video data other than the area where the obstacle exists are selected in the area where the obstacle exists. Compressed by the high compression ratio than the compression ratio of the image data, or is characterized in lowering the frame rate.

  In the remote control device having the configuration 1, the compression processing unit decomposes the image captured by the camera into a plurality of areas in a mesh shape, and based on the moving speed and the turning speed of the moving device, the moving state (during high-speed running) A predetermined area corresponding to whether the vehicle is traveling at a low speed or turning, and the video data in the predetermined area is compressed at a predetermined compression rate, or is preferentially sent to the video data processing unit, Since the video data other than the predetermined area is compressed at a compression rate higher than the compression ratio of the video data in the predetermined area, or the frame rate is reduced, the video of the predetermined area required for the remote operation of the mobile device By maintaining the data at a high resolution and frame rate and performing the minimum data compression required for remote operation, the band of radio waves used for wireless communication from the compression processor It is possible to narrow. In addition, by sending image data of a high resolution area preferentially among the images of each area divided in a mesh shape, even if the radio wave condition deteriorates, useful information is preferentially delivered to the operator. be able to.

  In the remote control device having the configuration 2, the compression processing unit decomposes the video imaged by the camera into a plurality of areas in a mesh shape, and determines the gaze area that the operator is gazing based on the detection result of the gaze sensor. Calculate and compress the video data in the gaze area at a predetermined compression rate, or send the video data to the video data processing unit with priority, and the video data other than the gaze area from the compression rate of the video data in the gaze area Since the compression is performed at a high compression rate or the frame rate is lowered, the video data of the gaze area that is required by the operator for remote operation of the mobile device is maintained at a high resolution and frame rate. By performing the necessary data compression, the band of the radio wave used for wireless communication from the compression processing unit can be narrowed. In addition, by sending image data of a high resolution area preferentially among the images of each area divided in a mesh shape, even if the radio wave condition deteriorates, useful information is preferentially delivered to the operator. be able to.

  In the remote control device having the configuration 3, the compression processing unit decomposes the image captured by the camera into a plurality of regions in a mesh shape, and calculates a region where the obstacle exists based on the detection result by the obstacle sensor. Depending on the relationship between the moving direction of the moving device and the direction in which the obstacle exists, and the distance from the moving device to the obstacle, the compression rate of the video data in the area where the obstacle exists is selected by the distance, and is close The video data of the area at the distance is preferentially sent to the video data processing unit, or the display color of the video data of the area where the obstacle exists is selected, and the video data other than the area where the obstacle exists is selected. Because the compression rate is higher than the compression rate of the video data in the area where the obstacle exists, or the frame rate is lowered, there is an obstacle necessary for remote operation of the mobile device. That the video data area is maintained at a high resolution and frame rate, by performing minimum necessary data compression to the remote operation, it is possible to narrow the band of radio waves used for wireless communication from the compression processing unit. In addition, even if the radio wave condition deteriorates by preferentially transmitting video data of high-resolution areas or obstacles at close distances among the video of each area divided into meshes, useful information Can be preferentially delivered to the operator. Further, even in a low-resolution video area with a high compression rate, an obstacle can be easily detected, and the occurrence of an accident can be suppressed.

  In the remote operation method having the configuration 4, the image captured by the camera is decomposed into a plurality of areas in a mesh shape, and based on the moving speed and the turning speed of the moving device, these moving states (high speed running or low speed running) Or a predetermined area according to whether the vehicle is turning, and the video data in the predetermined area is compressed at a predetermined compression rate, or is preferentially sent to the video data processing unit, and other than the predetermined area. Video data is compressed at a compression rate higher than the compression rate of the video data in the predetermined area, or the frame rate is reduced, so that the video data in the predetermined area required for the remote operation of the mobile device has high resolution and Maintain the frame rate and perform the minimum data compression required for remote operation, thereby narrowing the band of radio waves used for wireless communication from the compression processor It can be. In addition, by sending image data of a high resolution area preferentially among the images of each area divided in a mesh shape, even if the radio wave condition deteriorates, useful information is preferentially delivered to the operator. be able to.

  In the remote operation method having the configuration 5, the image captured by the camera is divided into a plurality of areas in a mesh shape, the gaze area that the operator is gazing at is calculated by the line-of-sight sensor, and the video data of the gaze area is obtained. The video data is compressed at a predetermined compression rate or preferentially sent to the video data processing unit, and the video data other than the gaze region is compressed at a compression rate higher than the compression rate of the video data in the gaze region, or a frame By reducing the rate, the video data of the gaze area that the operator who is required in the remote operation of the mobile device keeps the video data at a high resolution and frame rate, and by performing the minimum data compression necessary for the remote operation, The band of the radio wave used for wireless communication from the compression processing unit can be narrowed. In addition, by sending image data of a high resolution area preferentially among the images of each area divided in a mesh shape, even if the radio wave condition deteriorates, useful information is preferentially delivered to the operator. be able to.

  In the remote operation method having the configuration 6, the image captured by the camera is decomposed into a plurality of areas in a mesh shape, the area where the obstacle exists is calculated by the obstacle sensor attached to the movement apparatus, and the movement of the movement apparatus Depending on the relationship between the direction and the direction in which the obstacle exists, and the distance from the mobile device to the obstacle, the compression ratio of the video data in the area where the obstacle exists is selected by the distance, and The video data is preferentially transmitted to the video data processing unit, or the display color of the video data in the area where the obstacle exists is selected, and the video data other than the area where the obstacle exists is displayed by the obstacle. Since the image data is compressed at a compression rate higher than the compression rate of the video data in the existing area, or the frame rate is lowered, the image of the area where the obstacle necessary for remote operation of the mobile device exists Maintaining over data to the higher resolution and frame rate, by performing minimum necessary data compression to the remote operation, the band of radio waves used for wireless communication from the compression section can be narrowed. In addition, even if the radio wave condition deteriorates by preferentially transmitting video data of high-resolution areas or obstacles at close distances among the video of each area divided into meshes, useful information Can be preferentially delivered to the operator. Further, even in a low-resolution video area with a high compression rate, an obstacle can be easily detected, and the occurrence of an accident can be suppressed.

  That is, according to the present invention, in a remote operation device and a remote operation method for remotely operating a mobile device such as a self-propelled robot, it is possible to satisfactorily compress video data captured by a camera attached to the mobile device. In addition, even if there is an obstacle in the range that the operator is not paying attention to, it is easy to detect the obstacle, and without complicating the structure and control of the moving device, the moving device It is possible to provide a remote control device and a remote control method that can cope with turning and high speed running.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[Configuration of Remote Operation Device in First Embodiment]
FIG. 1 is a schematic side view showing a configuration of a remote control device according to the present invention.

  As shown in FIG. 1, the remote operation device includes an operation unit 1 that remotely controls a moving operation of a self-propelled robot 101 serving as a moving device by wireless transmission / reception.

  The self-propelled robot 101 is configured by incorporating a power source, a motor, and the like as a power source in a housing 102, and can drive the self-propelled mechanism including the wheels 103 and the like by the power source to self-run. The self-propelled robot 101 can drive the self-propelled mechanism based on a control signal wirelessly transmitted from the operation unit 1 of the remote control device, and can move forward, backward, or turn. The speed can be adjusted.

  The operation unit 1 can generate a control signal instructing forward, backward, or turn and speed adjustment according to the operation of the operator 104, and wirelessly transmit this control signal to the self-propelled robot 101. It is configured as follows.

  The remote control device includes a camera 2 that is attached to the self-propelled robot 101 and outputs captured video data. The camera 2 is, for example, a CCD camera. The video data output from the camera 2 is sent to the compression processing unit 3 provided on the self-propelled robot 101. The compression processing unit 3 compresses the video data sent from the camera 2 and wirelessly transmits it to the operation unit 1.

  The video data sent from the compression processing unit 3 is received by the video data processing unit 4 in the operation unit 1. The video data processing unit 4 displays a video on a monitor 5 provided in the operation unit 1 based on the received video data. The operator 104 can perform the remote operation method according to the present invention by performing an operation based on the image displayed on the monitor 5, and can accurately operate the self-propelled robot 101 remotely. That is, the operator 104 uses the video displayed on the monitor 5 to perform a remote operation such as moving the self-propelled robot 101 while avoiding an obstacle or moving the self-propelled robot 101 toward a desired target. It can be performed.

  In the remote control device according to this embodiment, the compression processing unit 3 decomposes an image captured by the camera 2 into a plurality of regions in a mesh shape. Then, the compression processing unit 3 is based on the movement state of the self-propelled robot 101 and the turning speed, that is, depending on whether the self-propelled robot 101 is moving at high speed, low speed, or turning. Then, a predetermined area in the captured image is calculated. For example, if the self-propelled robot 101 is traveling straight at high speed, the predetermined area is above the center, that is, far away from the front, and if the self-propelled robot 101 is traveling straight at low speed, If the self-propelled robot 101 is turning in the vicinity of the front, that is, in the vicinity of the front, it can be determined as a right part or a left part according to the turning direction. Then, the compression processing unit 3 compresses the video data in the predetermined area at a predetermined compression rate (low compression ratio), and converts the video data other than the predetermined area from the compression rate of the video data in the predetermined area. Also compress with high compression rate. That is, in this remote control device, a high-resolution or high-frame-rate video is displayed in a predetermined area, and a low-resolution or low-frame-rate video is displayed in surrounding areas other than the predetermined area. Is displayed.

  Thus, in this remote control device, the amount of video data sent from the compression processing unit 3 can be reduced, and the band of radio waves used for wireless transmission can be narrowed. Moreover, even if the radio wave condition is deteriorated by preferentially transmitting the video data of the high-resolution area among the video of each area divided into meshes, it is possible to deliver useful information to the operator 104. it can. Further, even in a low-resolution video area with a high compression rate, an obstacle can be easily detected, and the occurrence of an accident can be suppressed.

  As the compression processing of the video data in the compression processing unit 3, the Huffman compression is performed after the frequency conversion of the video data of each region and the resulting high frequency component are cut as employed in the so-called “JPEG” method. Can be used. In addition, this compression processing may be processing for applying Huffman compression to the video data itself of each area. Further, this compression processing may be processing in which Huffman compression is performed after wavelet transform is performed and the resulting high frequency component is cut.

  The compression rate is, for example, about 3% to 1% as a low compression rate, about 1% to 0.8% as a medium compression rate, and about 0.8% to 0.5% as a high compression rate. Is assumed.

  In a strict sense, it is not image compression, but by reducing the video update cycle (frame rate), reducing the video resolution to about 1/5 to 2/5, or reducing the color, The amount of data transfer can be reduced.

  As a video update cycle, a high frame rate is assumed to be about 30 fps to 10 fps, and a low frame rate is assumed to be about 10 fps to 1 fps. As the video resolution, for example, one pixel of video data captured as a high resolution and 4 × 4 pixels of video data captured as a low resolution are assumed. Further, as the color reduction, for example, it is assumed that 16 million colors are reduced to 256 colors.

[Configuration of Remote Operation Device in Second Embodiment]
As shown in FIG. 1, the remote control device according to the present invention may include a line-of-sight sensor 6 that detects the direction of the line of sight of the operator 104 on the monitor 5. As the line-of-sight sensor 6, a sensor that captures the face of the operator 104 and recognizes the face direction and the position of the pupil from the captured face image can be used.

  In this case, the compression processing unit 3 decomposes the image captured by the camera 2 into a plurality of areas in a mesh shape, and based on the detection result by the line-of-sight sensor 6, the gaze area that the operator 104 is gazing at. calculate.

  Then, the compression processing unit 3 compresses the video data in the gaze area at a predetermined compression rate (low compression ratio), and compresses the video data other than the gaze area higher than the compression rate of the video data in the gaze area. Compress by rate. That is, in this remote control device, a high resolution or high frame rate video is displayed in the gaze area, and a low resolution or low frame rate video is displayed in the surrounding area other than the gaze area. Is done.

  Thus, in this remote control device, the amount of video data sent from the compression processing unit 3 can be reduced, and the band of radio waves used for wireless transmission can be narrowed. Moreover, even if the radio wave condition is deteriorated by preferentially transmitting the video data of the high-resolution area among the video of each area divided into meshes, it is possible to deliver useful information to the operator 104. it can. Further, even in a low-resolution video area with a high compression rate, an obstacle can be easily detected, and the occurrence of an accident can be suppressed.

  Furthermore, in this remote operation device, as shown in FIG. 1, a distance sensor 7 for measuring the distance between the monitor 5 and the operator 104 may be provided. As the distance sensor 7, a laser distance meter or the like can be used. By using the distance sensor 7 as well, it is possible to more accurately detect the gaze area where the operator 104 is gazing.

[Configuration of Third Embodiment of Remote Operation Device]
As shown in FIG. 1, the remote control device according to the present invention may be provided with an obstacle sensor 8 that detects the presence or absence of an obstacle and the distance to the obstacle attached to the self-propelled robot 101. As the obstacle sensor 8, for example, a laser distance meter, a stereo camera, or the like can be used. The detection signal from the obstacle sensor 8 is sent to the signal processing unit 9. The signal processing unit 9 calculates and outputs the presence / absence of an obstacle and the distance to the obstacle based on the detection signal from the obstacle sensor 8. An output signal from the signal processing unit 9 is sent to the compression processing unit 3.

  In this case, the compression processing unit 3 decomposes the image captured by the camera 2 into a plurality of regions in a mesh shape, and calculates a region where an obstacle exists based on the detection result by the obstacle sensor 8.

  Then, the compression processing unit 3 is an area in which an obstacle exists according to the relationship between the moving direction of the self-propelled robot 101 and the direction in which the obstacle exists, and the distance from the self-propelled robot 101 to the obstacle. The compression rate of the video data is selected, or the display color of the video data in the area where the obstacle exists is selected. That is, the compression processing unit 3 displays a high-resolution image in an area where an obstacle exists, particularly in an area where an obstacle in the traveling direction of the self-propelled robot 101 exists. In an area where there is an obstacle in the vicinity of the robot 101, an image with a coloring process such as red to call attention is displayed.

  The compression processing unit 3 compresses the video data other than the area where the obstacle exists at a compression rate higher than the compression ratio of the video data in the area where the obstacle exists. That is, in this remote control device, a high resolution or high frame rate image is displayed in an area where an obstacle exists, and a low resolution or a surrounding area other than the area where an obstacle exists. A video with a low frame rate is displayed.

  Thus, in this remote control device, the amount of video data sent from the compression processing unit 3 can be reduced, and the band of radio waves used for wireless transmission can be narrowed. Moreover, even if the radio wave condition is deteriorated by preferentially transmitting the video data of the high-resolution area among the video of each area divided into meshes, it is possible to deliver useful information to the operator 104. it can. Further, even in a low-resolution video area with a high compression rate, an obstacle can be easily detected, and the occurrence of an accident can be suppressed.

[Processing algorithm in the remote control device]
FIG. 2 is a flowchart showing an algorithm of signal processing executed in the remote control device according to the present invention.

  Hereinafter, a signal processing algorithm executed by the compression processing unit 3 in this remote control device will be described. That is, when the compression processing unit 3 starts its operation, as shown in FIG. 2, the compression processing unit 3 acquires a video, that is, acquires video data from the camera 2 in step st1. Next, in step st2, the traveling speed of the self-propelled robot 101, the input value of the turning operation, and the position information of the camera 2 are acquired. Next, in step st3, a mesh for dividing the image captured by the camera 2 into a plurality of regions is created, and the process proceeds to step st4.

  FIG. 3 is a front view showing an example of an image displayed on a monitor in the remote control device according to the present invention.

  The mesh that divides the video into a plurality of regions is obtained by converting a mesh on an orthogonal coordinate system centered on the self-propelled robot 101 into a viewpoint from the camera 2 as shown in FIG. . Further, this mesh may be only in the direction of the ground without the horizontal line, or may be divided into meshes using the display screen itself as an orthogonal coordinate system.

  In step st4, the traveling locus of the self-propelled robot 101 is estimated. The estimation of the travel locus is indicated by an arrow in FIGS. 3B, 3C, and 3D based on the travel speed of the self-propelled robot 101, the input value of the turning operation, and the position information of the camera 2. As described above, the traveling locus of the self-propelled robot 101 in the image captured by the camera 2 is estimated. With respect to the traveling locus in the case of turning, the turning locus of the self-running robot 101 is estimated based on the input value of the turning operation and the turning characteristics of the self-running robot 101.

  On the other hand, if the operation unit 1 includes the line-of-sight sensor 6, viewpoint data is acquired by the line-of-sight sensor 6 in step st5.

  If the self-propelled robot 101 includes the distance sensor 7, in step st6, the distance sensor 7 acquires data indicating the direction and distance from the self-propelled robot 101 to the obstacle. In step st7, the data indicating the direction and distance to the obstacle is geometrically converted, corresponding to the position information of the camera 2 acquired in step st2, and the position of the obstacle in the image obtained by the camera 2 is calculated. Keep it.

  Next, in step st8, a predetermined area such as a gaze area is set. That is, based on the travel locus of the self-propelled robot 101 estimated in step st7, for example, a region in the direction from which the self-propelled robot 101 is headed is set as a predetermined region.

  For example, a gazing point with respect to the current traveling speed is calculated from a table showing a relationship between a preset traveling speed and a gazing point, and (b) in FIG. ) And (c), the gaze point in the display image is calculated. A range of a certain width centered on this gazing point is set as a predetermined area. (B) in FIG. 3 shows a predetermined area on the display screen during high-speed driving. Further, (c) in FIG. 3 shows a predetermined area on the display screen during low-speed traveling.

  And (d) in Drawing 3 shows a predetermined field on a display screen at the time of turning. As for the case where the self-propelled robot 101 turns, as shown in (d) of FIG. 3, from the estimated travel locus of the self-propelled robot 101 and a table indicating the relationship between the travel speed and the gazing point, A gaze point at a gaze point distance on the travel locus is calculated. A range of a certain width centered on this gazing point is set as a predetermined area.

  When the line-of-sight sensor 6 is provided in the operation unit 1, the direction in which the operator 104 is gazing on the monitor 5 is set as a predetermined area (gaze area). Furthermore, when the self-propelled robot 101 is provided with the distance sensor 7, as shown in (e) of FIG. 3, an area where an obstacle exists is set as a predetermined area. In this case, a predetermined area is set in accordance with the relationship between the moving direction of the self-propelled robot 101 and the direction in which the obstacle exists, and the distance from the self-propelled robot 101 to the obstacle. Also good. In addition, for an area where an obstacle exists, the video data may be subjected to a coloring process such as red in order to call attention.

  In the next step st9, the images by the camera 2, markings indicating obstacles, and images such as area division meshes are superimposed, and in step st10, an area for lowering the frame rate is set. In step st11, the compression rate for each region is set.

  In step st12, the video data in each area is compressed at the compression rate set in step st11. This compression process only needs to be performed for the area where transmission to the operation unit 1 is performed, and need not be performed for areas where transmission is not performed.

  In step st13, a preferential transmission area is set. In other words, by preferentially transmitting video data in an important area in remote operation, even if communication failure occurs and video data transmission is interrupted, the influence on remote operation can be minimized. it can. In step st14, the video data is transmitted to the operation unit 1.

  The operation unit 1 to which the video data has been transmitted acquires the video data in step st15, expands (expands) the video data in step st16, and displays the video on the monitor 5 in step st17.

  The operator 104 can remotely control the self-propelled robot 101 based on the video displayed on the monitor 5 in this way.

It is a typical side view showing composition of a remote control device concerning the present invention. It is a flowchart which shows the algorithm of the signal processing performed in the remote control apparatus which concerns on this invention. It is a front view which shows the example of the image | video displayed on a monitor in the remote control apparatus which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Operation part 2 Camera 3 Compression processing part 4 Image | video data processing part 5 Monitor 6 Eye sensor 7 Distance sensor 8 Obstacle sensor 101 Self-propelled robot 104 Operator

Claims (6)

An operation unit for remotely controlling the movement operation of the mobile device;
A camera that is attached to the mobile device and outputs captured video data;
A compression processing unit for compressing and sending video data output from the camera;
A video data processing unit that receives video data sent from the compression processing unit and displays a video based on the video data on a monitor provided in the operation unit;
The compression processing unit decomposes the image captured by the camera into a plurality of regions in a mesh shape, calculates a predetermined region according to the moving state based on the moving speed and the turning speed of the moving device, The video data in the predetermined area is compressed at a predetermined compression rate, or is preferentially sent to the video data processing unit, and the video data other than the predetermined area is more than the compression rate of the video data in the predetermined area. A remote control device characterized by compressing at a high compression rate or reducing the frame rate. An operation unit for remotely controlling the movement operation of the mobile device;
A camera that is attached to the mobile device and outputs captured video data;
A compression processing unit for compressing and sending video data output from the camera;
A video data processing unit that receives the video data sent from the compression processing unit and displays a video based on the video data on a monitor provided in the operation unit;
A line-of-sight sensor that detects the direction of the line of sight of the operator on the monitor,
The compression processing unit decomposes an image captured by the camera into a plurality of areas in a mesh shape, calculates a gaze area in which the operator is gazing based on a detection result by the gaze sensor, The video data is compressed at a predetermined compression rate or preferentially sent to the video data processing unit, and the video data other than the gaze area is compressed at a compression rate higher than the compression rate of the video data in the gaze area. A remote control device characterized by lowering the frame rate. An operation unit for remotely controlling the movement operation of the mobile device;
A camera that is attached to the mobile device and outputs captured video data;
A compression processing unit for compressing and sending video data output from the camera;
A video data processing unit that receives the video data sent from the compression processing unit and displays a video based on the video data on a monitor provided in the operation unit;
An obstacle sensor attached to the moving device for detecting the presence or absence of an obstacle and the distance to the obstacle;
The compression processing unit decomposes an image captured by the camera into a plurality of regions in a mesh shape, calculates a region where an obstacle exists based on a detection result by the obstacle sensor, and moves the moving device According to the relationship between the obstacle and the direction in which the obstacle exists, and the distance from the mobile device to the obstacle, the compression ratio of the video data in the area where the obstacle is present is selected according to the distance, The video data of a certain area is preferentially sent to the video data processing unit, or the display color of the video data of the area where the obstacle exists is selected, and the video data other than the area where the obstacle exists is A remote control device that compresses at a compression rate higher than the compression rate of video data in an area where an obstacle exists or lowers the frame rate. A remote operation method of compressing and transmitting video data captured by a camera attached to a mobile device, displaying a video based on the video data on a monitor, and remotely operating the moving operation of the mobile device based on the video Because
The video imaged by the camera is disassembled into a plurality of areas in a mesh shape, a predetermined area corresponding to the moving state is calculated based on the moving speed and the turning speed of the moving device, and video data of the predetermined area Is compressed at a predetermined compression rate, or is preferentially sent to the video data processing unit, and video data other than the predetermined region is compressed at a compression rate higher than the compression rate of the video data in the predetermined region. Or a remote operation method characterized by lowering the frame rate. A remote operation method of compressing and transmitting video data captured by a camera attached to a mobile device, displaying a video based on the video data on a monitor, and remotely operating the moving operation of the mobile device based on the video Because
The image captured by the camera is decomposed into a plurality of areas in a mesh shape, the gaze sensor calculates the gaze area that the operator is gazing at, and the video data of the gaze area is compressed at a predetermined compression rate, or Remotely transmitting to the video data processing unit with priority and compressing video data other than the gaze area at a compression rate higher than the compression rate of the video data in the gaze area or reducing the frame rate Method of operation. A remote operation method of compressing and transmitting video data captured by a camera attached to a mobile device, displaying a video based on the video data on a monitor, and remotely operating the moving operation of the mobile device based on the video Because
The image captured by the camera is decomposed into a plurality of areas in a mesh shape, the area where the obstacle exists is calculated by the obstacle sensor attached to the moving device, the moving direction of the moving device and the presence of the obstacle The compression ratio of the video data in the area where the obstacle is present is selected according to the distance according to the relationship with the direction to be performed and the distance from the mobile device to the obstacle, and the video data of the area in the near distance is selected. An area in which the obstacle exists, and a video data display unit that preferentially sends to the video data processing unit or selects a display color of the video data in the area in which the obstacle exists, A remote operation method characterized by compressing at a compression rate higher than the compression rate of video data or reducing the frame rate.

Images