JP2014071777A - Vehicle, and remote control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle and a remote control device, which are capable of providing an operator with a feeling of speed corresponding to the speed of a vehicle by displaying a moving image representing a surrounding of the vehicle for the operator remotely controlling the vehicle.SOLUTION: In a correction image data generation process, an image generated by deleting pixels from edges of a moving image toward the inside of the edges, out of a moving image representing a surrounding of a vehicle 1, captured by a camera 12, is generated as a frame image constituting the moving image to be displayed on a display 202 of an operation center 200, wherein a range of deletion is extended toward the inside, or retreated toward the edges, according to a travel section of the vehicle 1 indicated by information obtained from a navigation device 13. This causes the display 202 to display the moving image with a region near the edges made invisible depending upon a ratio of articles existing in the surrounding of the vehicle 1 assumed in the travel section of the vehicle 1.

Description

  The present invention relates to a vehicle and a remote control device, and in particular, displays a moving image indicating the periphery of the vehicle to an operator who remotely operates the vehicle, thereby giving the operator a sense of speed according to the speed of the vehicle. The present invention relates to a vehicle and a remote control device that can be used.

  A vehicle in which an operation for traveling is performed from a remote control device provided in a remote place is known. Such a vehicle captures the periphery of the vehicle as a moving image using a camera provided in the vehicle, converts the captured moving image into an image signal, and transmits the image signal to the remote control device. As a result, a moving image showing the periphery of the vehicle is displayed on the screen (display) provided in the remote control device. Then, an operator in the remote control device performs an operation related to driving, braking, or steering of the vehicle while confirming a situation around the vehicle with a moving image displayed on the screen, so that a command corresponding to the operation is issued. Sent from the remote control device to the vehicle. The vehicle controls traveling according to a command received from the remote control device (for example, Patent Document 1).

  At this time, the operator grasps how fast the vehicle is traveling (obtains a sense of vehicle speed) from a moving image showing the periphery of the vehicle displayed on a screen provided in the remote control device. More specifically, in the moving image displayed on the screen, the more the scenery composed of road surfaces and objects existing around the vehicle flows in the direction opposite to the traveling direction of the vehicle, the faster the operator Receive a sense of traveling.

JP 2004-206218 A

  However, when the environment in which the vehicle travels is different, the speed sensation of the vehicle given to the operator differs depending on the moving image displayed on the screen. For example, when driving a vehicle in an environment where there is an object such as a wall installed along the side of the road, it is more remote control than when driving the vehicle in an environment where there is no object around the vehicle. In the moving image displayed on the screen of the apparatus, the speed feeling is given to the operator from the moving image by the amount that the object flows as the vehicle travels.

  Thus, when the operator operates the vehicle while watching the moving image on the screen, even if the vehicle speed is the same, a different speed sensation is given depending on the environment in which the vehicle travels. There is a problem that it is difficult for the operator to grasp the accurate vehicle speed through the above moving image.

  The present invention has been made to solve the above-described problems, and displays a moving image showing the periphery of the vehicle for a remotely operated operator, thereby allowing the operator to change the speed according to the vehicle speed. An object of the present invention is to provide a vehicle and a remote control device capable of giving a sense.

Means for Solving the Problems and Effects of the Invention

  In order to achieve this object, according to the vehicle of the first aspect, the periphery of the vehicle is imaged as a moving image by the imaging means of the vehicle. The moving image information based on the moving image captured by the vehicle imaging means is generated by the vehicle moving image information generating means, and the moving image information is remotely operated by the vehicle transmitting means provided outside the vehicle. Sent to the device. The display unit of the remote control device displays a moving image based on the moving image information transmitted from the vehicle transmission unit. With this moving image, the operator can visually recognize the situation around the vehicle, and according to the visually recognized situation, an instruction regarding driving, braking, or steering of the vehicle is input from the operator to the remote control device. When the instruction information representing the instruction by the operator regarding driving, braking or steering of the vehicle received by the remote control device is received by the receiving means of the vehicle, the driving, braking or braking of the vehicle is performed based on the received instruction information. Steering is controlled by the vehicle travel control means.

  Here, the moving image picked up by the image pickup means of the vehicle has a greater sense of perspective that makes the area closer to the outer edge of the moving image closer to the outer edge of the moving image than the inner side of the moving image. Due to the perspective, the road surface and the object flow in the moving image the fastest in the region located at the outer edge of the moving image. The operator who sees the moving image is given a speed sensation according to the speed at which the road surface and objects flow in the moving image, so the faster the road surface and objects flow in the region located at the outer edge of the moving image, The image gives the operator a faster sense of speed.

  According to the moving image information generating means of the vehicle, a moving image in which at least a part of the region located at the outer edge of the moving image is not visible based on the information about the object existing around the vehicle acquired by the information acquiring means An image is generated, and moving image information for displaying the generated moving image on the display unit of the remote control device is generated. At this time, in the moving image generated by the moving image information generating unit of the vehicle, at least a part of the region located at the outer edge of the moving image is not visible. The invisible area is the area where the road surface and the flow of the object are the fastest in the moving image. Therefore, in the moving image in which at least a part of the area located on the outer edge of the moving image is invisible, it exists around the vehicle. The speed at which the object flows is suppressed by the amount of the region that cannot be visually recognized. Therefore, it is possible to suppress the speed sensation of the operator from being accelerated by the moving image displayed on the display unit of the remote control device due to the objects present around the vehicle. Therefore, regardless of the environment in which the vehicle travels, it is possible to give the operator a sense of speed according to the speed of the vehicle.

  In addition, in claim 1, “generating a moving image in which at least a part of a region located at the outer edge of the moving image is invisible” means that the moving image information generating unit performs the moving image information generating unit on the moving image captured by the imaging unit. In addition to the case of generating a moving image in which at least a part of the region located at the outer edge of the moving image is not visible, the region located at the outer edge of the moving image by the moving image information generating unit controlling the imaging unit This is a concept that includes a case where a moving image in which at least a part of the image is invisible is output from the imaging unit.

  The vehicle according to claim 2 has the following effect in addition to the effect of the vehicle according to claim 1. That is, information indicating the positional relationship of an object existing in the vicinity of the vehicle with respect to the vehicle is acquired as information related to the object existing in the vicinity of the vehicle in the environment where the vehicle travels. When the moving image information generating unit of the vehicle generates a moving image in which at least a part of the region located at the outer edge of the moving image is invisible, the first region setting unit of the moving image information generating unit generates the moving image. The region that is not visible in the image is set as a part of the region located at the outer edge of the moving image in the direction in which the object existing around the vehicle exists based on the information acquired by the information acquisition unit. For this reason, the object whose positional relationship with the vehicle is indicated by the information acquired by the information acquisition unit is not displayed on the display unit of the remote control device as to how it flows at high speed in the region located at the outer edge of the moving image. There is an effect that it is possible to prevent the speed sensation of the operator from being accelerated by the moving image displayed on the display means of the remote control device due to the objects present around the vehicle.

  The vehicle according to claim 3 has the following effect in addition to the effect produced by the vehicle according to claim 1 or 2. That is, the present inventor views the moving image by stimulating the operator's vision as the proportion of the object in the moving image displayed on the display unit of the remote control device increases. We found that the operator's sense of speed tends to increase.

  On the other hand, according to the vehicle of the third aspect, the vehicle information acquisition means causes the predetermined information regarding the proportion of the object in the range imaged by the vehicle imaging means in the environment where the vehicle travels to travel. It is acquired as information related to objects existing around the vehicle in the environment. Then, based on the predetermined information acquired by the information acquisition means, the moving image generated by the moving image information generation means when the proportion of the object in the range imaged by the imaging means of the vehicle is the first ratio is: It is set from the outer edge of the moving image to the inner side, rather than the range of the region that is not visible when the second ratio is lower than the first ratio. Therefore, based on the predetermined information acquired by the information acquisition means, the higher the proportion of objects in the range imaged by the imaging means of the vehicle, the larger the area that is not visible that is located at the outer edge of the moving image. The speed sensation given to the operator from the object flowing through the moving image can be slowed. Accordingly, there is an effect that it is possible to prevent the speed sensation of the operator from being accelerated by the moving image displayed on the display unit of the remote control device, regardless of the proportion of the object around the vehicle.

  According to the vehicle of the fourth aspect, in addition to the effect produced by the vehicle according to any one of the first to third aspects, the following effect is produced. That is, when the moving image information generating unit generates a moving image in which at least a part of the region located on the outer edge of the moving image is invisible, the region that is invisible in the moving image is first detected. The region where the object detected by the means is present is set to be visible. The object detected by the first detection unit is an object that exists within a predetermined distance from the vehicle in a range where a moving image is captured by the imaging unit. Therefore, the operator can visually recognize an object existing within a predetermined distance from the vehicle by the moving image displayed on the display unit of the remote control device. Therefore, when an object existing in the vicinity of the vehicle suppresses an increase in the speed sensation of the operator and an object existing within a predetermined distance from the vehicle hinders the vehicle from running, There is an effect that the operator can perform an operation for avoiding it.

  According to the vehicle of the fifth aspect, in addition to the effect produced by the vehicle according to any one of the first to fourth aspects, the following effect is produced. That is, when the moving image information generating unit generates a moving image in which at least a part of the region located at the outer edge of the moving image is not visible, the region that is not visible in the moving image is The area where the movable object detected by the second detection means is present is set by the third area setting means of the moving image information generation means so as to be visible. In such a case, the moving object displayed on the display unit of the remote control device can be viewed by the operator about the movable object imaged by the imaging unit. Therefore, a region that is invisible in the moving image displayed on the display means is set, and the movement that exists in the vicinity of the vehicle is suppressed while suppressing an increase in the speed sensation of the operator by an object that exists in the vicinity of the vehicle. In the case where a possible object hinders traveling of the vehicle, there is an effect that the operator can perform an operation for avoidance.

  According to the vehicle of the sixth aspect, in addition to the effect produced by the vehicle of the fifth aspect, the following effect is produced. That is, in the second detection means, the optical flow in each region of the moving image is calculated from the moving image captured by the imaging means of the vehicle by the optical flow calculation means. The optical flow of each area calculated by the optical flow calculation means is compared with the optical flow calculated for the area surrounding each area by the optical flow comparison means of the second detection means. Then, an optical flow that does not include at least one of the direction and the size is calculated with respect to at least one distribution of the direction and the size of the optical flow calculated for the surrounding regions compared by the optical flow comparison unit. The detection by the second detection means is performed assuming that a moving object exists in the area. Here, the optical flow for each area calculated by the optical flow calculation means is in accordance with the road surface and the flow of the object in each area. For this reason, by detecting a region where an optical flow that does not include at least one of the direction and the size is calculated with respect to at least one of the direction and the size of the optical flow calculated for the surrounding region. There is an effect that a moving object can be detected by reliably specifying a region where a moving object having a different flow from a region having a uniform flow such as a road surface or a stationary object exists.

  According to the remote control device of the seventh aspect, the following effects are obtained. That is, when the moving image information generated based on the moving image around the vehicle imaged by the imaging unit provided in the vehicle is received from the vehicle by the receiving unit of the remote control device, the receiving unit receives the moving image information. A moving image is displayed based on the moving image information received by. With this moving image, the operator can visually recognize the situation around the vehicle, and an instruction relating to driving, braking, or steering of the vehicle, which is performed by the operator according to the recognized situation, is received by the receiving unit. Then, the instruction information representing the instruction received by the receiving unit is transmitted to the vehicle by the transmitting unit. In this way, in the remote control device, operations related to driving, braking or steering of the vehicle are performed from the outside of the vehicle.

  According to the display control unit of the remote operation device, the moving image indicated by the moving image information received by the receiving unit of the remote operation device exists around the vehicle in the environment where the vehicle acquired by the information acquisition unit travels. Based on the information related to the object, a moving image is generated in which at least a part of the region located at the outer edge of the moving image indicated by the received moving image information is invisible, and the moving image is displayed on the display unit of the remote control device Let At this time, in the moving image generated by the display control means of the remote control device, at least a part of the area where the road surface and the object flow are fastest in the moving image, which is located at the outer edge of the moving image, is not visible. Thus, it is possible to slow down the speed sensation given to the operator who viewed the moving image by slowing the flow of the object and the road surface of the entire moving image by the amount of the region that cannot be viewed. Therefore, it is possible to prevent the speed sensation of the operator from being accelerated by the moving image displayed on the display means of the remote control device due to objects existing around the vehicle in the environment where the vehicle is traveling. Therefore, regardless of the environment in which the vehicle travels, it is possible to give the operator a sense of speed according to the speed of the vehicle.

It is the schematic diagram which showed typically the vehicle and operation center in 1st Embodiment. (A) is a schematic diagram which shows the imaging range of a camera, (b) is a schematic diagram which shows the moving image imaged in the imaging range of (a). It is the block diagram which showed the electric structure of the vehicle and the operation center. It is the schematic diagram which illustrated the angle-of-view map typically. It is the schematic diagram which illustrated typically the minimum frame rate memory. It is a flowchart which shows the image transmission process performed by CPU of a vehicle side control apparatus. It is a flowchart which shows the correction image data generation process performed by CPU of a vehicle side control apparatus. It is explanatory drawing for demonstrating the method of calculating the center position angle of a horizontal direction from the gaze point on a prediction path | route. (A) is a schematic diagram showing a frame image A generated by the corrected image data generation process when the vehicle travels in the “other” travel section, and (b) is a diagram illustrating the vehicle traveling in the “urban area”. FIG. 7C is a schematic diagram showing a frame image B generated by the corrected image data generation process when the vehicle travels through the “tunnel” when the vehicle travels through the “tunnel”. FIG. It is a flowchart which shows the image display process performed by CPU of an operation side control apparatus. It is the block diagram which showed the electric structure of the vehicle and operation center in 2nd Embodiment. It is a flowchart which shows the image transmission process performed by CPU of the vehicle side control apparatus in 2nd Embodiment. It is a flowchart which shows the image display process performed by CPU of the operation side control apparatus in 2nd Embodiment. It is the block diagram which showed the electric constitution of the vehicle in 3rd Embodiment. It is a flowchart which shows the correction image data generation process performed by CPU of the vehicle side control apparatus in 3rd Embodiment. It is explanatory drawing for demonstrating the optical flow of the moving image imaged with the camera.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings. First, with reference to FIGS. 1-10, one Embodiment at the time of applying this invention to the vehicle 1 is described as 1st Embodiment.

  FIG. 1 is a schematic diagram schematically showing a vehicle 1 and an operation center 200 which is an operation facility (remote operation device) for performing an operation of the vehicle 1 from a remote place. First, a schematic configuration of the vehicle 1 and the operation center 200 will be described with reference to FIG. The vehicle 1 can give the operator a sense of speed according to the speed of the vehicle by displaying a moving image showing the periphery of the vehicle 1 to an operator who remotely operates the vehicle 1 from the operation center 200. It is configured.

  As shown in FIG. 1, the vehicle 1 is a two-way wireless communication via a relay station 300 between a vehicle-side control device 100 provided in the vehicle 1 and an operation-side control device 201 provided in the operation center 200. In order to display a moving image obtained by imaging the periphery of the vehicle 1 to the operator of the vehicle 1 in the operation center 200, the moving image is based on processing performed by the vehicle-side control device 100. Is to send. Further, traveling of the vehicle 1 is controlled in accordance with a command transmitted from the operation side control device 201. In addition to the vehicle-side control device 100, the vehicle 1 includes a plurality of (four wheels in this embodiment) wheels 2FL, 2FR, 2RL, 2RR and a part of these wheels 2FL to 2RR (in this embodiment). , Front wheel 2FL, 2FR), a wheel drive device 3 for rotating and driving, a steering drive device 5 for steering and driving a part of the plurality of wheels 2FL-2RR (in this embodiment, front wheels 2FL, 2FR), and vehicle 1 A speed sensor 10 for detecting the vehicle speed, a steering angle sensor 11 for detecting the steering angle of the vehicle 1, a camera 12 for imaging the periphery of the vehicle 1, a navigation device 13, and a communication device 14. Has mainly.

  The vehicle-side control device 100 is a computer device that controls the traveling of the vehicle 1 in response to a command from the operation-side control device 201 received by the communication device 14. In addition, the vehicle-side control device 100 controls to transmit image data based on a moving image around the vehicle 1 captured by the camera 12 to the operation-side control device 201 of the operation center 200 via the communication device 14. Is done. Details of the vehicle-side control device 100 will be described later with reference to FIG.

  The wheels 2FL and 2FR are left and right front wheels disposed on the front side of the vehicle 1 (left side in FIG. 1), and are configured as drive wheels that are rotationally driven by the wheel drive device 3. On the other hand, the wheels 2RL and 2RR are left and right rear wheels disposed on the rear side of the vehicle 1 (the right side in FIG. 1), and are configured as driven wheels that are driven as the vehicle 1 travels.

  The wheel driving device 3 is a device for applying a rotational driving force to the left and right front wheels 2FL, 2FR. The wheel driving device 3 applies a rotational driving force to the left and right front wheels 2FL, 2FR based on a control signal indicating the target value of the vehicle speed notified from the vehicle-side control device 100. Thereby, the vehicle 1 travels at a speed corresponding to the vehicle speed notified from the vehicle-side control device 100.

  The steering drive device 5 is a device for steering the left and right front wheels 2FL, 2FR. The steering drive device 5 steers the left and right front wheels 2FL and 2FR based on a control signal notified from the vehicle-side control device 100 and indicating a target steering angle. Thus, in the vehicle 1, the front wheels 2FL and 2FR are steered to the steering angle instructed from the vehicle side control device 100.

  The speed sensor 10 is a sensor that detects the wheel speed of each of the wheels 2FL to 2RR. The wheel speeds of the wheels 2FL to 2RR detected by the speed sensor 10 are input to the vehicle-side control device 100, and the vehicle-side control device 100 determines the speed of the vehicle 1 (vehicle speed) based on the input wheel speed. And acceleration are calculated.

  The steering angle sensor 11 is a sensor that detects the steering angle of the vehicle 1. The steering angle of the vehicle 1 detected by the steering angle sensor 11 is input to the vehicle side control device 100.

  The camera 12 is an imaging device for capturing the periphery of the vehicle 1 as a moving image, and is configured by a digital camera on which an imaging element such as a CCD image sensor or a CMOS image sensor is mounted. The camera 12 is disposed on the front side of the vehicle 1 and captures an image of the front of the vehicle (left side in FIG. 1) that can be the traveling direction of the vehicle 1. Here, the moving image imaged by the camera 12 is demonstrated with reference to FIG. 2 (a), (b).

  2A is a schematic diagram illustrating an imaging range of the camera 12, and FIG. 2B is a schematic diagram illustrating a moving image captured in the imaging range of FIG. 2A.

  The camera 12 is configured to be able to image a range of a horizontal field angle of 120 ° and a vertical field angle of 70 ° with the camera 12 as a center.

  As shown in FIG. 2A, when the vehicle 1 travels, an object (mainly a stationary object) existing on the road surface around the vehicle 1 or around the vehicle 1 such as a building is captured by an imaging range ( It is included in the range surrounded by a dotted line. A moving image captured by the camera 12 in the range shown in FIG. 2A is the moving image shown in FIG.

  In FIG. 2B, the plurality of auxiliary lines m extending from the edges E1 to E4 of the moving image toward the vanishing point (not shown) at the center of the moving image are parallel straight lines. The plurality of auxiliary lines n orthogonal to the auxiliary line m are also parallel straight lines, and the interval between the adjacent auxiliary lines n corresponds to a certain distance in real space for any auxiliary line n. For example, in the auxiliary line m1, the interval a delimited by the adjacent auxiliary line n and the interval b delimited by the auxiliary line n adjacent on the vanishing point side from the interval a are the same distance in real space. Become. The vanishing point that converges all the auxiliary lines m shown in FIG. 2B is an infinite point on the optical axis of the camera 12. The auxiliary lines m and n are for convenience of explanation and are not displayed on the display 202.

  In FIG. 2B, as the vehicle 1 travels, the camera 12 captures a moving image in which the road surface or the like flows in a direction opposite to the direction in which the vehicle 1 travels. Here, as can be seen from the comparison between the distance a and the distance b indicating the same distance on the auxiliary line m1, the moving image captured by the camera 12 is closer to the vehicle 1 in the region closer to the edges E1 to E4. Things look more magnified. Therefore, when an object existing in a region close to the edges E1 to E4 and a region farther from the edges E1 to E4 than that region moves in the same direction in the real space at the same speed, it is close to the edges E1 to E4. The region has a larger amount of visual movement and appears to move faster. For this reason, when the vehicle 1 travels, the road surface, the object, etc. flow the fastest in the region located at the edges E1 to E4 of the moving image where the size of the appearance is maximized, and from the edges E1 to E4 to the inside of the moving image. The further away, the more slowly the road surface and objects flow.

  When the moving image picked up by the camera 12 is displayed to the operator in the operation center 200, the operator sensuously grasps the speed of the vehicle 1 based on the speed of the flow on the road surface (vehicle). 1 speed sensation). In the moving image, the speed of the vehicle given to the operator from the moving image becomes faster as the road surface flows faster.

  Returning to FIG. 1, the description will be continued. The camera 12 generates moving image image data in units of frame images. In other words, the camera 12 converts frame images (still images) taken every time into image data one by one and outputs the image data to the vehicle-side control device 100. In the vehicle side control device 100, based on the image data acquired from the camera 12, image data of a moving image to be displayed at the operation center 200 is output to the communication device 14. Note that the image data of one frame image includes at least color information about each pixel (pixel) constituting the frame image.

  The navigation device 13 is a device for acquiring the current position of the vehicle 1 using GPS and acquiring information related to the section in which the vehicle 1 travels in light of map information, and receives radio waves from a plurality of GPS satellites. A current position acquisition unit (not shown) for acquiring the current position and traveling direction of the vehicle 1, and an information storage unit (not shown) for storing map information in which data on roads, intersection data, and the like are recorded. The vehicle side control device displays the current position of the vehicle 1 acquired by the current position acquisition unit, the current position and the map information stored in the information storage unit, and information indicating the section in which the vehicle 1 travels. And an output unit (not shown) for outputting to 100.

  In the present embodiment, the information indicating the section in which the vehicle 1 travels output from the navigation device 13 is the presence of an object such as a building or a road structure around the vehicle 1 in the section in which the vehicle 1 travels, and It shows what positional relationship the object has with the vehicle 1. Such information in the present embodiment includes information indicating that the section in which the vehicle 1 travels is a “tunnel” in which a ceiling and a wall exist above and to the left and right sides of the vehicle 1, Information indicating that it is an “urban area” in which a building or a road structure exists as an object in at least one and information indicating that it is an “other” travel section are included. The “tunnel” here is a section where a tunnel is provided on the travel path in the map information of the navigation device 13, and the “urban area” is a section other than “tunnel” in the map information of the navigation device 13. Thus, it includes at least a section where a building exists on land adjacent to the traveling road and a section where fences and side walls exist on the side of the traveling road. The “other” traveling section is a section where there is no structure or building conspicuous around the vehicle, which is exemplified by a section in which the land adjacent to the traveling road is a rice field or a field.

  On the other hand, the current position of the vehicle 1 output from the navigation device 13 to the vehicle-side control device 100 is transmitted to the operation center 200 via the communication device 14. Thereby, in the operation center 200, the present position of the vehicle 1 can be acquired, and the guidance from the present location of the vehicle 1 to the destination can be performed.

  The communication device 14 is a device that performs wireless communication with a remote operation center 200. The communication device 14 transmits the image data received from the vehicle side control device 100, the vehicle speed data, the steering angle, and the output from the navigation device 13 to the operation center 200 via the relay station 300. Further, the communication device 14 receives a signal transmitted from the operation center 200 and outputs it to the vehicle side control device 100. A communication system used by the communication device 14 includes a communication system used in a general wireless communication device, a mobile phone, a communication system (for example, W-CDMA) used for PHS, a wireless LAN, and the like. A communication system such as

  As shown in FIG. 1, the operation center 200 communicates wirelessly between the operation side control device 201 provided in the operation center 200 and the vehicle side control device 100 of the vehicle 1 via the relay station 300. It is equipment that performs. The operation center 200 displays a moving image in which the periphery of the vehicle 1 transmitted from the vehicle-side control device 100 is captured to the operator, and performs an operation of the vehicle 1 performed by the operator based on the moving image. It is for operating the vehicle 1 from a remote place by receiving and transmitting a command corresponding to the operation to the vehicle 1.

  In the operation center 200, in addition to the operation side control device 201, a display 202, a steering device 203, an accelerator pedal device 204, a brake pedal device 205, and a communication device 206 are provided.

  The operation side control device 201 transmits the image data (frame image image data) received from the vehicle side control device 100 by the communication device 206 to the display 202 each time it is received, as well as the steering device 203 and the accelerator pedal device 204. The computer device generates command information for controlling the travel of the vehicle 1 based on the input to each device of the brake pedal device 205. The detailed configuration of the operation side control device 201 will be described later with reference to FIG.

  The display 202 is a display device that displays a moving image based on the image data output from the operation-side control device 201. The display 202 is configured in a curved shape so as to surround an operator who views the display 202. Although details will be described later, every time the display 202 receives the image data of the frame image output from the operation-side control device 201, the display 202 switches the display content of the display 202 to the frame image indicated by the newly received image data. In the present embodiment, image data based on a moving image indicating the periphery of the vehicle 1 captured by the camera 12 of the vehicle 1 is output from the operation-side control device 201, so that the display 202 indicates the periphery of the vehicle 1. A moving image is displayed. Therefore, on the display 202, an object on the road surface or around the vehicle 1 flows in a direction opposite to the traveling direction of the vehicle 1 as the vehicle 1 travels based on the moving image captured by the camera 12 of the vehicle 1. A state of going is displayed. By viewing this moving image, the operator visually recognizes the situation around the vehicle 1 and performs the operation of the vehicle 1 corresponding to the situation with respect to the steering device 203, the accelerator pedal device 204, or the brake pedal device 205.

  Further, the operator grasps the speed of the vehicle 1 sensuously by the flow on the road surface or the like displayed as a moving image on the display 202 (obtains a sense of speed of the vehicle 1). As the road surface and the like flow faster in the moving image, a sense of speed that the vehicle 1 is traveling at a higher speed is obtained.

  The steering device 203 receives an instruction on the steering direction of the vehicle 1 when a rotation operation is input from an operator in the operation center 200. When the steering device 203 is rotated by the operator, the steering device 203 transmits a rotation angle corresponding to the operation to the operation-side control device 201. Note that the steering device 203 may transmit the rotation angular velocity rotated by the operator to the operation-side control device 201. Then, the operation-side control device 201 may calculate the rotation angle by integrating the rotation angular velocity acquired from the steering device 203.

  The accelerator pedal device 204 and the brake pedal device 205 receive an instruction about the vehicle speed of the vehicle 1 when a stepping operation is input from an operator in the operation center 200. The accelerator pedal device 204 includes a pedal that is stepped on by an operator to advance the vehicle 1 in the front-rear direction, and a command value corresponding to the pedal depression state (stepping amount, stepping speed, etc.) is given to the operation side control device. To 201. The operation side control device 201 calculates a target value of the vehicle speed according to the command value. On the other hand, the brake pedal device 205 includes a pedal that is depressed to brake (decelerate) the vehicle 1, and transmits a command value corresponding to the depressed state of the pedal to the operation-side control device 201. Thereby, the target value of the vehicle speed managed by the operation side control device 201 is reduced according to the pedal depression state in the brake pedal device 205. The driving force and braking force of the vehicle 1 are controlled based on the target value of the vehicle speed determined according to the operation of the accelerator pedal device 204 and the brake pedal device 205. The operation side control device 201 sequentially generates speed command information based on the target value of the vehicle speed and outputs it to the communication device 206.

  The communication device 206 is a device that performs wireless communication with the vehicle 1. The communication device 206 transmits a signal such as command information received from the operation side control device 201 to the vehicle 1 via the relay station 300. The communication device 206 receives a signal such as image data transmitted from the vehicle 1 and outputs the signal to the operation side control device 201.

  Next, detailed configurations of the vehicle 1 and the operation center 200 will be described with reference to FIG. FIG. 3 is a block diagram showing an electrical configuration of the vehicle 1 and the operation center 200.

  First, the vehicle 1 will be described. The vehicle side control device 100 of the vehicle 1 includes a CPU 91, a ROM 92, and a RAM 93, which are connected to an input / output port 95 via a bus line 94. The input / output port 95 is connected to the wheel drive device 3, the steering drive device 5, the speed sensor 10, the steering angle sensor 11, the camera 12, the timer 15, the communication device 14, and other input / output devices 99. .

  The CPU 91 controls the wheel driving device 3 and the steering driving device 5 on the basis of various information transmitted from the sensors 10 and 11 connected to the input / output port 95, the communication device 14, and the like, as well as the camera 12 and the communication. The arithmetic device generates image data of an image to be displayed on the display 202 of the operation center 200 on the basis of various information transmitted from the device 14 or the like.

  The ROM 92 is a non-rewritable nonvolatile memory for storing a control program executed by the CPU 91, fixed value data, and the like. The ROM 92 is provided with a program memory 92a, a view angle map memory 92b, and a minimum frame rate memory 92c.

  The program memory 92a is an area on the ROM 92 in which various programs executed by the CPU 91 are stored. Each program for causing the CPU 91 to execute various processes such as an image transmission process shown in the flowchart of FIG. 6 to be described later is stored in the program memory 92a. The CPU 91 executes processing in accordance with the program stored in the program memory 92a, thereby displaying a moving image showing the periphery of the vehicle 1 to an operator who remotely operates the vehicle 1 from the operation center 200, thereby performing the operation. Control is performed to give the person a sense of speed according to the speed of the vehicle.

  The view angle map memory 92b is a memory for generating a frame image of a moving image that gives the operator a sense of speed according to the actual vehicle speed. The angle-of-view map memory 92b stores an angle-of-view map that defines the angle of view (horizontal angle of view) of the moving image displayed on the display 202. In this angle-of-view map, the travel rate of the vehicle 1 and the frame rate of the moving image displayed on the display 202 (the number of frame images based on the moving image captured by the camera 12 displayed per hour on the display 202). The angle of view is defined in association with the vehicle speed acquired from the speed sensor 10. The angle-of-view map memory 92b is referred to by the CPU 91 for correcting the angle of view of the frame image indicated by the image data acquired from the camera 12 in an image transmission process (see FIG. 6) described later. . Specifically, data indicating pixels on the wide angle side (outside) from the angle of view obtained from the angle of view map memory 92b is deleted from the image data indicating the frame image of the moving image captured by the camera 12. Thus, the angle of view of the frame image is corrected. When the corrected image data indicating the frame image with the corrected angle of view is transmitted to the operation center 200, a moving image obtained by cutting out a part of the display range from the moving image acquired from the camera 12 is displayed on the display 202. The

  Here, the scenery around the vehicle 1 in the traveling direction imaged by the vehicle 1 found by the present inventor is displayed on the display 202 of the operation center 200, and the operator who views the display 202 is operated by the operator of the vehicle 1. A problem that occurs when the process is performed will be described.

  As described above, the display 202 displays a moving image showing the periphery of the vehicle 1 captured by the camera 12 of the vehicle 1. In this moving image, as the vehicle 1 travels, the road surface and objects around the vehicle 1 flow in a direction opposite to the traveling direction of the vehicle 1. An operator who remotely operates the vehicle 1 while looking at the display 202 is given a sense of speed of the vehicle 1 according to the speed of the road surface and the like flowing on the display 202. However, when the frame rate of the moving image is low, the flow on the road surface or the like displayed as the moving image is not smooth. When such a moving image is displayed to the operator, the operator's sense of speed tends to be higher than the actual vehicle speed.

  In addition, as one of the causes that the frame rate of the moving image displayed on the display 202 is lowered, the communication speed of the transmission path for transmitting the image data from the communication device 14 of the vehicle 1 to the communication device 206 of the operation center 200 is slow. Can be mentioned. If the communication speed on this transmission path is slow, the communication time required to transmit one frame image of the moving image from the communication device 14 of the vehicle 1 to the communication device 206 of the operation center 200 becomes long. This is because the frame rate of the moving image displayed on the display 202 of the operation center 200 is lower than when the communication speed of the transmission path is sufficient.

  Also, if the angle of view of the moving image is constant (for example, horizontal angle of view 120 °, vertical angle of view 70 °), the edge of the moving image that flows the fastest on the road surface among the moving images as the vehicle speed increases. In the vicinity, the road surface etc. will flow at a higher speed than the vehicle speed, and the speed sensation given to the operator who sees the moving image will be faster than that according to the actual vehicle speed. Tend.

  Moreover, the speed sensation of the operator received from the moving image is affected by the speed of the road surface and the flow of the object in the moving image, so the presence of the object in the moving image, the proportion of the object in the display range in the moving image, The speed sensation varies depending on environmental factors such as the distance between the vehicle 1 and the object. For example, when the vehicle 1 is driven in an environment in which a wall is disposed along the travel path on the side of the travel path, a state in which the wall flows is displayed in a moving image captured as the vehicle 1 travels. Thus, a speed sensation faster than when the vehicle 1 is driven in an environment where no object exists around the vehicle 1 is given to the operator who has viewed the moving image. In addition, even when the vehicle is the same wall, when the vehicle 1 travels along the wall and when traveling along the wall at a position farther than that, the vehicle 1 travels along the wall. Since the wall appears to be enlarged in the moving image, the flow of the wall in the moving image is faster, giving a faster sense of speed to the operator.

  Thus, even if a moving image is displayed to the operator due to factors such as the frame rate and angle of view of the moving image and the environment in which the vehicle 1 travels, the operator can grasp the accurate vehicle speed through the moving image. There was a problem that it was difficult. If the speed sensation of the operator becomes faster than the actual vehicle speed, for example, the vehicle speed of the vehicle 1 is matched with the vehicle speed of other vehicles existing around the vehicle 1 and rides on the flow of the other vehicle. Even if it is going to travel, the acceleration / deceleration operation of the vehicle 1 in accordance with the flow of other vehicles becomes difficult for the operator.

  With respect to such a problem, in the present embodiment, according to the traveling section of the vehicle 1, the frame rate of the moving image displayed on the display 202 of the operation center 200, and the vehicle speed of the vehicle 1, FIG. A moving image to be displayed on the display 202 is generated by the vehicle-side control device 100 at an angle of view defined by the angle of view map shown.

  Here, the angle-of-view map will be described with reference to FIG. FIG. 4 is a schematic diagram schematically showing an angle-of-view map. As shown in FIG. 4, the view angle map is a frame rate of a moving image displayed on the display 202 of the operation center 200, a vehicle speed of the vehicle 1, and an angle of view of an image transmitted as image data to the operation center 200 ( 2 is a map showing a relationship with a horizontal angle of view) for each travel section of the vehicle 1. The horizontal axis indicates the frame rate, and the vertical axis indicates the angle of view. In addition, the view angle map indicates that the traveling section of the vehicle 1 is “when the vehicle speed is low (for example, 0 to 30 km / h), medium (30 to 60 km / h), and high (60 km / h)”. The angle of view corresponding to the frame rate is defined for each of “urban area” and “tunnel”.

  The view angle map memory 92b is referred to according to the vehicle speed or the frame rate of the moving image displayed on the display 202 when the traveling section of the vehicle 1 is “tunnel” or “city”. Is obtained from the navigation device 13, the image data of the frame image obtained from the camera 12 of the vehicle 1 is output to the communication device 14 as image data of a moving image to be displayed on the display 202. Is done.

  As shown in FIG. 4, the angle-of-view map shows that the frame rate of the moving image displayed on the display 202 acquired from the operation center 200 in each travel section and vehicle speed is a threshold Nl, which is set for each vehicle speed. A constant angle of view (reference angle of view) is shown until it falls below Nm and Nh. On the other hand, the view angle map shows a view angle smaller (narrower) than the reference view angle at a frame rate below the threshold values Nl, Nm, and Nh. When the vehicle speed is low and the travel section is “urban area”, the reference angle of view indicated by the frame rate range exceeding the threshold Nl is the angle of view of the moving image (horizontal image) captured by the camera 12. The angle of view is the same as the angle 120 ° and the vertical angle 70 °.

  The thresholds Nl, Nm, and Nh are the cases where the vehicle speed is low, medium, and high, respectively, and the operator who sees the moving image displayed on the display 202 starts to feel a speed sensation faster than the actual vehicle speed. This is the frame rate of the moving image. In the running experiment, it was found that the operator feels a speed sensation faster than the actual vehicle speed when the frame rate of the moving image on the display 202 is higher as the vehicle speed is higher. Therefore, in the angle-of-view map, the frame rate thresholds Nl, Nm, and Nh are set to larger values (Nl <Nm <Nh) as the vehicle speed increases.

  As shown in FIG. 4, when the frame rate of the moving image displayed on the display 202 is below the threshold values Nl, Nm, and Nh, the view angle map shows a small view angle in proportion to the decrease in the frame rate.

  In the view angle map, if the frame rate of the moving images displayed on the display 202 is the same, a smaller view angle is set as the vehicle speed increases.

  Further, if the vehicle speed and the frame rate are the same, the angle-of-view map is smaller when the traveling section of the vehicle 1 is “tunnel” than when the traveling section of the vehicle 1 is “urban area”. A corner is set.

  Although details will be described later with reference to FIG. 7, when the traveling environment of the vehicle 1 is “urban area”, only pixels near the left and right edges are deleted from the image captured by the camera 12. Thus, modified image data indicating a frame image with a reduced horizontal angle of view with respect to the image captured by the camera 12 is generated. On the other hand, when the traveling environment of the vehicle 1 is “tunnel”, the pixels near the top, bottom, left, and right edges are deleted from the image captured by the camera 12 and the image captured by the camera 12 is perpendicular to the image. Corrected image data indicating a frame image with a reduced angle of view and horizontal angle of view is generated. The vertical angle of view of the frame image is acquired by multiplying the horizontal angle of view defined in the angle-of-view map by a predetermined proportionality constant (positive constant).

  Returning to FIG. 3, the description will be continued. The minimum frame rate memory 92c is a memory in which a minimum frame rate (minimum frame rate) to be secured in the moving image displayed on the display 202 is stored in association with each vehicle speed. The minimum frame rate memory 92c is referred to by the CPU 91 in an image transmission process (see FIG. 6) described later.

  Here, the minimum frame rate memory will be described with reference to FIG. FIG. 5 is a schematic diagram schematically showing the minimum frame rate memory 92c.

  In FIG. 5, the vertical axis indicates the minimum frame rate, and the horizontal axis indicates the vehicle speed. As shown in FIG. 5, the minimum frame rate memory 92c indicates that the minimum frame rate increases as the vehicle speed increases.

  When the frame rate of the moving image displayed on the display 202 is lowered, the speed sensation of the operator who sees the moving image becomes faster than the actual vehicle speed. In addition, as the vehicle speed increases, the difference between the actual vehicle speed and the operator's sense of speed tends to increase.

  On the other hand, although details will be described later, in the image transmission processing (see FIG. 6) described later, when the frame rate of the moving image displayed on the display 202 is lower than the minimum frame rate, the CPU 91 Reduce the target vehicle speed. Thereby, since the vehicle 1 decelerates, the difference with the speed sensation of the operator with respect to the actual vehicle speed can be reduced.

  Further, when the frame rate of the moving image displayed on the display 202 is lower than the minimum frame rate, there is a possibility that a communication failure has occurred in the transmission path for transmitting image data from the vehicle 1 to the operation center 200. Therefore, by decelerating the vehicle 1, it is possible to prompt the operator to stop until the vehicle 1 recovers from the communication failure.

  Returning to FIG. 3, the description will be continued. The RAM 93 is a rewritable volatile memory, and is a memory for temporarily storing various data when a program executed by the CPU 91 is executed. The RAM 93 includes at least an image data memory 93a, a transmission image buffer 93b, a transmission permission flag 93c, a time memory 93d, a vehicle speed memory 93e, a steering angle memory 93f, and a reception frame rate memory 93g. .

  The image data memory 93a is a memory used for temporarily storing image data acquired from the camera 12. The image data memory 93a has two storage areas. The image data memory 93a stores, in one storage area, image data transferred from the camera 12 in response to a command from the CPU 91 when the transmission permission flag 93c is turned on in an image transmission process (see FIG. 6) described later. . At this time, image data stored in advance in the other storage area is transferred to a transmission image buffer 93c described later. In an image transmission process (see FIG. 6) to be described later, a transmission permission flag 93c includes a storage area that is a transfer destination of an image captured by the camera 12 and a storage area in which image data is transferred to the transmission image buffer 93c. Each time is turned on, it is used by switching alternately. In the description of the image transmission process of FIG. 6 to be described later, the description of the flow of processing using these two storage areas is omitted.

  The transmission image buffer 93b is a memory for temporarily storing image data (corrected image data) to be transmitted to the operation center 200. The transmission image buffer 93b has two storage areas. The transmission image buffer 93b stores image data (corrected image data) based on the image data transferred from the image data memory 93a in one storage area in an image transmission process (see FIG. 6) described later. At this time, image data (corrected image data) stored in advance in the other storage area is transferred to the communication device 14. Then, the image data (corrected image data) output from the transmission image buffer 93 b to the communication device 14 is transmitted from the communication device 14 via the relay station 300 to the operation center as a moving image frame image displayed on the display 202. 200 is transmitted. In an image transmission process (see FIG. 6) described later, a storage area to which image data (corrected image data) transferred from the image data memory 93a is transferred and a storage in which the image data is transferred to the transmission image buffer 93c. The areas are alternately used each time the transmission permission flag 93c is turned on. In the description of the image transmission process of FIG. 6 to be described later, the description of the flow of processing using these two storage areas is omitted.

  The transmission permission flag 93 c is a flag indicating whether or not the communication device 14 is permitted to output image data (corrected image data) from the vehicle-side control device 100 to the communication device 14. The transmission permission flag 93d is turned on by an interrupt process (not shown) that occurs when a transmission permission signal indicating permission from the communication device 14 is input to the CPU 91. Here, the communication permission signal is a signal transmitted from the communication device 14 at a timing when transmission of image data (corrected image data) for one frame image to the operation center is completed. This is because data is overwritten during transmission because another image data (corrected image data) is output from the vehicle-side control device 100 while the communication device 14 is transmitting image data (corrected image data). This is to prevent it from being done. When the transmission permission flag 93d is on, the image data (corrected image data) in the transmission image buffer 93b is output (transferred) to the communication device 14 and transmitted in an image transmission process (see FIG. 6) described later. The permission flag 93d is turned off. On the other hand, when the transmission permission flag 93d is in the OFF state, the image transmission process waits for the image data (corrected image data) to be output to the communication device 14.

  The time memory 93d is a memory for storing a time stamp indicating the date and time when the image data based on the image captured by the camera 12 is acquired. The time stamp stored in the time memory 93d is date / time information output from the timer 15 described later. In the image transmission process (see FIG. 6), every time image data from the camera 12 is acquired, the CPU 91 acquires date / time information output from the timer 15 and stores the date / time information in the time memory 93d. . The time stamp stored in the time memory 93d is output to the communication device 14 together with image data (corrected image data) based on the image acquired by the CPU 91 at the date and time indicated by the time stamp. As a result, the time stamp is transmitted to the operation center 200 via the communication device 14 together with the image data (corrected image data) captured at the date and time indicated by the time stamp.

  Here, the timer 15 will be described. The timer 15 is a digital timer and outputs information indicating the current date and time in milliseconds. On the other hand, in this embodiment, the CPU 91 acquires about 60 images from the camera 12 per second (about one image in about 17 milliseconds). Therefore, by using the date / time information generated by the timer 15, the CPU 91 can create a time stamp indicating the correct date / time.

  Next, the vehicle speed memory 93e will be described. The vehicle speed memory 93e is a memory for temporarily storing speed data indicating the vehicle speed of the vehicle 1 calculated by the CPU 91 based on the detection result of the speed sensor 10. The speed data stored in the vehicle speed memory 93e is updated to data newly acquired by the CPU 91 from the speed sensor 10 every time the image transmission process (see FIG. 6) is executed. The updated value of the vehicle speed memory 93e is referred to by the CPU 91 in order to acquire the value of the view angle map memory 92b and the value of the minimum frame rate memory 92c in the image transmission process.

  The steering angle memory 93 f is a memory for temporarily storing steering angle data indicating the steering angle of the vehicle 1 acquired by the CPU 91 from the steering angle sensor 11. The steering angle data stored in the steering angle memory 93f is updated to data newly acquired by the CPU 91 from the steering angle sensor 11 every time the image transmission process (see FIG. 6) is executed. Then, the updated value of the steering angle memory 93f is referred to by the CPU 91 in order to determine a range to be deleted from the original image data in the corrected image data in the image transmission process (see FIG. 6).

  The reception frame rate memory 93g is a memory for storing the frame rate of the moving image displayed on the display 202 calculated by the operation side control device 201 of the operation center 200. Although details will be described later, the frame rate of the moving image displayed on the display 202 is calculated by the operation center 200 and transmitted from the operation center 200 to the vehicle 1. The value of the reception frame rate memory 93g is updated to a newly received value by an interrupt process (not shown) of the CPU 91 that occurs each time the frame rate is received by the communication device 14. Further, the value of the reception frame rate memory 93g is referred to by the CPU 91 in order to acquire the value of the view angle map memory 92b in the image transmission process (see FIG. 6).

  Next, the operation center 200 will be described. The operation side control device 201 of the operation center 200 includes a CPU 291, a ROM 292, and a RAM 293, which are connected to the input / output port 295 via the bus line 294. The input / output port 295 is connected to a display 202, a steering device 203, an accelerator pedal device 204, a brake pedal device 205, a communication device 206, other input / output devices 299, and the like.

  The CPU 291 generates a control command for controlling the vehicle 1 based on various information transmitted from the steering device 203, the accelerator pedal device 204, the brake pedal device 205, the communication device 206, and the like connected to the input / output port 295. Or an arithmetic device that controls the display 202.

  The ROM 292 is a non-rewritable nonvolatile memory for storing a control program executed by the CPU 291 and fixed value data. The ROM 292 is provided with a program memory 292a.

  The program memory 292a is an area on the ROM 292 in which various programs executed by the CPU 291 are stored. Each program for causing the CPU 291 to execute various processes such as an image display process shown in the flowchart of FIG. 10 described later is stored in the program memory 292a. The CPU 291 outputs image data (corrected image data) received by the communication device 206 to the display 202 by executing image display processing (see FIG. 10) according to the program stored in the program memory 292a.

  The RAM 293 is a rewritable volatile memory, and is a memory for temporarily storing various data when a program executed by the CPU 291 is executed. The RAM 293 is provided with at least a reception image memory 293a, a reception time memory 293b, and a previous time memory 293c.

  The received image memory 293a is a memory that temporarily stores a frame image of a moving image displayed on the display 202 as image data. The image data stored in the received image memory 293a is image data (corrected image data) generated by the vehicle side control device 100 of the vehicle 1. The content of the received image memory 293a includes an interrupt process (not shown) that occurs every time the image data (corrected image data) and the time stamp of the image data (corrected image data) are received by the communication device 206. ). In this interrupt process, newly received image data (corrected image data) is acquired (transferred) from the communication device 206, and the content of the received image memory 293a is overwritten with the image data (corrected image data). To do. In the image display process (see FIG. 10), when the image data (corrected image data) stored in the received image memory 293a is updated, the CPU 291 displays the image data (corrected image data) on the display 202. Output to. Therefore, every time new image data (corrected image data) is received from the vehicle 1, the frame image displayed on the display 202 is updated, whereby a moving image can be displayed on the display 202.

  The reception time memory 293b is a memory used for calculating a frame rate of a moving image displayed on the display 202, and stores a time stamp of image data (corrected image data) stored in the reception image memory 293a. It is. The contents of the reception time memory 293b are updated together with the reception image memory 293a by an interrupt process (not shown). In this interrupt process, the CPU 291 acquires a newly received time stamp from the communication device 206, and overwrites the received time memory 293b with the time stamp. The value in the reception time memory 293b is referred to by the CPU 291 in the image display process (see FIG. 10). Although details will be described later, the frame rate of the moving image displayed on the display 202 is calculated from the value of the reception time memory 293b and the value of the previous time memory 293c referred to at this time.

  The previous time memory 293c is a memory used for calculating the frame rate of the moving image displayed on the display 202, and stores the time stamp of the immediately previous image data (corrected image data) output to the display 202. It is memory. The previous time memory 293c is updated by writing the value of the reception time memory 293b into the previous time memory 293c at the timing when the image data (corrected image data) of the reception image memory 293a is output to the display 202. Further, the CPU 291 refers to the value of the previous time memory 293c and the value of the reception time memory 293b, so that the generation date and time of the frame image displayed on the display 202 (from the camera 12 in the vehicle side control device 100). Two time stamps indicating the date and time when image data was acquired can be obtained. In an image display process (see FIG. 10) described later, the difference between the two time stamps is obtained, and the reciprocal of the difference is calculated as the frame rate of the moving image displayed on the display 202. The calculated frame rate is output to the communication device 206 by the CPU 291. Thereby, the frame rate of the moving image displayed on the display 202 is transmitted from the communication device 206 to the vehicle 1.

  Next, image transmission processing executed by the CPU 91 in the vehicle control device 100 will be described with reference to the flowcharts and explanatory diagrams of FIGS. FIG. 6 is a flowchart showing image transmission processing. This image transmission process is a process performed when a signal indicating the start of remote operation transmitted from the operation center 200 is received by the CPU 91 via the communication device 14 and is captured by the camera 12 of the vehicle 1. In this process, image data (corrected image data) indicating a frame image of a moving image to be displayed on the display 202 is generated from the image, and the generated image data is output to the communication device 14.

  When the image transmission process is executed by the CPU 91, the CPU 91 first determines whether or not the transmission permission flag 93c is turned on based on an interrupt process (not shown) (S11). When the transmission permission flag 93c is turned on (S11: Yes), the transmission permission flag 93c is turned off (S12), and image data indicating an image captured by the camera 12 is transferred from the camera 12 to the image data memory 93a. And stored in the image data memory 93a (S13).

  Next, the CPU 91 obtains the date and time information output by the timer 15, thereby obtaining a time stamp indicating the date and time when the image data obtained in the process of S1 was generated, and stores it in the time memory 93d ( S14). The vehicle speed is calculated from the output of the speed sensor 10, the calculated vehicle speed is stored in the vehicle speed memory 93e (S15), the steering angle of the vehicle 1 is acquired from the steering angle sensor 11, and the steering angle is stored in the steering angle memory 93f. (S16).

  After execution of S16, the corrected image data generation process (S17) is executed, and image data indicating a frame image of a moving image to be displayed on the display 202 based on the image data stored in the image data memory by the process of S13. (Corrected image data) is acquired and stored in the transmission image buffer 93b.

  Here, with reference to FIG. 7, the corrected image data generation process (S17) performed by CPU91 of the vehicle side control apparatus 100 is demonstrated.

  In this corrected image data generation process, information related to the section in which the vehicle 1 travels is acquired from the navigation device 13 (S31), and the travel section in which the vehicle 1 is traveling is determined based on the acquired information (S32).

  When it is determined that the travel section of the vehicle 1 is “urban area” by the process of S32 (S32: “urban area”), the CPU 91 refers to the view angle map memory 92b and generates the display generated in this process. The angle of view of the frame image displayed on 202 is acquired (S33). That is, in the view angle map stored in the view angle map memory 92b, the vehicle speed stored in the vehicle speed memory 93e and the frame rate stored in the reception frame rate memory 93g when the traveling section is “city”. Get the angle of view corresponding to.

  Next, from the vehicle speed stored in the vehicle speed memory 93e and the steering angle of the vehicle 1 stored in the steering angle memory 93f, the optical axis of the camera 12 (in this embodiment, the y-axis), and the camera 12 An angle φs (horizontal central position angle φs) formed by a line connecting the gazing points Q set on the road surface of the predicted route on which the vehicle 1 is predicted to travel is obtained (S34).

  Here, an example of a method of calculating the horizontal center position angle φs by the process of S34 will be described with reference to FIG. FIG. 8 is an explanatory diagram for explaining a method of calculating the horizontal center position angle φ from the gazing point on the predicted route. Here, the center axis in the front-rear direction of the vehicle 1 will be described as y-axis, and the rear wheel axis of the vehicle 1 will be described as x-axis.

A formula that gives the turning curvature ρ (1 / m) of the vehicle from the vehicle speed (V) with respect to the vehicle 1 and the steering angle (δ) is generally well known and can be calculated from the following formula.
ρ = δ / HN (1 + AV ² ) (1)
In Expression (1), H is the wheel base of the vehicle 1 and N is the steering gear ratio. A is a positive constant called a stability factor defined for the vehicle 1.

From this equation (1), the turning radius R is determined by the following equation.
R = 1 / ρ (2)
Therefore, as shown in FIG. 8, the predicted route of the vehicle 1 is a point separated from the vehicle 1 by a distance R in the vertical direction (x-axis direction) in the front-rear direction (y-axis direction) of the vehicle 1 (in the case of FIG. 8). Is predicted as an arc of radius R centered on a point at a distance R on the left side of the vehicle 1.

Here, when the operator actually gets on the vehicle 1 and performs an operation, the operator of the vehicle 1 sets a point at a position farther away from the vehicle 1 as the vehicle speed increases, and sets the target. The vehicle 1 is operated while checking the surroundings. Therefore, the position of the gazing point Q is determined at a position away from the reference point of the vehicle 1 (the intersection of the x-axis and the y-axis) by a distance L of the following equation that is proportional to the vehicle speed V along the predicted route.
L = aV + b (a and b are positive constants) (3)
Here, using the sector central angle θ (θ = L / R) obtained when the radius is R and the arc is L, the position coordinate of the gazing point Q is Q (−R sin θ, R (1−cos θ)). Can be obtained as

When the position P of the camera 12 is given as P (0, p) in xy coordinates, the horizontal center position angle φs is given by the following expression from the gazing point Q and the coordinates of the position P of the camera 12.
φs = tan ⁻¹ {Rsin θ / (R−R cos θ−p)} (4)
The center position angle φs indicates the center position in the horizontal direction of the frame image indicated by the corrected image data in the corrected image data generated in the subsequent process of S35.

  Returning to FIG. 7, the description will be continued. In the process of S35, the data of the region located at the left and right outer edges (edges) of the frame image indicated by the image data is deleted from the image data in the image data memory 93a to generate corrected image data (S35). Here, if the horizontal angle of view acquired from the angle-of-view map memory 92b by the processing of S33 is α, φs−α / 2 ≦ φ in the frame image indicated by the image data from the image data in the image data memory 93a. Data of each pixel arranged on the wide angle side (outside) from the range of ≦ φs + α / 2 (φ is a horizontal field angle) is deleted. Then, the remaining deleted data is generated as corrected image data. As a result, the data amount of the corrected image data becomes a size according to the angle of view acquired in the process of S33.

  Then, the corrected image data generated by the process of S35 is stored in the transmission image buffer 93b (S36), and the corrected image data generation process ends.

  Returning to the process of S32, the description will be continued. In the process of S32, when it is determined that the travel section of the vehicle 1 is “tunnel” (S32: “tunnel”), the CPU 91 determines that the travel section is in the view angle map stored in the view angle map memory 92b. In the case of “tunnel”, an angle of view corresponding to the vehicle speed stored in the vehicle speed memory 93e and the frame rate stored in the reception frame rate memory 93g is acquired (S37).

  Next, from the vehicle speed stored in the vehicle speed memory 93e and the steering angle of the vehicle 1 stored in the steering angle memory 93f, the vehicle speed stored in the vehicle speed memory 93e and the vehicle 1 stored in the steering angle memory 93f. From the steering angle, there is a line connecting the optical axis of the camera 12 (y axis in this embodiment) and the gazing point Q set from the camera 12 on the road surface of the predicted route where the vehicle 1 is predicted to travel. The formed angle is obtained (S38). When the traveling section of the vehicle 1 is a “tunnel”, only the pixels located at the left and right outer edges (edges) of the image are obtained from the image data in the image data memory 93 by the process of S39 executed after S38. Instead, the pixels located at the upper and lower outer edges (edges) are also deleted. In the present embodiment, the horizontal center position angle φs, which serves as a reference when cutting pixels located on the left and right outer edges (edges), is obtained in the same manner as in the above-described processing of S34. On the other hand, the vertical center position angle ωs used as a reference when cutting pixels located on the upper and lower outer edges (edges) is a predetermined fixed value (for example, “0”). As for the vertical center position angle ωs, the vertical angle formed by the optical axis of the camera 12 and the line connecting the camera 12 and the gazing point Q described above is obtained, and the angle is adopted as the center position angle ωs. May be.

  Next, from the image data in the image data memory 93, the data of the area located at the upper, lower, left and right outer edges (edges) of the frame image indicated by the image data is deleted to generate corrected image data (S39). Here, assuming that the horizontal field angle acquired from the field angle map memory 92b is α and the vertical field angle is β, from the image data of the image data memory 93, φs−α / 2 ≦ in the frame image indicated by the image data. Delete data of each pixel arranged on the wide angle side (outside) from the range of φ ≦ φs + α / 2, ωs−β / 2 ≦ ω ≦ ωs + β / 2 (φ is the horizontal field angle, and ω is the vertical field angle) . Then, the remaining deleted data is generated as corrected image data. Thereby, the data amount of the corrected image data becomes a size according to the angle of view acquired in the process of S37.

  Next, from the image data in the image data memory 93, the data of each pixel arranged outside the range of the view angle obtained from the view angle map memory 92b, centered on the reference view angle calculated in S38. The data is deleted and the remaining data is generated as corrected image data (S39). The generated corrected image data is stored in the transmission image buffer 93b (S36), and the corrected image data generation process ends.

  On the other hand, in the process of S32, when it is determined that the travel section of the vehicle 1 is another section (S32: Other), the CPU 91 stores the image data in the image data memory 93b in the transmission image buffer 93c. Then, the corrected image data generation process ends.

  Returning to FIG. 6, the description of the image transmission process will be continued.

  When image data (corrected image data) is stored in the transmission image buffer 93b by the process of S17, the CPU 91 stores the image data (corrected image data) stored in the transmission image buffer 93b and the time memory 93d. The current time stamp is output to the communication device 14 (S18). Thereby, the image data (corrected image data) and the time stamp generated by the processes of S11 to S17 are transmitted to the operation center 200 via the communication device 14.

  Next, the minimum frame rate corresponding to the vehicle speed of the vehicle 1 is acquired in the minimum frame rate memory 92c (S19), and the value stored in the reception frame rate memory 93g is larger than the acquired minimum frame rate. Whether it is a value or not is discriminated (S20).

  In the process of S20, when it is determined that the value stored in the reception frame rate memory 93g is larger than the minimum frame rate (S20: Yes), the process returns to S11 as it is. On the other hand, in the process of S20, when it is determined that the value stored in the reception frame rate memory 93g is equal to or lower than the minimum frame rate (S20: No), a deceleration process is executed, and the vehicle commanding the wheel drive device 3 After reducing the target speed value (S21), the process returns to S11. As described above, when the value of the reception frame rate memory (that is, the frame rate of the moving image displayed on the display 202) is lower than the minimum frame rate that becomes a large value according to the vehicle speed (S20: No). ) Since the target value of the vehicle speed is lowered (S21) and the vehicle 1 is decelerated, the speed sensation of the operator can be brought close to the actual vehicle speed.

  Returning to the process of S11, the description will be continued. If it is determined by the processing of S11 that the transmission permission flag 93c is not turned on (that is, it is in an off state) (S11: No), the processing of S11 is repeated until the transmission permission flag 93c is turned on. Thus, the processing after S12 is made to wait.

  As described above, after returning to the process of S11, the processes of S11 to S21 are repeated. By repeating the process, the image data (corrected image data) generated by the corrected image data generation process and the time stamp of the image data are transmitted to the operation center 200 every time. Thereby, the moving image comprised from the frame image which image data (corrected image data) shows can be displayed on the display 202 of the operation center 200. FIG.

  Here, with reference to FIG. 9, the moving image comprised by the frame image produced | generated by the correction image data production | generation process is demonstrated. FIG. 9A is a schematic diagram showing the frame image A generated by the corrected image data generation process when the vehicle 1 travels in the “other” travel section, and FIG. FIG. 9C is a schematic diagram showing the frame image B generated by the corrected image data generation process when the vehicle 1 travels in the “city”, and FIG. 9C illustrates the corrected image when the vehicle 1 travels in the “tunnel”. It is a schematic diagram which shows the frame image C produced | generated by a data production | generation process.

  The frame image A shown in FIG. 9A is a frame image indicated by the image data as it is of the frame image data constituting the moving image captured by the camera 12 when the vehicle 1 travels in the “other” travel section. (Refer to S40 in FIG. 7). That is, the frame image A is the same as the moving image frame image described with reference to FIG. In the “other” traveling section, that is, an opened section where there is no conspicuous structure or building around the vehicle 1, a moving image composed of the frame image A having the widest display range (angle of view) is displayed on the display 202. .

  A frame image B shown in FIG. 9B is a frame image generated when the vehicle 1 travels in a “city” traveling section. In the frame image B, pixels arranged in the vicinity of the left and right edges E2 and E4 are deleted from the frame image (frame image A) of the moving image captured by the camera 12. Specifically, the horizontal angle of view acquired from the angle-of-view map memory 92b centered on the horizontal angle of the gazing point Q on the predicted path of the vehicle 1 with respect to the optical axis of the camera 12 (horizontal center position angle φs). Pixels arranged between the edges E5 and E6 of the area secured in the range and the left and right edges E2 and E4 are deleted (see S33 to S35 in FIG. 7). Since the area from which the pixels have been deleted cannot be displayed in the frame image B, the area from which the pixels near the left and right edges E2 and E4 have been deleted is not displayed in the moving image formed by the frame image B.

  As described above with reference to FIG. 2B, in the moving image captured by the camera 12, the road surface and the flow of the object are faster as the area is located at the edges E1 to E4. However, in the moving image composed of the frame image B, the areas from the edges E2 and E4 to the edges E5 and E6 with the fastest road surface and object flow are deleted, and the camera 12 captures only the area to be deleted. Compared with the moving image, the flow of the road surface or the like of the entire moving image becomes slower. Therefore, by displaying the moving image composed of the frame image B on the display 202, it is possible to give the operator a slower speed sensation than when the moving image captured by the camera 12 is displayed as it is.

  Here, in the “urban area”, there are buildings such as buildings adjacent to the traveling path of the vehicle 1 and side walls provided along the traveling path. The state of flowing through the side of the vehicle 1 may be imaged. However, when traveling in an “urban area”, a moving image in which areas from the left and right edges E2, E4 to E5, E6 are deleted from the moving image captured by the camera 12 is displayed on the display 202. Thereby, in the moving image, the action of increasing the speed sensation of the operator by the flow of the object on the side of the vehicle 1 can be canceled by slowing the flow of the road surface, the object, and the like of the entire moving image. Moreover, since the side object of the vehicle 1 that may exist in the “city” does not display the fastest flow around the left and right edges E2 and E4, the side of the vehicle 1 displayed in the moving image of the display 202 is displayed. It is possible to reliably suppress the speed sensation of the operator from being accelerated by the flowing object.

  A frame image C shown in FIG. 9C is a frame image generated when the vehicle 1 travels in a “tunnel” travel section. In the frame image C, pixels arranged near the top, bottom, left, and right edges E1 to E4 with respect to the frame image (frame image A) of the moving image captured by the camera 12 are deleted. More specifically, a region between edges E7 and E8 of the area secured in the range of the vertical angle of view acquired from the angle-of-view map memory 92b around the predetermined center position angle ωs and the upper and lower edges E1 and E3. The arranged pixels and the pixels arranged between the above-mentioned edges E5 and E6 and the left and right edges E2 and E4 are deleted (see S37 to S39 in FIG. 7). As a result, in the moving image composed of the frame image C, the area where the pixels near the top, bottom, left, and right edges E1 to E4 are deleted is not displayed. For this reason, in the moving image composed of the frame image C, the areas from the edges E1 to E4 to the edges E5 to E8 where the road surface and the flow of the object are the fastest are deleted. Compared with the captured moving image, the flow of the road surface or the like of the entire moving image is slowed down. Therefore, by displaying the moving image composed of the frame image B on the display 202, it is possible to give the operator a slower speed sensation than when the moving image captured by the camera 12 is displayed as it is.

  Here, in the “tunnel”, a ceiling and a wall exist above the vehicle 1 and on the left and right sides. Therefore, in a moving image captured by the camera 12, these objects (ceiling and wall) are located above the vehicle 1. The state of flowing sideways is imaged. However, when traveling in a “tunnel”, a moving image composed of a frame image C in which the areas from the top and bottom edges E1 to E4 to the edges E5 to E8 are deleted from the moving image captured by the camera 12 is displayed. It is displayed on the display 202. For this reason, in the moving image, the action of increasing the speed sensation of the operator by the flow of the road surface and objects (ceiling and wall) can be canceled by slowing the flow of the road surface and objects in the entire moving image.

  In such a moving image, the action of increasing the speed sensation of the operator due to the flow of the wall existing on the side of the vehicle 1 is suppressed by not displaying the state in which the wall flows fastest in the vicinity of the left and right edges E2 and E4. The action of accelerating the speed sensation of the operator by the flow of the ceiling above the vehicle 1 and the flow of the road surface below the vehicle 1 is shown to flow fastest near the upper edge E1 and the lower edge E3, respectively. Suppress it by not letting it. For this reason, it is possible to reliably prevent the speed sensation of the operator from being accelerated due to the flow of the road surface and objects (ceiling and wall) of the vehicle 1 existing in the “tunnel”.

  Thus, according to the vehicle 1, based on the information acquired from the navigation device 13, the vehicle 1 travels in a travel section in which the presence of an object is assumed around the vehicle 1 (that is, “urban area”, “tunnel”). In this case, a moving image obtained by deleting pixels near the edges E1 to E4 from the moving image acquired from the camera 12 is displayed on the display 202 to slow down the road surface and the flow of objects in the entire moving image. For this reason, it is possible to suppress the speed sensation of the operator from being accelerated by displaying the flow of the object existing in the travel section of the vehicle 1 in the moving image displayed on the display 202.

  Further, according to the vehicle 1, an object is present from a moving image acquired from the camera 12 in accordance with the positional relationship between the vehicle 1 and surrounding objects assumed based on information acquired from the navigation device 13. Then, since the moving image from which pixels in the vicinity of the edges E1 to E4 in the assumed direction are deleted is displayed on the display 202, an object existing around the vehicle 1 flows fastest near the edges E1 to E4 of the moving image. Can be reliably prevented from being accelerated by the moving image displayed on the display 202.

  In the “tunnel”, it is assumed that there are objects continuously around the vehicle 1, and it is assumed that the moving image captured by the camera 12 accounts for a large proportion of the object that flows as the vehicle 1 travels. The On the other hand, in the “urban area”, it is assumed that objects are scattered around the vehicle 1. Therefore, in the moving image picked up by the camera 12, the proportion of the object flowing along with the traveling of the vehicle 1 is “tunnel”. It is assumed that it is less than In the moving image picked up by the camera 12, the more the proportion of the object that flows as the vehicle 1 travels, the more objects that work to speed up the operator's speed sense. The speed sensation may be further accelerated.

  Here, according to the angle-of-view map (see FIG. 4) stored in the angle-of-view map memory 93b, if the vehicle speed and the frame rate of the moving image displayed on the display 202 are the same, the “city” In the “tunnel”, a frame image having a smaller angle of view is generated by the corrected image data generation process (see FIG. 7). Therefore, in the corrected image data generation process (see FIG. 7), the more the traveling section of the vehicle 1 is a section in which the object flowing with the advance of the vehicle 1 is more easily captured by the camera 12, the edges from the edges E1 to E4 are further inward. A frame image from which pixels have been deleted is generated as a frame image of a moving image displayed on the display 202. This slows down the flow of the entire moving image displayed on the display 202 as the vehicle travels in a traveling section where the proportion of objects in the moving image captured by the camera 12 is large, regardless of the traveling section of the vehicle 1. The operator can be given a sense of speed according to the actual vehicle speed.

  In addition, as described above, if the vehicle speed is high with respect to the angle of view of the moving image and the road surface or the like flows faster in the area near the edge of the moving image than the vehicle speed, There is a tendency that the speed sensation given to the operator who has seen the image becomes faster than the actual vehicle speed. On the other hand, according to the view angle map (see FIG. 4) stored in the view angle map memory 93b, when the traveling section is “urban area” or “tunnel”, the smaller the vehicle speed, the smaller the view angle. This frame image is generated by the modified image data generation process (see FIG. 7). For this reason, in the corrected image data generation process, as the vehicle speed increases, a frame image in which pixels are deleted from the edges E1 to E4 to the inner side is generated as a frame image of a moving image displayed on the display 202. . Therefore, in the moving image to be displayed on the display 202, as the vehicle speed increases, the range of the area from which the pixels are deleted is changed in order of the flow of the road surface from the edges E1 to E4 having the fastest flow of the road surface (i.e. Since the image is enlarged (to the inner side of the moving image), the flow speed of the entire moving image displayed on the display 202 can be suppressed. As a result, the speed sensation of the operator can be brought close to the actual vehicle speed regardless of the vehicle speed.

  When the frame rate of the moving image displayed on the display 202 is low, the speed sensation of the operator who operates the vehicle 1 while viewing the moving image may be faster than the actual vehicle speed.

  On the other hand, according to the image map (see FIG. 4) stored in the image map memory 92b, the moving image displayed on the display 202 is displayed when the traveling section of the vehicle 1 is “urban area” or “tunnel”. Reference when the frame image generated by the modified image data generation process (see FIG. 7) is greater than or equal to the thresholds Nl, Nm, and Nh when the frame rate is less than the thresholds Nl, Nm, and Nh according to the vehicle speed It is shown that the image is generated with an angle of view smaller than the angle of view. For this reason, the frame image generated when the frame rate is less than the threshold value Nl, Nm, Nh corresponding to the vehicle speed is the reference field angle generated when the frame rate is equal to or greater than the threshold value Nl, Nm, Nh. In this frame image, pixels in the vicinity of the edges E1 to E4 are deleted. Therefore, in the moving image displayed on the display 202, when the frame rate is less than the threshold values Nl, Nm, and Nh, the moving image is displayed at the reference angle of view when the frame rate is equal to or greater than the threshold values Nl, Nm, and Nh. In addition, since the area near the edges E1 to E4 where the flow on the road surface is fast is not displayed in the moving image, the flow speed of the entire moving image displayed on the display 202 can be suppressed. Therefore, when the frame rate of the moving image displayed on the display 202 is low, it is possible to suppress the speed sensation of the operator from becoming faster than the actual vehicle speed.

  Further, as the frame rate of the moving image displayed on the display 202 decreases, the speed sensation of the operator tends to be faster than the actual vehicle speed.

  On the other hand, as shown in FIG. 4, the view angle map is proportional to the decrease in the frame rate when the frame rate of the moving image displayed on the display 202 falls below the thresholds Nl, Nm, and Nh corresponding to the vehicle speed. Thus, the view angle acquired from the view angle map memory 92b is reduced. When the frame rate of the moving image displayed on the display 202 is lower than the threshold values Nl, Nm, and Nh corresponding to the vehicle speed, the pixels are deleted from the edges E1 to E4 to the inner side as the frame rate is lower. The frame image is generated by the corrected image data generation process (see FIG. 7). Therefore, in the moving image to be displayed on the display 202, the slower the frame rate, the smaller the range of the area where the pixels are deleted from the edges E1 to E4 where the flow on the road surface and the like is the fastest (ie, the flow on the road surface is the fastest). Therefore, as the frame rate is lower, the flow speed of the entire moving image displayed on the display 202 can be suppressed. Thereby, irrespective of the frame rate of the moving image displayed on the display 202, the speed sensation of the operator can be brought close to the actual vehicle speed.

  In addition, as the vehicle speed increases, the tendency of an operator who operates the vehicle 1 to see the moving image displayed on the display 202 becomes faster than the actual vehicle speed becomes more prominent. Specifically, as the vehicle speed increases, the operator feels a speed sensation faster than the actual vehicle speed due to a slight decrease in the frame rate in the moving image displayed on the display 202.

  On the other hand, as shown in FIG. 4, in the angle-of-view map, the frame rate thresholds Nl, Nm, and Nh become larger as the vehicle speed increases. For this reason, when the frame rate of the moving image displayed on the display 202 decreases, the CPU 91 displays a smaller angle of view than the reference angle of view in the image transmission process of the vehicle-side control device 100 as the vehicle speed increases. It becomes easy to obtain from the corner map memory 92b. Therefore, in a situation where the vehicle speed increases and the operator's sense of speed is likely to increase as the frame rate decreases, the angle of view of the moving image displayed on the display 202 is actively reduced, and the operator The speed sensation can be suppressed. Therefore, it is possible to give the operator a sense of speed close to the actual vehicle speed from the moving image displayed on the display 202 regardless of the vehicle speed.

  Further, in the corrected image data generation process (see FIG. 7), the center of the region where the angle of view is secured is set to the gazing point Q on the predicted route of the vehicle 1 where the travel of the vehicle 1 is predicted. Therefore, in the moving image displayed on the display 202, the region that gives the operator a sense of speed faster than the actual vehicle speed is not viewed by the operator, but the traveling direction of the vehicle is It can be surely visually recognized.

  In addition, in the display 202 on which such a moving image is displayed, the gazing point Q, which is the center of the region visually recognized by the operator, is a point on the predicted route of the vehicle 1, so You can make it.

  Moreover, when the operator actually gets on the vehicle 1 and operates the vehicle 1, the operator sets a travel target at a point farther away from the vehicle 1 as the vehicle speed increases, Operate while looking at the surroundings. Here, the gazing point Q is a point predicted on the predicted route of the vehicle 1 at a position away from the vehicle 1 by a distance proportional to the vehicle speed. Therefore, the operation of the vehicle 1 can be performed while directing the attention of the operator who operates the vehicle 1 from the operation center 200 to the same area as when the operator actually gets on the vehicle 1.

  As described above, the communication speed of the transmission path for transmitting the image data (corrected image data) from the vehicle 1 to the operation center 200 is low, and communication necessary for transmitting one frame image from the vehicle 1 to the operation center 200. If the time becomes longer, the frame rate of the moving image displayed on the display 202 of the operation center 200 is reduced accordingly.

  On the other hand, in the present embodiment, in the corrected image data generation process (see FIG. 7), the corrected image data is acquired from the view angle map memory 92b from the image data of one frame image captured by the camera 12. It is generated by deleting data of pixels located on the wide-angle side from the angle of view. When the frame rate of the moving image displayed on the display 202 is less than the threshold value Nl, Nm, Nh corresponding to the vehicle speed, the frame rate is set to the threshold value Nl, Nm in the modified image data generation process (see FIG. 7). , Nh or more, the angle of view smaller than the reference angle of view when it is equal to or greater than Nh is acquired from the angle-of-view map memory 92b. Therefore, the data amount per frame image indicated by the corrected image data when the frame rate is less than the threshold values Nl, Nm, and Nh corresponding to the vehicle speed is such that the frame rate is equal to or greater than the threshold values Nl, Nm, and Nh. Smaller than when. For this reason, when the communication speed in the transmission path is slow and the frame rate is less than the threshold values Nl, Nm, and Nh, one frame image is used as compared with the case where the frame rate is equal to or greater than the threshold values Nl, Nm, and Nh. Since image data with a reduced per-data amount is generated and transmitted from the vehicle 1 to the operation center 200, it is possible to suppress a reduction in the number of frame images per time transmitted from the vehicle 1 to the operation center 200. Accordingly, it is possible to prevent a moving image having a reduced frame rate from being displayed on the display 202 of the operation center 200 based on a reduction in communication speed. Therefore, by displaying a moving image with a low frame rate on the display 202, it is possible to suppress the speed sensation of the operator from becoming higher than the actual vehicle speed.

  When the frame rate is less than the threshold values Nl, Nm, and Nh, the angle of view acquired from the angle-of-view map memory 92b becomes smaller as the frame rate is lower, so that the corrected image data depends on the angle of view. The data amount of the corrected image data generated by the generation process is also reduced. For this reason, the fall of the frame rate of the moving image displayed on the display 202 based on the fall of the communication speed of the transmission path which transmits image data (correction image data) can be suppressed more suitably.

  Next, image display processing executed by the CPU 291 of the operation side control device 201 provided in the operation center 200 will be described with reference to the flowchart of FIG. FIG. 10 is a flowchart showing image display processing. This image display process is a process executed when the vehicle 1 is remotely operated in the operation center 200. The image data (corrected image data) received by the communication device 206 is output to the display 202 every time it is received. This is a process for displaying a moving image around the vehicle 1 on the display 202.

  In this image display process, first, the time stamps stored in the reception time memory 293b and the previous time memory 293c are respectively compared to determine whether the time stamp in the reception time memory 293b indicates a later date and time. (S61). Here, the value of the reception time memory 293b is an interrupt process (not shown) that occurs at the timing when the image data (corrected image data) and the time stamp transmitted from the vehicle control device 100 are received by the communication device 206. ), The time stamp received by the communication device 206 is written by the CPU 291. The value of the previous time memory 293c is the value written in the reception time memory 293b in the later-described processing of S65 described later.

  Therefore, if new image data (corrected image data) and a time stamp have not been received by the communication device 206 after the previous processing of S65, no interrupt processing occurs and the reception time memory 293b The time stamp and the time stamp of the previous time memory 293c indicate the same date and time. For this reason, in the process of S61, it is not determined that the time stamp of the reception time memory 293b indicates a later date and time (S61: No). In such a case, the process of S61 is repeatedly executed until new image data (corrected image data) and a time stamp are received by the communication device 206 and the value of the reception time memory 293b is updated by the interrupt process.

  On the other hand, if new image data (corrected image data) and a time stamp are received by the communication device 206, the time stamp in the reception time memory 293b is more time-sensitive than the time stamp in the previous time memory 293c in the process of S61. Is determined to indicate a later date and time (S61: Yes), and the processing of S62 to S65 is executed.

  In the process of S62, the generation date and time of the image data (corrected image data) stored in the reception image memory 293a indicated by the time stamp of the reception time memory 293b and the time stamp of the previous time memory 293c indicate on the display 202 immediately before. From the difference between the output date and time of the output image data (corrected image data), the time interval at which the two image data are generated is calculated. Then, the reciprocal of the time interval is calculated as the frame rate of the moving image displayed on the display 202. The frame rate calculated by the process of S62 is output to the communication device 206 (S63). Thereby, the frame rate of the moving image displayed on the display 202 is transmitted from the operation side control device 201 to the vehicle side control device 100 via the communication device 206.

  Next, the CPU 291 outputs the image data (corrected image data) stored in the received image memory 293a to the display 202 (S64). Thereby, the frame image displayed on the display 202 is updated based on the image data.

  Next, the value stored in the previous time memory 293c is rewritten to the time stamp stored in the reception time memory 293b (S65), and the process returns to S61. Then, the processes from S61 to S65 are repeatedly executed. Thereby, the frame image displayed on the display 202 can be sequentially updated to display a moving image.

  Next, with reference to FIGS. 11 to 13, an embodiment in which the present invention is applied to the operation center 200 will be described as a second embodiment. In the first embodiment described above, the control device 100 of the vehicle 1 is caused to obtain a range (view angle) to be displayed on the display 202 of the operation center 200 among the moving images captured by the camera 12 of the vehicle 1, and the range. Image data (corrected image data) for displaying a moving image included in the operation center 200 is transmitted to the operation-side control device 201 of the operation center 200.

  In contrast, in the second embodiment, the vehicle-side control device 100 is configured to transmit image data based on a moving image captured by the camera 12 of the vehicle 1 to the operation-side control device 201 as it is, and The operation-side control device 201 obtains a display range (angle of view) of a moving image to be displayed on the display 202 based on the image data received from the vehicle control device 100, and displays a moving image on the display 202 within the obtained display range. It was configured to make it. Hereinafter, the same reference numerals are given to the same elements as those in the first embodiment, and illustration and description thereof are omitted.

  First, with reference to FIG. 11, the detailed structure of the vehicle 1 and the operation center 200 is demonstrated. FIG. 11 is a block diagram showing the electrical configuration of the vehicle 1 and the operation center 200.

  First, the vehicle 1 will be described. The electrical configuration of the vehicle 1 in the second embodiment is different from the first embodiment in the ROM 92 and the RAM 93 of the vehicle side control device 100.

  The ROM 92 is provided with a program memory 92a. The program memory 92a stores a program for executing image transmission processing, which is partly different from the processing in the first embodiment. Details will be described later with reference to FIG. 12, but the range (view angle) to be displayed on the display 202 of the operation center 200 among the images captured by the camera 12 is not obtained as in the first embodiment. A program for processing for transmitting image data indicating an image captured by the camera 12 to the operation-side control device 201 as it is is stored in the program memory 92a.

  In the first embodiment, an angle-of-view map memory 92b is provided in the ROM 92, and the angle of view of the moving image displayed on the display 202 of the operation center 200 in the vehicle side control device 100 is referred to the angle-of-view map memory 92b. Although the case where it calculates | requires was demonstrated, the angle-of-view map memory 92b is not provided in ROM92 of 2nd Embodiment. Although details will be described later, in the second embodiment, the operation-side control device 201 determines the angle of view of the moving image to be displayed on the display 202, so that it is stored in the ROM 92 of the vehicle-side control device 100 as in the first embodiment. This is because it is not necessary to provide the view angle map memory 92b.

  In the ROM 92 of the first embodiment, the minimum frame rate memory 92c is provided, and the vehicle-side control apparatus 100 determines that the frame rate of the moving image displayed on the display 202 is lower than the minimum frame rate. However, the ROM 92 of the second embodiment is not provided with the minimum frame rate memory 92c. Although details will be described later, in the second embodiment, the operation-side control device 201 compares the frame rate of the moving image displayed on the display 202 with a predetermined minimum frame rate to determine the vehicle 1 This is because the process for generating the deceleration command is performed, and therefore it is not necessary to provide the minimum frame rate memory 92c in the ROM 92 of the vehicle control device 100 as in the first embodiment.

  Similar to the first embodiment, the RAM 93 of the second embodiment includes an image data memory 93a, a transmission permission flag 93c, a time memory 93d, a vehicle speed memory 93e, and a steering angle memory 93f. On the other hand, the RAM 93 of the second embodiment is not provided with the transmission image buffer 93b and the reception frame rate memory 93g provided in the RAM 93 of the first embodiment.

  The reason why the transmission image buffer 93b is not provided in the RAM 93 of the second embodiment is as follows. That is, although details will be described later, in the image transmission process (see FIG. 12) of the second embodiment, the image data stored in the image data memory 93a is not corrected to the corrected image data as in the first embodiment. Then, it is transmitted to the operation side control device 201 as it is. For this reason, in the second embodiment, unlike the first embodiment, it is not necessary to provide the transmission image buffer 93b and temporarily store the corrected image data.

  The reason why the reception frame rate memory 93g is not provided in the RAM 93 of the second embodiment is as follows. That is, based on the frame rate of the moving image displayed on the display 202 of the operation center 200, the vehicle-side control device 100 determines the angle of view and the target value of the vehicle speed of the vehicle 1, but in the second embodiment. These are obtained by the operation side control device 201. Thus, in 2nd Embodiment, it is because the process based on the frame rate received from the operation side control apparatus 201 is not performed in the vehicle side control apparatus 100. FIG.

  Next, the operation center 200 will be described. In the electrical configuration of the operation center 200 in the second embodiment, the difference from the first embodiment is the ROM 292 and the RAM 293 of the operation side control device 201.

  The ROM 292 is provided with a program memory 292a, an angle-of-view map memory 392b, and a minimum frame rate memory 392c. The program memory 292a stores a program for executing an image display procedure, which is partly different from the first embodiment. Although details will be described later with reference to FIG. 13, when image data is received from the vehicle-side control device 100, a range (image) to be displayed on the display 202 of the operation center 200 among the images indicated by the received image data. A program for obtaining a corner) and displaying only an image included in the range on the display 202 is stored in the program memory 292a.

  The view angle map memory 392b is a memory for generating a frame image of a moving image that gives the operator a speed sensation according to the actual vehicle speed. In the first embodiment, the view angle map memory of the vehicle side control device 100 is used. An angle-of-view map similar to that stored in 92b is stored. This view angle map memory 392b is a range (view angle) that the CPU 291 displays on the display 202 in the frame image indicated by the image data received from the vehicle control device 100 in the image display process (see FIG. 13) described later. In accordance with information indicating the travel section of the vehicle 1 acquired by the navigation device 13 provided in the vehicle 1, the frame rate of the moving image displayed on the display 202, and the vehicle speed of the vehicle 1. Referenced. The angle of view map stored in the angle of view map memory 392b is the same as the angle of view map stored in the angle of view map memory 92b of the first embodiment (see FIG. 4). Omitted.

  The minimum frame rate memory 392c is a memory in which a minimum frame rate (minimum frame rate) to be secured in the moving image displayed on the display 202 is stored in association with each vehicle speed. The minimum frame rate memory 392c is used to set a vehicle speed command value to be transmitted to the vehicle 1 in accordance with the frame rate of the moving image displayed on the display 202 in an image display process (see FIG. 13) described later. The CPU 291 refers to it. The correspondence relationship between the vehicle speed and the minimum frame rate in the minimum frame rate memory 392c is the same as that of the minimum frame rate memory 92c of the first embodiment, and thus detailed description thereof is omitted (see FIG. 5).

  The RAM 293 is provided with a vehicle speed memory 393d, a steering angle memory 393e, a navigation information memory 393g, and a display image memory 393g in addition to the reception image memory 293a, the reception time memory 293b, and the previous time memory 293c.

  The vehicle speed memory 393d is a memory for temporarily storing speed data indicating the vehicle speed of the vehicle 1 received by the communication device 206. The value in the vehicle speed memory 393d is referred to by the CPU 291 in order to obtain the value in the view angle map memory 392b and the value in the minimum frame rate memory 392c in the image display process (see FIG. 13).

  The steering angle memory 393e is a memory for temporarily storing steering angle data indicating the steering angle of the vehicle 1 received by the communication device 206. The value of the steering angle memory 393e is referred to by the CPU 291 in order to calculate the predicted route of the vehicle 1 and the gazing point Q on the predicted route.

  The navigation information memory 393f is a memory for temporarily storing information indicating the travel section of the vehicle 1 acquired by the navigation device 13 of the vehicle 1 and received by the communication device 206. The navigation information memory 393f is referred to in order to acquire the value of the view angle map memory 392b in the image display process (see FIG. 13).

  In the second embodiment, the vehicle speed memory 393d, the steering angle memory 393e, and the navigation information memory 393f are interrupts that are generated when the communication device 206 receives speed data, steering angle data, and information indicating a travel section, respectively. Each process (not shown) is updated to one newly received by the communication device 206.

  The display image memory 393g is a memory for temporarily storing image data (corrected image data) of an image displayed on the display 202. The corrected image data is obtained by correcting the image data stored in the received image memory 293a according to the angle of view acquired by the CPU 291 from the angle of view map memory 292b. The display image memory 393g has two storage areas. The display image memory 393g stores image data based on the image data (corrected image data) transferred from the received image memory 93a in one storage area in an image display process (see FIG. 13) described later. At this time, image data (corrected image data) stored in advance in the other storage area is transferred to the display 202. As a result, the frame image indicated by the image data (corrected image data) is displayed on the display 202. In the image display process described later, the storage area to which the image data (corrected image data) transferred from the received image memory 93a is transferred and the storage area to which the image data is transferred to the display 202 are corrected. Each time the image data generation process (S81) is executed, the image data is alternately switched and used. In the description of the image display process of FIG. 13 to be described later, description of the flow of the process using these two storage areas provided in the display image memory 393g is omitted.

  Next, an image transmission process executed by the CPU 91 of the vehicle-side control device 100 mounted on the vehicle 1 will be described with reference to the flowchart of FIG. FIG. 12 is a flowchart showing image transmission processing. This image transmission process is performed when the vehicle 1 is remotely operated from the operation center 200, and the angle of view of the image captured by the camera 12 of the vehicle 1 is corrected as in the first embodiment. Without being performed, the image data is converted into image data every time and the image data is sequentially output to the communication device 14.

  Among the image transmission processes, the processes of S11 to S16 are executed in the same manner as the processes with the same step numbers as those processes in the first embodiment. The detailed explanation is omitted.

  In the image transmission processing of the second embodiment, the processing after S12 is waited until the transmission permission flag 93b is turned on based on the communication permission signal from the communication device 14 (S11: No). When the transmission permission flag 93b is turned on (S11: Yes), the image data is acquired from the camera 12, stored in the image data memory 93a (S13), and the date / time information acquired from the timer 15 is used as a time stamp as a time memory. The vehicle speed acquired from the speed sensor 10 is stored in the vehicle speed memory 93e (S15), and the steering angle acquired from the steering angle memory 93e is stored in the steering angle memory 93e (S16). The process of S71 is executed.

  In the process of S <b> 71, information indicating the travel section of the vehicle 1 is acquired from the navigation device 13. Then, along with the information indicating the travel section, the data stored in the image data memory 93a, the time memory 93d, the vehicle speed memory 93e, and the steering angle memory 93e by the processes of S13 to S16 (image data, date and time of generation of the image data). (Time stamp, vehicle speed, steering angle) are output to the communication device 14, respectively. As a result, various information input to the communication device 14 is transmitted from the communication device 14 to the operation-side control device 201.

  After executing the process of S72, the process returns to the process of S11, and the processes of S11 to S16, S71, and S72 are repeatedly executed. By repeating the process, the frame image captured by the camera 12, the time stamp indicating the generation date and time of the frame image, the vehicle speed of the vehicle 1, the steering angle of the vehicle 1, and the travel section of the vehicle 1 are illustrated. Information is sequentially transmitted from the vehicle side control device 100 to the operation side control device 201.

  Next, image display processing executed by the CPU 291 of the operation side control device 201 provided in the operation center 200 will be described with reference to the flowchart of FIG. FIG. 13 is a flowchart showing image display processing. This image display process is a process executed when the vehicle 1 is remotely operated in the operation center 200. When image data is received by the communication device 206, the image display process is displayed on the display 202 based on the image data. By generating image data (corrected image data) indicating a frame image of the moving image to be output and outputting the generated image data to the display 202, the angle of view is determined from the moving image around the vehicle 1 captured by the camera 12. This is a process for displaying on the display 202 what is partly or entirely cut out.

  Among these image display processes, the processes of S61, S62, and S65 are executed in the same manner as the processes with the same step numbers as those processes in the first embodiment. Will not be described in detail.

  In this image display process, the time stamps stored in the reception time memory 293b and the previous time memory 293c are compared to determine whether the time stamp in the reception time memory 293b indicates a later date and time ( S61). In the process of S61, if it is not determined that the time stamp of the reception time memory 293b indicates a later date and time (S61: No), new image data and a time stamp are received by the communication device 206, The process of S61 is repeatedly executed until the value of the reception time memory 293b is updated by the interrupt process executed by the CPU 291 at that timing.

  On the other hand, in the process of S61, when it is determined that the time stamp of the reception time memory 293b indicates a date and time later than the time stamp of the previous time memory 293c (S61: Yes), the process of S62 is performed. Transition.

  In the process of S62, a time interval at which image data of frame images continuously displayed on the display 202 is generated is calculated from the time stamp of the reception time memory 293b and the time stamp of the previous time memory 293c. Then, the reciprocal of the time interval is calculated as the frame rate of the moving image displayed on the display 202.

  Next, a corrected image data generation process is executed (S81). In this corrected image data generation process (S81), the same process as the corrected image data generation process (see FIG. 7) executed by the vehicle side control device 100 of the first embodiment is performed. Therefore, with respect to the description of the corrected image data generation process performed with reference to FIG. 7, “CPU 91” is “CPU 291”, “view angle map memory 92b” is “view angle map memory 392b”, and “image data memory”. 93a ”is“ received image memory 293a ”,“ vehicle speed memory 93e ”is“ vehicle speed memory 393d ”,“ steering angle memory 93e ”is“ steering angle memory 393e ”,“ navigation device 13 ”is“ navigation information memory 393f ”,“ The “transmission image buffer 93b” is replaced with “display image memory 393g” and “the frame rate stored in the reception frame rate memory 93g” is replaced with “the frame rate calculated by the process of S62”, and this modified image data generation process (S81) The description about is omitted.

  Next, the CPU 291 outputs the image data (corrected image data) stored in the display image memory 393g to the display 202 as a result of the corrected image data generation process (S81) (S82).

  Next, the value stored in the previous time memory 293c is rewritten to the time stamp stored in the reception time memory 293b (S65).

  After the execution of S65, the minimum frame rate corresponding to the vehicle speed of the vehicle 1 stored in the vehicle speed memory 393d is acquired in the minimum frame rate memory 392c (S83), and the processing of S62 is performed more than the acquired minimum frame rate. It is determined whether the calculated frame rate is a larger value (S84).

  If it is determined in the process of S84 that the frame rate calculated in the process of S62 is larger than the minimum frame rate (S84: Yes), the process returns to the process of S61 as it is. On the other hand, in the process of S84, when it is determined that the frame rate calculated in the process of S62 is equal to or lower than the minimum frame rate (S84: No), the deceleration command process is executed and stored in the vehicle speed memory 393d. A speed command with the vehicle speed reduced by a certain rate (for example, 5%) as a target value is generated from the vehicle speed and transmitted to the vehicle side control device 100 via the communication device 206 (S85). When the frame rate of the moving image displayed on the display 202 is low and the speed sensation of the operator is faster than the actual vehicle speed, the speed sensation of the operator is higher than the actual vehicle speed as the vehicle speed increases. Misalignment increases. Therefore, when the frame rate is below the minimum frame rate, the vehicle 1 is decelerated to bring the operator's sense of speed closer to the traveling speed of the vehicle 1. After executing the process of S85, the process returns to the process of S61, and the main image display process is executed from S61.

  As described above, in the second embodiment, the moving image around the vehicle imaged by the camera 12 of the vehicle 1 by the image transmission process (see FIG. 12) executed in the vehicle-side control device 100 is the moving image. The image data is converted into image data indicating a frame image of the image and transmitted from the vehicle 1 to the operation side control device 201. On the other hand, the operation-side control device 201 that has received the image data from the vehicle 1 displays the image indicated by the image data received from the vehicle 1 on the display 202 by the modified image data generation process (S81). Are cut out at an angle of view corresponding to the frame rate, the vehicle speed, and the environment in which the vehicle 1 travels, and the cut-out image is displayed on the display 202 as a frame image constituting the moving image.

  Here, the corrected image data generation process (S81) is the same process as that executed by the vehicle-side control device 100 of the first embodiment, and the parameters used in the corrected image data generation process (S81), An angle-of-view map that defines the angle of view of a moving image (frame image) to be displayed on the display 202 is the same as that in the first embodiment.

  Therefore, in the first embodiment, the processing for generating a moving image to be displayed on the display 202 of the operation center 200 from the moving image captured by the camera 12 of the vehicle 1 is performed in the vehicle 1. In the second embodiment, although the processing is different in the operation center 200, the second embodiment is performed if the frame rate of the moving image displayed on the display 202, the vehicle speed, and the environment in which the vehicle 1 travels are the same. Also in the embodiment, a moving image having the same display range (angle of view) as that in the first embodiment can be displayed on the display 202. For this reason, the operation center 200 of 2nd Embodiment can enjoy the effect mentioned above about the vehicle 1 of 1st Embodiment.

  However, the vehicle 1 according to the first embodiment has a display range (view angle) when the frame rate of the moving image displayed on the display 202 is low based on a decrease in the moving image transmission path from the vehicle 1 to the operation center 200. ) To generate image data in which the amount of data per frame image is lower than when the frame rate is sufficient (for example, exceeding the threshold values Nl, Nm, and Nh). Is transmitted from the vehicle 1 to the operation center 200. For this reason, in 1st Embodiment, even if communication speed fell, the effect that it can suppress that the frame rate of the moving image displayed on the display 202 can be enjoyed can be enjoyed. On the other hand, in the second embodiment, since the image data captured by the camera 12 is transmitted as it is from the vehicle 1 to the operation center 200, the image data transmitted from the vehicle 1 to the operation center 200 is per frame image. The amount of data is constant. For this reason, the effect that the vehicle 1 of the first embodiment enjoys cannot be achieved by the operation center 200 of the second embodiment.

  Next, with reference to FIGS. 14 to 16, an embodiment in which the present invention is applied to the vehicle 1 will be described as a third embodiment. In the first embodiment described above, an area in the moving image captured by the camera 12 of the vehicle 1 that makes the operator's speed sensation faster than the actual vehicle speed is larger than the angle of view defined in the angle-of-view map. It is configured to generate a moving image to be displayed on the display 202 of the operation center 200 at a remote location from the vehicle 1 by deleting the wide-angle side area as being on the wide-angle side (outside).

  On the other hand, in the third embodiment, the optical flow in the moving image captured by the camera 12 of the vehicle 1 is calculated, and the region where the calculated optical flow is larger than the size corresponding to the vehicle speed is the speed of the operator. The moving image to be displayed on the display 202 is generated by deleting from the moving image captured by the camera 12 assuming that the sensation is an area where the sensation is faster than the actual vehicle speed. Hereinafter, the same reference numerals are given to the same elements as those in the first embodiment, and illustration and description thereof are omitted.

  First, the detailed configuration of the vehicle 1 will be described with reference to FIG. FIG. 14 is a block diagram showing an electrical configuration of the vehicle 1.

  The electrical configuration of the vehicle 1 in the third embodiment is different from the first embodiment in the ROM 92 and the RAM 93 of the vehicle side control device 100.

  The ROM 92 is provided with a program memory 92a and a minimum frame rate memory 92c. The program memory 92a stores a program for executing modified image data generation processing, which is partly different from that of the first embodiment. The details will be described later with reference to FIG. 15, but an optical flow of a moving image captured by the camera 12 is calculated, and an area where an optical flow larger than the reference value is calculated is a moving image captured by the camera 12. A program memory 92a stores a program for processing for generating an image deleted from the image as a moving image to be displayed on the display 202 of the operation center 200 at a remote location.

  In this embodiment, since the display range of the moving image to be displayed on the display 202 is obtained based on the result of determining the optical flow size of the moving image captured by the camera 12, the moving image to be displayed on the display 202 is obtained. The angle-of-view map memory 92b according to the first embodiment that predefines the angle of view (display range) is not provided in the ROM 92 according to the third embodiment.

  The RAM 93 includes a previous image data memory 493a in addition to an image data memory 93a, a transmission image buffer 93b, a transmission permission flag 93c, a time memory 93d, a vehicle speed memory 93e, a steering angle memory 93f, and a reception frame rate memory 93g. Yes.

  The previous image data memory 493a temporarily stores image data one frame before the image data stored in the image data memory 93a, and calculates an optical flow of a moving image captured by the camera 12 together with the image data memory 93a. It is a memory used to The previous image data memory 493a is referred to by the CPU 91 when the corrected image data is generated based on the image data in the image data memory 93a in the corrected image data generation process (see FIG. 15) described later. At this time, the optical flow of the moving image is calculated from the image data stored in the previous image data memory 493a and the image data stored in the image data memory 93a. On the other hand, in the corrected image data generation process (see FIG. 15), the corrected image data is not generated based on the image data in the image data memory 93a, and the image data in the image data memory 93a is stored as it is in the transmission image buffer 93b. If not referenced.

  In the previous image data memory 493a, image data (corrected image data) of a moving image to be displayed on the display 202 of the operation center 200 based on the image data (image data acquired from the camera 12) in the image data memory 93a is stored in the transmission image buffer. After being stored, the image data stored in the image data memory 93a is stored. As a result, when the optical flow is calculated in the state where the image data acquired from the camera 12 is stored in the image data memory 93a next time (that is, when the modified image data generation process is newly performed), the previous image data memory 493a is calculated. In this state, image data one frame before the image data stored in the image data memory 93a is stored.

  The navigation device 13 of the present embodiment is configured to output the number of GPS satellites that receive radio waves from the output unit. If an object that blocks radio waves, such as a building, exists between the vehicle 1 and GPS hygiene, the vehicle 1 cannot receive radio waves from the GPS hygiene. Therefore, the smaller the number of GPS sanitations that the radio waves are received, the more objects such as buildings and walls exist in the vicinity of the vehicle 1, or the more objects exist near the vehicle 1. Therefore, the smaller the number of GPS satellites, the easier it is to capture an object present in the vicinity by the camera 12 of the vehicle 1, and the speed sensation of the operator is made faster than the actual vehicle speed by the moving image displayed on the display 202. It is assumed that the environment is likely to end up. In addition, when the number of GPS hygiene becomes 0, it is a case where it travels in a tunnel etc., and is a case where an electromagnetic wave is interrupted completely.

  In the present embodiment, the number of GPS sanitations output from the navigation device 13 is acquired by the CPU 91 in modified image data generation processing (see FIG. 15) described later. Based on the number of GPS sanitations, it is determined whether an object exists around the vehicle 1.

  Next, the corrected image data generation process (S17) executed by the CPU 91 of the vehicle side control device 100 mounted on the vehicle 1 will be described with reference to FIG. FIG. 15 is a flowchart showing the corrected image data generation process. This corrected image data generation process is a process executed in the image transmission process (see FIG. 6) executed by the CPU 91 of the vehicle-side control device 100, and a moving image to be displayed on the display 202 of the operation center 200. This is processing for generating image data (corrected image data) indicating a frame image to be formed.

  In the corrected image data generation process (S17), first, the number of GPS satellites that receive radio waves in the navigation device 13 is acquired from the output of the navigation device 13 (S91), and whether the number of GPS satellites acquired is 5 or less. Is determined (S92). As described above, as the number of GPS satellites that receive radio waves by the navigation device 13 increases, the vehicle 1 is assumed to be in an open environment in which no objects such as buildings exist in the vicinity. It is assumed that the smaller the number of GPS satellites that receive radio waves, the vehicle 1 is in an environment in which objects such as buildings exist. In the present embodiment, when the number of GPS satellites is 5 or less, it is assumed that an object is present around the vehicle 1, but the present invention is not necessarily limited to this, and preferably 4 Values may be set as appropriate within the above range.

  If the navigation device 13 determines that the number of GPS satellites that receive radio waves is 5 or less (S92: Yes), the process of S93 is executed.

  At the stage where the corrected image data generation process (S17) is started, the image data of the frame image newly acquired from the camera 12 by the process of S13 of the image transmission process (FIG. 6) is stored in the image data memory 93a. In the previous image memory 493a, image data indicating a frame image that is one frame before the image data stored in the image data memory 93a has been performed in the previous modified image data generation process. It is stored in advance by the process of S98 described later.

  The optical flow of the moving image captured by the camera 12 is calculated from the image data stored in the image data memory 93a and the image data stored in the previous image memory 493a (S93). Accordingly, the optical flows for two temporally continuous frame images indicated by the image data in the image data memory 93a and the previous image memory 493a are calculated.

  Here, an optical flow of a moving image captured by the camera 12 will be described with reference to FIG. FIG. 16 is an explanatory diagram for describing an optical flow of a moving image captured by the camera 12.

  FIG. 16 shows an optical flow (flow vector) superimposed on a frame image indicated by image data in the image data memory 93a. When the optical flow is calculated, a part of the image included in the temporally previous frame image (that is, the frame image indicated by the image data of the previous image memory 493a) is converted into the temporally subsequent frame image (that is, the image). When moving to another position in the frame image indicated by the image data in the data memory 93a, a vector of the movement is obtained. That is, the moving direction and the moving amount of the image area indicating the same point (corresponding point) in the two frame images are obtained. A plurality of arrows shown in FIG. 16 indicate flow vectors calculated for each region of the moving image.

  As described above with reference to FIG. 2B, in the moving image picked up by the camera 12, the speed of the road surface and the object flowing in the moving image increases toward the edge of the moving image. The amount of movement of the corresponding point in the frame image increases toward the edge of the moving image. For this reason, as shown in FIG. 16, the flow vector indicating the movement of the road surface or the like of the moving image captured by the camera 12 becomes a larger vector as the region is closer to the edge of the moving image.

  Further, FIG. 16 shows a flow vector when the vehicle 1 goes straight, but when the vehicle 1 is steered in either of the left and right directions, the predicted route of the vehicle 1 according to the steering (the first route) As the position is farther from the predicted route described with reference to FIG. 8 of the embodiment, the movement amount of the corresponding point in the two frame images increases. For this reason, a larger flow vector is calculated in the region on the opposite side to the steering direction than in the region in the same direction as the steering direction.

  On the other hand, if there is a moving object in the moving image, the moving object performs relative motion different from that of the road surface or the stationary object with respect to the vehicle 1, so the flow vector of the area where the moving object exists is The vector around the object is shown as a vector (vector M in FIG. 16) having a different orientation and size.

  Returning to FIG. 15, the description will be continued. Next, the size of the reference flow vector to be compared with the flow vector calculated in the process of S93 is calculated according to the number of GPS satellites receiving radio waves by the navigation device 13 and the vehicle speed ( S94). The size of the reference flow vector defines the size (maximum value) of the flow vector set in a region that gives the operator a sense of speed corresponding to the vehicle speed in the moving image.

  Here, the movement amount of the corresponding points in the two consecutive frame images increases as the vehicle speed increases. Therefore, in this embodiment, the magnitude of the reference flow vector is increased as the vehicle speed is higher.

  Further, as the number of GPS satellites that receive radio waves by the navigation device 13 is smaller, more objects such as buildings and walls are present in the vicinity of the vehicle 1 or near the vehicle 1. It is assumed that the speed sensation given to the operator by the moving image in which the periphery of the image is captured is faster as the number of GPS satellites is smaller. Therefore, in the processing of S94, the reference flow vector is increased as the number of GPS satellites that receive radio waves by the navigation device 13 is smaller.

  Further, in the present embodiment, as in the first embodiment, new image data is acquired from the camera 12 after the communication device 14 of the vehicle 1 completes communication of image data to the communication device 206 of the operation center 200. (See FIG. 6).

  For this reason, frame images are acquired from the camera 12 at a time interval corresponding to the communication speed of the image data. Therefore, when the communication speed decreases, the time interval for acquiring the frame image becomes long, and the vehicle 1 captures images from the camera 12. The frame rate of the moving image is lowered. Therefore, the frame rate of the moving image displayed on the display 202 based on this moving image is also low.

  On the other hand, when the frame rate of the moving image captured by the camera 12 is low, that is, when the time interval for acquiring frame images from the camera 12 is long, the time interval is short (that is, the frame rate is high). As compared with, the amount of movement of the image area between successive frame images becomes large. Therefore, when the time interval for acquiring frame images from the camera 12 is long, the flow vector for each region calculated by the processing of S93 is larger than when the time interval is short (that is, the frame rate is high). That's a big thing overall. Therefore, even if the size of the reference flow vector is not changed according to the frame rate of the moving image displayed on the display 202, the size of the region that gives the operator a sense of speed according to the vehicle speed in the moving image. Can be used as a reference.

  Subsequently, in the process of S95, a flow vector having an irregular size or direction with respect to the flow vector calculated for the surrounding area is searched from the flow vectors of the respective areas calculated in the process of S93, and the irregularity is determined. The presence or absence of a regular flow vector is determined (S95). The irregular flow vector referred to here is a flow vector in which at least one of the magnitude or direction deviates from the distribution when the distribution of the size and direction of a plurality of flow vectors calculated for the surrounding area is obtained. That is. As described above with reference to FIG. 16, when a moving object is present in a moving image, the flow vector of the moving object is different in direction and size from the road surface or the like. Therefore, in the process of S95, the presence or absence of the irregular flow vector is determined as the flow vector of the moving object. According to such a method, a moving object can be detected simply by calculating an optical flow of a moving image, and therefore, compared with other image processing (for example, an edge extraction method) for detecting a moving object. A moving object can be detected with a light processing load.

  If it is determined that there are no irregular flow vectors with respect to the surrounding flow vectors (S95: No), the process of S96 is executed. In the process of S96, data in a range from an area where a large flow vector exists with respect to the reference flow vector calculated in the process of S94 to the edge of the frame image to which the flow vector faces is transmitted to the operator as to the actual vehicle speed. It is deleted from the image data in the image data memory 93a as data in a region that gives a higher sense of speed. Thereby, the corrected image data is generated (S96). Therefore, the corrected image data is obtained from the frame image of the moving image captured by the camera 12 based on the vehicle speed, the frame rate of the moving image displayed on the display 202, and the environment in which the vehicle 1 exists. A region that gives a speed sensation faster than the actual vehicle speed is shown as a deleted frame image.

  Next, the corrected image data is stored in the transmission image buffer 92b (S97). Thereby, after the execution of the corrected image data generation process, the image data stored in the transmission image buffer 92b is transmitted to the operation center 200 via the communication device 14 (see S18 in FIG. 6). As a result, a moving image based on the corrected image data is displayed on the display 202. This moving image is obtained from the moving image captured by the camera 12 based on the vehicle speed, the frame rate of the moving image displayed on the display 202, and the environment in which the vehicle 1 exists. Since the area that gives a fast speed sensation is deleted, this moving image can give the operator a speed sensation according to the actual vehicle speed.

  After execution of the process of S97, the image data of the image data memory 93a is stored in the previous image memory 493a (S98), and this process is terminated.

  Returning to the process of S95, the description will be continued. When it is determined that there is a flow vector having an irregular size or direction with respect to the flow vectors calculated for the surrounding areas among the calculated flow vectors of the respective areas calculated in the process of S93 ( (S95: Yes), the process of S99 is executed.

  In the process of S99, except for the area where the irregular flow vector indicating the presence of the moving object is calculated, the flow vector is determined from the area where the flow vector larger than the reference flow vector calculated in the process of S94 exists. Data in the range up to the edge of the frame image facing is deleted from the image data to generate corrected image data (S99). The frame image indicated by the corrected image data is deleted from the frame image captured by the camera 12 for an area that gives the operator a speed sensation faster than the actual vehicle speed, and displays a moving object. It becomes. Therefore, when a moving image composed of the corrected image data is displayed on the display 202, the moving object can be surely visually recognized while giving the operator a sense of speed according to the actual vehicle speed. The operator can be made to operate the vehicle 1 according to the state of the moving object.

  Next, the corrected image data generated in the process of S99 is stored in the transmission image buffer 92b (S97), the image data in the image data memory 93a is stored in the previous image memory 493a (S98), and this process ends. To do.

  Returning to the process of S92, the description will be continued. If the navigation device 13 determines that the number of GPS satellites that receive radio waves is greater than 5 by the processing of S92 (S92: No), the image data in the image data memory 93a is transmitted to the transmission image buffer 93b and the previous image. The data is stored in the memory 493a (S100, S98), and this process ends. In such a case, the moving image captured by the camera 12 is transmitted to the display 202 as it is.

  Thus, in the third embodiment, when the number of GPS sanitations received by the navigation device 13 is 5 or less by the corrected image data generation process (see FIG. 17), the image data memory 93 and the previous image are displayed. An optical flow (flow vector) is calculated based on two frame images successively captured by the camera 12 and stored in the memory 493a (S93), and an image in which a flow vector larger than the reference flow vector is calculated The image data indicating the frame image of the moving image to be displayed on the display 202 is generated by deleting the data of the edge-side region indicated by the flow vector from the image data stored in the image data memory 93 from the region (S95). , S99).

  Here, the magnitude of the reference flow vector is set larger in the process of S94 as the vehicle speed is higher. Therefore, as the vehicle speed increases, a frame image in which a region from the edge to the inner side is deleted is generated as a frame image of a moving image displayed on the display 202. Therefore, in the moving image to be displayed on the display 202, the higher the vehicle speed, the larger the range of the area to be deleted (that is, not displayed) from the edge with the fastest flow on the road surface to the inside of the moving image. Therefore, the flow speed of the entire moving image displayed on the display 202 can be suppressed. Thereby, it can suppress that the speed sensation given to an operator by the flow of the road surface etc. which are reflected on a moving image becomes faster than an actual vehicle speed.

  Further, the size of the reference flow vector is set larger in the process of S94 as the number of GPS satellites that receive radio waves by the navigation device 13 is smaller. As described above, it is assumed that as the number of GPS satellites that receive radio waves by the navigation device 13 is smaller, there are objects closer to the periphery of the vehicle 1 or more objects are present. . Therefore, the frame image in which the region from the edge to the inner side is deleted as the degree of approach between the vehicle 1 and the object or the degree of existence of the object around the vehicle 1 is higher depending on the number of GPS satellites that receive radio waves. Is generated as a frame image of a moving image displayed on the display 202. Therefore, in the moving image displayed on the display 202, the extent of the area to be deleted (that is, not displayed) is increased as the degree of approach between the vehicle 1 and the object or the degree of existence of the object around the vehicle 1 is higher. Since the image is enlarged from the edge with the fastest flow on the road surface to the inside of the moving image, the flow speed of the entire moving image displayed on the display 202 can be suppressed. Therefore, regardless of the environment in which the vehicle 1 travels, it is possible to give the operator a sense of speed according to the actual vehicle speed.

  As described above, in the present embodiment, the frame rate of the moving image captured by the camera 12 is lowered according to the decrease in the communication speed in the image data transmission path from the vehicle 1 to the operation center 200. For this reason, as the frame rate decreases, the size of the optical flow calculated for each region of the moving image captured by the camera in the processing of S93 is relatively larger than the reference flow vector. Therefore, as the communication speed decreases, the data corresponding to many areas are deleted from the image data captured by the camera 12 by the processes of S96 and S99, and the corrected image data is generated. Therefore, as the communication speed in the transmission path decreases, the data amount of the corrected image data, that is, the data amount per frame image displayed on the display 202 can be reduced, so the communication speed decreases. However, it is possible to suppress a decrease in the number of frame images that can be transmitted per hour from the vehicle 1 to the operation center 200, and to suppress a decrease in the frame rate of the moving image displayed on the display 202. For this reason, it can suppress that an operator's speed sensation becomes higher than an actual vehicle speed by displaying a moving image with a low frame rate on the display 202.

  Moreover, the corrected image data deleted by the processes of S96 and S99 indicates a frame image in which the region from the edge of the optical flow to the inner side is deleted as the frame rate is lower. Therefore, in the moving image displayed on the display 202, the lower the frame rate of the moving image, the more the range of the area to be deleted (that is, not displayed) from the edge with the fastest flow on the road surface or the like. Since the image is enlarged toward the inside, the flow of the entire moving image displayed on the display 202 is slowed down. Accordingly, the speed sensation of the operator, which is accelerated by the moving image with a reduced frame rate, can be canceled by slowing the flow of the entire moving image. Therefore, by displaying a moving image with a low frame rate on the display 202, it is possible to suppress the speed sensation of the operator from becoming higher than the actual vehicle speed.

  Further, according to the processing of S93, a smaller flow vector is calculated in the region of the steering direction that is the traveling direction than in the region on the opposite side to the steering direction. Therefore, when the process of S95 or S99 is performed, it is deleted from the image data of the frame image captured by the camera 12 in preference to the data of the region with the large flow vector located on the side opposite to the steering direction. Modified image data is generated. As a result, the corrected image data of the frame image that can be visually recognized is generated for the traveling direction, and therefore the field of view of the traveling direction of the vehicle 1 is ensured in the moving image displayed on the display 202 based on the corrected image data. be able to.

  The image display process (see FIG. 13) described in the above embodiment corresponds to the display control unit described in claim 7.

  As described above, the present invention has been described based on the embodiments, but the present invention is not limited to the above-described embodiments, and various improvements and modifications can be easily made without departing from the spirit of the present invention. It can be guessed. For example, the numerical values given in the above embodiments are examples, and other numerical values can naturally be adopted.

  Also in the first and second embodiments, as in the third embodiment, it is detected whether there is a moving object in the range imaged by the camera 12, and if there is a moving object, at least the About a moving object, you may make it display in the moving image displayed on the display 202. FIG. For example, also in the vehicle control device 100 of the first embodiment and the operation side control device 201 of the second embodiment, the previous image memory 493a is provided as in the third embodiment, and the image data stored in the image data memory 93a is stored. The image data of one frame before is stored. Then, in the corrected image data generation process (S17), when the travel section of the vehicle 1 is “tunnel” or “urban area”, the optical flow is calculated using each image data in the image data memory 93a and the previous image memory 493a. The process for detecting a moving object from the optical flow and the process for determining whether the area where the moving object is detected is included in the display range set according to the angle of view acquired from the gazing point and the angle of view map When the moving object is included in the display range, the image data is deleted while leaving the display range, and the corrected image data is generated. When the moving object is not included in the display range, the moving object The image data is deleted and the corrected image data is generated so that at least the area including the moving object and the display range are left. Configured to.

  According to such a configuration, similarly to the third embodiment, when a moving object exists in the imaging range of the camera 12, the moving object can be reliably included in the moving image displayed on the display 202. Therefore, the moving object displayed on the display 202 can make the moving object reliably visible to the operator, and the operation of the vehicle 1 according to the state of the moving object can be performed.

  In each of the above embodiments, the vehicle 1 is provided with a sensor such as a laser sensor that detects a distance and an angle with an object existing in the imaging range captured by the camera 12 of the vehicle 1, and the object existing in the imaging range by the sensor. In this case, an object existing within a predetermined distance (for example, within 10 m) from the vehicle 1 may be displayed at least in the moving image displayed on the display 202. For example, in the corrected image data generation process (S17), when the travel section of the vehicle 1 is “tunnel” or “urban area”, the angle of view acquired from the gaze point and the angle-of-view map based on the output result of the sensor If the display range of the frame image set in accordance with the processing for determining whether an object existing within a predetermined distance from the vehicle 1 is included in the display range, and if the object detected by the sensor is included in the display range, Based on the process of deleting the image data while leaving the display range and generating the corrected image data, and the distance and angle of the object detected by the sensor when the object detected by the sensor is not included in the display range In order to display at least the object, the image data is deleted and the process of generating the corrected image data is executed, leaving at least the region including the object and the display range.

  According to such a configuration, when an object exists within a predetermined distance from the vehicle 1 in the imaging range of the camera 12, the object can be displayed as a moving image displayed on the display 202. Therefore, the operator can make an operation for avoiding contact between the object and the vehicle 1 after making the operator visually recognize the object existing within the predetermined distance by the moving image displayed on the display 202. Can be done.

  In each of the above-described embodiments, detection is performed when a specific target that can be moved, such as a person, another vehicle, or a two-wheeled vehicle, is present within the range imaged by the camera 12, and detected In such a case, at least the specific target may be displayed in the moving image displayed on the display 202. For example, a ROM 92 of the vehicle control device 100 according to the embodiment includes a pattern memory that stores a plurality of outline patterns for a specific target such as a person, another vehicle, or a two-wheeler for each angle or size at which the specific target can be imaged. It is provided in the ROM 292 of the operation side control device 201 of the second embodiment. In the corrected image data generation process (S17), when the travel section of the vehicle 1 is “tunnel” or “urban area”, the luminance greatly changes in the frame image indicated by the image data stored in the image data memory 93a. A process for extracting a matching edge, a process for determining whether the edge and the pattern stored in the pattern memory match, and an object that matches the pattern depends on the gaze point and the angle of view acquired from the angle of view map. A process for determining whether the display range is set, and a process for generating corrected image data by deleting the image data while leaving the display range when a target that matches the pattern is included in the display range. If the target that matches the pattern is not included in the display range, the target area and the display range so that the target is displayed at least Remove the image data at least leaving, configured to perform a process of generating the corrected image data.

  According to such a configuration, when a specific target exists in the imaging range of the camera 12, the specific image is displayed on the moving image displayed on the display 202 regardless of whether or not the specific target is moving. The subject can be reliably included. Therefore, the moving image displayed on the display 202 allows the operator to visually recognize a specific target that can be moved, and allows the operator to operate the vehicle 1 in accordance with the status of the target. it can.

  In each of the above-described embodiments, the region where the operator can no longer visually recognize the moving image captured by the camera 12 on the display 202 has been described with respect to the case where the data for displaying the region is deleted. For example, the area of the frame image captured by the camera 12 is filled with a single color, or the brightness is extremely increased or decreased so that the operator cannot view the scenery. May be.

  In each of the above embodiments, the corrected image data generation process (S17) converts the image data of the image captured by the camera 12 according to the vehicle speed, the frame rate of the moving image displayed on the display 202, and the environment of the vehicle 1. The case has been described where predetermined image processing is performed to generate a frame image of a moving image displayed on the display 202 in which a display range is adjusted from a moving image captured by the camera 12, but the present invention is not limited to this. The camera is provided with a telephoto lens whose angle of view can be adjusted and a plurality of lenses with different angles of view, depending on the vehicle speed, the frame rate of the moving image displayed on the display 202, and the environment of the vehicle 1 Alternatively, the lens of the camera 12 may be switched to capture an image, and the captured moving image may be displayed on the display 202 as it is.

  In the first and second embodiments, a case has been described in which the operation-side control device 201 obtains the frame rate of the moving image displayed on the display 202 and obtains the angle of view of the corrected image data according to the frame rate. However, the present invention is not necessarily limited to this, and the operation-side control device 202 obtains the communication speed of the image data from the communication device 14 of the vehicle 1 to the communication device 206 of the operation center 200 as information indicating the frame rate. The angle of view of the corrected image data may be obtained according to the communication speed. At this time, for example, the view angle map memories 92b and 393a may be configured to define the view angle in association with the communication speed of the image data instead of the moving image frame rate on the display 202.

  Further, in the first and third embodiments, the vehicle speed or the speed of the image data transmitted from the vehicle 1 to the operation center 200 (or the frame rate of the moving image displayed on the display 202) is reduced. Regardless of the traveling environment of the vehicle, data regarding pixels included in the vicinity of the upper edge of the moving image may be deleted from the image data indicating the frame image of the moving image captured by the camera 12. For example, in the image transmission process (see FIG. 7), the image data is stored in the reception frame rate memory 93g after the image data is stored in the transmission image buffer 93b until the image data is output to the communication device 14. Processing for determining whether the frame rate is lower than a predetermined threshold (for example, vehicle speed), and if it is lower than the predetermined threshold, from the image data stored in the transmission image buffer 93b, near the upper edge of the moving image And processing to output 14 to the communication device after deleting the data about the pixels included in the.

  Here, since the upper part of the vehicle 1 is imaged in the area near the upper edge of the moving image captured by the camera 12, an object that exists only in this area does not become an object that hinders the traveling of the vehicle. Therefore, according to the above configuration, when the communication speed of the image data transmitted from the vehicle 1 to the operation center 200 is low, the data for the pixels near the upper edge of the frame image is deleted from the image data indicating the frame image. Thus, a frame image of a moving image to be displayed on the display 202 is generated. Therefore, by deleting the data in the display range of the moving image captured by the camera 12 so that the operator does not interfere with the operation of the vehicle 1 while avoiding the contact between the vehicle 1 and the object, the frame is deleted. By reducing the amount of image data per image, it is possible to suppress a decrease in frame rate.

  Further, in the first and third embodiments, the vehicle speed or the speed of the image data transmitted from the vehicle 1 to the operation center 200 (or the frame rate of the moving image displayed on the display 202) is reduced. Regardless of the traveling environment of the vehicle, data regarding pixels included in the vicinity of the upper edge of the moving image may be deleted from the image data indicating the frame image of the moving image captured by the camera 12. For example, in the image transmission process (see FIG. 7), the image data is stored in the reception frame rate memory 93g after the image data is stored in the transmission image buffer 93b until the image data is output to the communication device 14. Processing for determining whether the frame rate is lower than a predetermined threshold (for example, vehicle speed), and if it is lower than the predetermined threshold, from the image data stored in the transmission image buffer 93b, near the upper edge of the moving image And processing to output 14 to the communication device after deleting the data about the pixels included in the.

  Here, since the upper part of the vehicle 1 is imaged in the area near the upper edge of the moving image captured by the camera 12, an object that exists only in this area does not become an object that hinders the traveling of the vehicle. Therefore, according to the above configuration, when the communication speed of the image data transmitted from the vehicle 1 to the operation center 200 is low, the data for the pixels near the upper edge of the frame image is deleted from the image data indicating the frame image. Thus, a frame image of a moving image to be displayed on the display 202 is generated. Therefore, by deleting the data in the display range of the moving image captured by the camera 12 so that the operator does not interfere with the operation of the vehicle 1 while avoiding the contact between the vehicle 1 and the object, the frame is deleted. By reducing the amount of image data per image, it is possible to suppress a decrease in frame rate.

  In the first and third embodiments, the frame rate of the moving image displayed on the display 202 is prevented from being lowered in accordance with the reduction in the communication speed of the image data transmitted from the vehicle 1 to the operation center 200. In order to do this, in the vehicle 1, data on some pixels is deleted from the image data indicating the frame image of the moving image captured by the camera 12, and the remaining image data is displayed on the display 202. Although the case where it transmits to the operation center 200 from the vehicle 1 as what shows one frame image was demonstrated, it is not necessarily restricted to this. For example, in response to a decrease in communication speed of image data transmitted from the vehicle 1 to the operation center 200, image data captured by the camera 12 is quantized for each of a plurality of pixels to reduce the data amount, Pixel color information may be deleted to reduce the amount of data. Further, a plurality of frame images may be compressed by the MPEG method and then transmitted from the vehicle 1 to the operation center 200. Further, these methods may be combined with a method of deleting data for some pixels from image data indicating a frame image of a moving image captured by the camera 12.

  Further, in the first and second embodiments, the case where the pixels near the left and right edges of the moving image are deleted when the traveling section of the vehicle 1 is “urban area” has been described. Pixels near the lower edge of the image may be deleted. Since the road surface below the vehicle 1 is displayed near the lower edge of the moving image, the frame rate of the moving image displayed on the display 202 is low by deleting the pixels near the lower edge of the moving image. In addition, when the vehicle speed is high with respect to the angle of view of the moving image, it is possible to suppress the speed sensation of the operator that is accelerated by the road surface flow.

  In the first and second embodiments, when the travel section of the vehicle 1 is “others” (S32: “others”), when the frame rate of the moving image displayed on the display 202 is low, When the vehicle speed is higher than the angle of view of the moving image, the pixels near the lower edge of the moving image may be deleted. Even in a clear section such as the “other” travel section, when the frame rate of the moving image displayed on the display 202 is low, or when the vehicle speed is high relative to the angle of view of the moving image, the road surface It will be accelerated by the flow. On the other hand, according to such a configuration, when the frame rate of the moving image displayed on the display 202 is low or the vehicle speed is higher than the angle of view of the moving image in the “other” travel section, By deleting pixels near the lower edge of the image, it is possible to suppress the speed sensation of the operator that is accelerated by the flow on the road surface.

  Further, in each of the embodiments described above, the vehicle 1 is the information indicating the environment in which the vehicle 1 exists (the positional relationship between the vehicle 1 and objects around the vehicle 1 and the ratio of the object in the imaging range of the camera. However, the present invention is not necessarily limited to this, and may be configured to be acquired by road-to-vehicle communication or vehicle-to-vehicle communication. Alternatively, an image captured by the camera 13 may be analyzed to obtain information indicating an environment where the vehicle 1 exists.

  Further, in each of the above embodiments, a part or a plurality of parts of the configuration of the other embodiments are added to the embodiment or replaced with a part or a plurality of parts of the configuration of the embodiment. The embodiment may be modified and configured.

1 vehicle 12 camera (imaging means)
13 Navigation device (part of information acquisition means)
14 Communication device (transmission means, reception means)
100 vehicle side control device (travel control means)
200 Operation center (remote control device)
202 Display (display means)
203 Steering device (accepting means)
204 Accelerator pedal device (reception means)
205 Brake pedal device (reception means)
206 Communication device (transmission means, reception means, part of information acquisition means)
393f Navigation information memory (part of information acquisition means)
S17 Modified image data generation processing (moving image information generation means)
S32, S35, S39 First area setting means S91 Part of information acquisition means S93 Part of second detection means, Optical flow calculation means S95 Part of second detection means, Optical flow comparison means S96, S99 Third area setting means

Claims (7)

Imaging means for imaging the periphery of the vehicle as a moving image;
Moving image information generating means for generating moving image information based on the moving image captured by the imaging means;
Transmitting means for transmitting the moving image information generated by the moving image information generating means to a remote control device provided outside the vehicle;
Based on the moving image information transmitted by the transmitting means, the driving, braking, or driving of the vehicle received from the operator by the remote operation device based on the moving image displayed on the display means provided in the remote operation device Receiving means for receiving, from the remote control device, instruction information representing an instruction relating to steering;
Travel control means for controlling driving, braking or steering of the vehicle based on the instruction information received by the receiving means;
Comprising information acquisition means for acquiring information related to an object existing around the vehicle,
The moving image information generating unit generates a moving image in which at least a part of a region located at an outer edge of the moving image is invisible based on the information acquired by the information acquiring unit, and the generated moving image is A vehicle that generates moving image information to be displayed on the display means of a remote control device. The information acquisition means acquires information indicating a positional relationship of an object existing around the vehicle with respect to the vehicle as information relating to an object existing around the vehicle,
When the moving image information generating unit generates a moving image in which at least a part of the region located on the outer edge of the moving image is invisible, the information acquiring unit displays an area invisible in the moving image. And a first region setting unit configured to set a part of a region located on an outer edge of a moving image in a direction in which an object existing around the vehicle exists based on the information acquired by the step (a). The vehicle according to 1. The information acquisition means acquires predetermined information relating to a ratio of an object in a range imaged by the imaging means as information relating to an object existing around the vehicle,
The moving image information generation means is made invisible when the ratio of the object in the range imaged by the imaging means is the first ratio based on the predetermined information acquired by the information acquisition means. A moving image set from the outer edge of the moving image to the inner side of the range of the region that is not visible when the range of the region is the second rate in which the probability is lower than the first rate The vehicle according to claim 1, wherein the vehicle generates an image. A first detection unit for detecting an object existing within a predetermined distance from the vehicle in a range in which a moving image is captured by the imaging unit;
The moving image information generating means generates a moving image in which at least a part of an area located at an outer edge of the moving image is invisible, and the first detection is performed on an area that is invisible in the moving image. The vehicle according to any one of claims 1 to 3, further comprising second area setting means for setting an area where an object detected by the means is present so as to be visible. A second detection means for detecting an object that is in a range where a moving image is picked up by the image pickup means and that can move;
When the moving image information generating unit generates a moving image in which at least a part of the region located on the outer edge of the moving image is invisible, the region where the object detected by the second detecting unit exists The vehicle according to any one of claims 1 to 4, further comprising third area setting means for setting an area that is not visible so as to be visible. The second detection means includes
Optical flow calculation means for calculating an optical flow in each region of the moving image from the moving image indicated by the moving image information received by the receiving means;
Optical flow comparison means for comparing the optical flow of each area calculated by the optical flow calculation means with the optical flow calculated for the area surrounding each area,
An optical flow in which at least one of the direction and the size is not included in the distribution of at least one of the direction and the size of the plurality of optical flows calculated for the surrounding regions compared by the optical flow comparison unit is calculated. 6. The vehicle according to claim 5, wherein the detection is performed on the assumption that a moving object exists in the area where the vehicle is located. A remote control device capable of operating a vehicle for driving, braking or steering from outside the vehicle,
Receiving means for receiving, from the vehicle, moving image information generated based on moving images around the vehicle imaged by an imaging means provided in the vehicle;
Display means for displaying a moving image based on the moving image information received by the receiving means;
Receiving means for receiving an instruction related to driving, braking or steering of the vehicle instructed by an operator based on a moving image displayed on the display means;
Transmitting means for transmitting instruction information representing the instruction received by the receiving means to the vehicle;
Information acquisition means for acquiring information related to an object existing around the vehicle;
For the moving image indicated by the moving image information received by the receiving unit, at least a part of the region located at the outer edge of the moving image is made invisible based on the information acquired by the information acquiring unit. A remote control device comprising: a display control unit that generates a moving image and displays the generated moving image on the display unit.

Images