JP2019014397A - Periphery monitoring device - Google Patents

Abstract

To provide a periphery monitoring device that can reduce burdens which are applied to an occupant of a vehicle when displaying a first image on a display part.SOLUTION: A periphery monitoring device according to one embodiment, as one example, comprises: a generation part that generates a first image, in which a gazing point fixed into a virtual space having images of a vehicle provided in spaces surrounding the vehicle is viewed from a virtual viewpoint inputted through an operation input part, on the basis of photographed images obtained by photographing the circumference of the vehicle by a photographing part mounted on the vehicle; and an output part that outputs on a display part the first image generated by the generation part.SELECTED DRAWING: Figure 4

Description

  Embodiments described herein relate generally to a periphery monitoring device.

  Based on the captured image obtained by imaging the surroundings of the vehicle by the imaging unit, a display image that is a three-dimensional image around the vehicle and that looks at the gazing point around the vehicle from a virtual viewpoint is generated, and the generation A technique for displaying the displayed image on the display unit has been developed.

International Publication No. 2014/156220

  However, when the user sets both the gazing point and the virtual viewpoint when displaying the display image on the display unit, the burden on the user due to the setting of the gazing point and the virtual viewpoint increases.

  As an example, the periphery monitoring device of the embodiment is fixed in a virtual space in which a vehicle image is provided in a space around the vehicle based on a captured image obtained by imaging the periphery of the vehicle by an imaging unit mounted on the vehicle. A generation unit that generates a first image of the gazing point that is viewed from a virtual viewpoint input via the operation input unit; and an output unit that outputs the first image generated by the generation unit to the display unit. Prepare. Therefore, as an example, it is possible to reduce the burden on the vehicle occupant when the first image is displayed on the display unit.

  Further, as an example, in the periphery monitoring device, the generation unit sets a gaze point at a position away from a vehicle image by a preset distance. Therefore, as an example, by setting the gazing point in advance at a position that becomes a blind spot as seen from the vehicle occupant, the vehicle occupant can set the blind spot of the vehicle without inputting the gazing point via the operation input unit. It is possible to confirm an image at a position via the display unit.

  For example, in the periphery monitoring device, the generation unit moves the gazing point with the movement of the vehicle image corresponding to the movement of the vehicle. Therefore, as an example, it is possible to reduce the burden on the vehicle occupant when the first image is displayed on the display unit.

  For example, in the periphery monitoring device, the generation unit generates a first image including an operation image that allows a virtual viewpoint to be input, and when a new virtual viewpoint is input using the operation image, a new virtual viewpoint is input. Regenerate the first image that looks at the gazing point from the viewpoint. Therefore, as an example, the burden on the vehicle occupant when the first image is displayed on the display unit can be further reduced.

  Moreover, as for an example of the periphery monitoring device, the vehicle image is a three-dimensional image of the vehicle. Therefore, as an example, the positional relationship between the vehicle image included in the first image and the surrounding three-dimensional object can be made easier to understand.

FIG. 1 is a perspective view illustrating an example of a state in which a part of a passenger compartment of a vehicle on which the periphery monitoring device according to the present embodiment is mounted is seen through. FIG. 2 is a plan view of an example of a vehicle according to the present embodiment. FIG. 3 is a block diagram illustrating an example of a functional configuration of the vehicle according to the present embodiment. FIG. 4 is a block diagram illustrating an example of a functional configuration of the ECU included in the vehicle according to the present embodiment. FIG. 5 is a flowchart illustrating an example of a flow of display processing of the display image by the vehicle according to the present embodiment. FIG. 6 is a diagram for explaining an example of a camera image model used for generating a display image by the vehicle according to the present embodiment. FIG. 7 is a diagram for explaining an example of a camera image model used for generating a display image by the vehicle according to the present embodiment. FIG. 8 is a diagram for explaining an example of a camera image model and a vehicle image used for generating a display image by the vehicle according to the present embodiment. FIG. 9 is a diagram for explaining an example of a camera image model and a vehicle image used for generating a display image by the vehicle according to the present embodiment. FIG. 10 is a diagram illustrating an example of a display image generated in the vehicle according to the present embodiment. FIG. 11 is a diagram illustrating an example of a display image generated in the vehicle according to the present embodiment.

  Hereinafter, exemplary embodiments of the present invention are disclosed. The configuration of the embodiment shown below and the operations, results, and effects brought about by the configuration are examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and at least one of various effects based on the basic configuration and derivative effects can be obtained.

  The vehicle equipped with the periphery monitoring device (perimeter monitoring system) according to the present embodiment may be an automobile (internal combustion engine automobile) using an internal combustion engine (engine) as a drive source, or an electric motor (motor) as a drive source. The vehicle may be a vehicle (electric vehicle, fuel cell vehicle, etc.), or a vehicle (hybrid vehicle) using both of them as drive sources. The vehicle can be mounted with various devices (systems, components, etc.) necessary for driving various transmissions, internal combustion engines, and electric motors. In addition, the method, number, layout, and the like of devices related to driving of wheels in a vehicle can be variously set.

  FIG. 1 is a perspective view illustrating an example of a state in which a part of a passenger compartment of a vehicle on which the periphery monitoring device according to the present embodiment is mounted is seen through. As shown in FIG. 1, the vehicle 1 includes a vehicle body 2, a steering unit 4, an acceleration operation unit 5, a braking operation unit 6, a speed change operation unit 7, and a monitor device 11. The vehicle body 2 has a passenger compartment 2a in which an occupant rides. In the passenger compartment 2a, a steering section 4, an acceleration operation section 5, a braking operation section 6, a transmission operation section 7 and the like are provided in a state where a driver as an occupant faces the seat 2b. The steering unit 4 is, for example, a steering wheel that protrudes from the dashboard 24. The acceleration operation unit 5 is, for example, an accelerator pedal positioned under the driver's foot. The braking operation unit 6 is, for example, a brake pedal that is positioned under the driver's feet. The shift operation unit 7 is, for example, a shift lever that protrudes from the center console.

  The monitor device 11 is provided, for example, at the center of the dashboard 24 in the vehicle width direction (that is, the left-right direction). The monitor device 11 may have a function such as a navigation system or an audio system, for example. The monitor device 11 includes a display device 8, an audio output device 9, and an operation input unit 10. The monitor device 11 may also include various operation input units such as switches, dials, joysticks, and push buttons.

  The display device 8 includes an LCD (Liquid Crystal Display), an OELD (Organic Electroluminescent Display), and the like, and can display various images based on image data. The audio output device 9 is configured by a speaker or the like, and outputs various sounds based on audio data. The sound output device 9 may be provided in a different position other than the monitor device 11 in the passenger compartment 2a.

  The operation input unit 10 is configured with a touch panel or the like, and allows various information to be input by an occupant. The operation input unit 10 is provided on the display screen of the display device 8 and can transmit an image displayed on the display device 8. Thereby, the operation input unit 10 enables an occupant to visually recognize an image displayed on the display screen of the display device 8. The operation input unit 10 receives an input of various information by the occupant by detecting the occupant's touch operation on the display screen of the display device 8.

  FIG. 2 is a plan view of an example of a vehicle according to the present embodiment. As shown in FIGS. 1 and 2, the vehicle 1 is a four-wheeled vehicle or the like, and has two left and right front wheels 3 </ b> F and two right and left rear wheels 3 </ b> R. All or some of the four wheels 3 can be steered.

  The vehicle 1 is equipped with a plurality of imaging units 15. In the present embodiment, the vehicle 1 includes, for example, four imaging units 15a to 15d. The imaging unit 15 is a digital camera having an imaging element such as a charge coupled device (CCD) or a CMOS image sensor (CIS). The imaging unit 15 can capture the surroundings of the vehicle 1 at a predetermined frame rate. Then, the imaging unit 15 outputs a captured image obtained by imaging the periphery of the vehicle 1. The imaging unit 15 has a wide-angle lens or a fisheye lens, respectively, and can image a range of, for example, 140 ° to 220 ° in the horizontal direction. Further, the optical axis of the imaging unit 15 may be set obliquely downward.

  Specifically, the imaging unit 15a is located at, for example, the rear end 2e of the vehicle body 2, and is provided on a wall portion below the rear window of the rear hatch door 2h. And the imaging part 15a can image the area | region of the back of the said vehicle 1 among the circumference | surroundings of the vehicle 1. FIG. The imaging unit 15b is located, for example, at the right end 2f of the vehicle body 2 and is provided on the right door mirror 2g. And the imaging part 15b can image the area | region of the side of the said vehicle among the circumference | surroundings of the vehicle 1. FIG. The imaging unit 15c is located, for example, on the front side of the vehicle body 2, that is, on the front end 2c in the front-rear direction of the vehicle 1, and is provided on a front bumper, a front grill, or the like. And the imaging part 15c can image the area | region ahead of the said vehicle 1 among the circumference | surroundings of the vehicle 1. FIG. The imaging unit 15d is located, for example, on the left side of the vehicle body 2, that is, on the left end 2d in the vehicle width direction, and is provided on the left door mirror 2g. Then, the imaging unit 15 d can capture an area on the side of the vehicle 1 in the periphery of the vehicle 1.

  FIG. 3 is a block diagram illustrating an example of a functional configuration of the vehicle according to the present embodiment. As shown in FIG. 3, the vehicle 1 includes a steering system 13, a brake system 18, a steering angle sensor 19, an accelerator sensor 20, a shift sensor 21, a wheel speed sensor 22, an in-vehicle network 23, an ECU ( Electronic Control Unit) 14. The monitor device 11, the steering system 13, the brake system 18, the steering angle sensor 19, the accelerator sensor 20, the shift sensor 21, the wheel speed sensor 22, and the ECU 14 are electrically connected via an in-vehicle network 23 that is an electric communication line. ing. The in-vehicle network 23 is configured by a CAN (Controller Area Network) or the like.

  The steering system 13 is an electric power steering system, an SBW (Steer By Wire) system, or the like. The steering system 13 includes an actuator 13a and a torque sensor 13b. And the steering system 13 is electrically controlled by ECU14 grade | etc., Operates the actuator 13a, adds the torque with respect to the steering part 4, and steers the wheel 3 by supplementing a steering force. The torque sensor 13 b detects the torque that the driver gives to the steering unit 4 and transmits the detection result to the ECU 14.

  The brake system 18 includes an ABS (Anti-lock Brake System) that controls the locking of the brake of the vehicle 1, a skid prevention device (ESC: Electronic Stability Control) that suppresses the skidding of the vehicle 1 during cornering, and increases the braking force. Includes an electric brake system that assists brakes and BBW (Brake By Wire). The brake system 18 includes an actuator 18a and a brake sensor 18b. The brake system 18 is electrically controlled by the ECU 14 or the like, and applies a braking force to the wheel 3 via the actuator 18a. The brake system 18 detects the lock of the brake, the idle rotation of the wheel 3, and the sign of the skid from the rotation difference of the left and right wheels 3, and performs control for suppressing the brake lock, the idling of the wheel 3, and the skid. Run. The brake sensor 18b is a displacement sensor that detects a position of a brake pedal as a movable part of the braking operation unit 6, and transmits a detection result of the position of the brake pedal to the ECU 14.

  The steering angle sensor 19 is a sensor that detects a steering amount of the steering unit 4 such as a steering wheel. In the present embodiment, the rudder angle sensor 19 is configured by a hall element or the like, detects the rotation angle of the rotating portion of the steering unit 4 as a steering amount, and transmits the detection result to the ECU 14. The accelerator sensor 20 is a displacement sensor that detects the position of an accelerator pedal as a movable part of the acceleration operation unit 5, and transmits the detection result to the ECU 14.

  The shift sensor 21 is a sensor that detects the position of the movable part (bar, arm, button, etc.) of the speed change operation part 7, and transmits the detection result to the ECU 14. The wheel speed sensor 22 has a Hall element and the like, and is a sensor that detects the amount of rotation of the wheel 3 and the number of rotations of the wheel 3 per unit time, and transmits the detection result to the ECU 14.

  The ECU 14 generates a three-dimensional image in which the gazing point around the vehicle 1 is viewed from the virtual viewpoint based on the captured image obtained by imaging the surroundings of the vehicle 1 by the imaging unit 15, and the generated three-dimensional Are displayed on the display device 8. The ECU 14 is configured by a computer or the like, and governs overall control of the vehicle 1 by cooperation of hardware and software. Specifically, the ECU 14 includes a CPU (Central Processing Unit) 14a, a ROM (Read Only Memory) 14b, a RAM (Random Access Memory) 14c, a display control unit 14d, an audio control unit 14e, and an SSD (Solid State Drive) 14f. Is provided. The CPU 14a, ROM 14b, and RAM 14c may be provided on the same circuit board.

  The CPU 14a reads a program stored in a nonvolatile storage device such as the ROM 14b, and executes various arithmetic processes according to the program. For example, the CPU 14a executes image processing on image data to be displayed on the display device 8, calculation of a distance to an obstacle existing around the vehicle 1, and the like.

  The ROM 14b stores various programs and parameters necessary for executing the programs. The RAM 14c temporarily stores various data used in the calculation by the CPU 14a. The display control unit 14d mainly performs image processing on image data acquired from the imaging unit 15 and output to the CPU 14a, and an image for display that causes the display device 8 to display the image data acquired from the CPU 14a. Perform conversion to data. The voice control unit 14e mainly executes a voice process acquired from the CPU 14a and output to the voice output device 9 among the calculation processes in the ECU 14. The SSD 14f is a rewritable nonvolatile storage unit and continues to store data acquired from the CPU 14a even when the ECU 14 is powered off.

  FIG. 4 is a block diagram illustrating an example of a functional configuration of the ECU included in the vehicle according to the present embodiment. As shown in FIG. 4, the ECU 14 includes a display image generation unit 401 and a display image output unit 402. For example, when a processor such as the CPU 14a mounted on the circuit board executes a peripheral monitoring program stored in a storage medium such as the ROM 14b or the SSD 14f, the ECU 14 causes the display image generation unit 401 and the display image to be displayed. The function of the output unit 402 is realized. Part or all of the display image generation unit 401 and the display image output unit 402 may be configured by hardware such as a circuit.

  The display image generation unit 401 uses the imaging unit 15 to image the surroundings of the vehicle 1 at a position (hereinafter referred to as past position) of the vehicle 1 at a certain time (hereinafter referred to as past time) from the imaging unit 15. Obtained captured images are acquired. Next, the display image generation unit 401 generates a display image that makes it possible to visually recognize the positional relationship between the vehicle 1 and an obstacle existing around the vehicle 1 based on the acquired captured image.

  Specifically, the display image generation unit 401 is an image obtained by viewing a gazing point in a virtual space around the vehicle 1 from a virtual viewpoint input via the operation input unit 10 based on the acquired captured image. Is generated as a display image. Here, the virtual space is a space around the vehicle 1 at the past position, and the position (for example, the current position) of the vehicle 1 at a time later than the past time (for example, the current time) in the space. This is a space where vehicle images are provided. The vehicle image is an image of the three-dimensional vehicle 1 that can be seen through the virtual space. In the present embodiment, the display image generation unit 401 arranges a vehicle image that can be seen through the virtual space in the virtual space, but may arrange a vehicle image that cannot be seen through the virtual space in the virtual space. .

  In the present embodiment, the display image generation unit 401 pastes the acquired captured image on a three-dimensional surface (hereinafter referred to as a camera image model) around the vehicle 1 at a past position, and the camera image model. Is generated as a space around the vehicle 1 at the past position. Next, the display image generation unit 401 generates a space in which the vehicle image is arranged as a virtual space with respect to the current position of the vehicle 1 in the generated space. Thereafter, the display image generation unit 401 generates, as a display image, an image of the fixed gazing point in the generated virtual space viewed from the virtual viewpoint input via the operation input unit 10.

  In the present embodiment, the display image generation unit 401 views the fixed gazing point in the virtual space from the new virtual viewpoint every time a new virtual viewpoint is input via the operation input unit 10. Regenerate the display image. The display image output unit 402 outputs the display image generated by the display image generation unit 401 to the display device 8. As a result, when the display device 8 displays the display image in which the gazing point around the vehicle 1 is viewed from the virtual viewpoint, the display image is displayed only by the occupant of the vehicle 1 indicating only the virtual viewpoint. Therefore, it is possible to reduce the burden on the occupant of the vehicle 1 when the display image is displayed on the display device 8.

  Next, an example of the flow of the display processing of the display image by the vehicle 1 according to the present embodiment will be described with reference to FIG. FIG. 5 is a flowchart illustrating an example of a flow of display processing of the display image by the vehicle according to the present embodiment.

  In the present embodiment, the display image generation unit 401 determines whether or not the vehicle speed of the vehicle 1 has become equal to or lower than a predetermined vehicle speed based on the detection result of the rotation speed of the wheel 3 by the wheel speed sensor 22 (step S501). ). Here, the predetermined vehicle speed is a vehicle speed set in advance, and is, for example, an extremely low speed equivalent to the speed at which a person walks, or 10 km / h. And when it is judged that the vehicle speed of the vehicle 1 became below the predetermined vehicle speed (step S501: Yes), the image generation part 401 for a display is the image of the vehicle 1 in the past position among the captured images acquired by the imaging part 15. A captured image obtained by imaging the surroundings by the imaging unit 15 is acquired (step S502). For example, the display image generation unit 401 may display the past position of the vehicle 1 at a past time (for example, several seconds) before the current time (or a distance set in advance from the current position of the vehicle 1 (for example, 2m) A captured image obtained by imaging the surroundings of the vehicle 1 at a previous position) by the imaging unit 15 is acquired. In the present embodiment, the display image generation unit 401 acquires a captured image including an image corresponding to the floor under the vehicle 1 at the current position. As a result, a display image including an image under the floor of the vehicle 1 at the current position can be generated, so that the situation under the floor of the vehicle 1 at the current position can be confirmed via the display device 8.

  In the present embodiment, the display image generation unit 401 generates a display image based on a captured image obtained by imaging the surroundings of the vehicle 1 at a past position by the imaging unit 15. What is necessary is just to produce | generate the image for a display based on the captured image obtained by imaging the circumference | surroundings of the vehicle 1. FIG. For example, the display image generation unit 401 may generate a display image based on a captured image obtained by imaging the surroundings of the vehicle 1 at the current position by the imaging unit 15.

  Next, the display image generation unit 401 generates a display image in which a fixed gazing point in the virtual space is viewed from the virtual viewpoint input via the operation input unit 10 based on the acquired captured image. (Step S503). In the present embodiment, the display image generation unit 401 pastes the acquired captured image on the camera image model. Then, the display image generation unit 401 defines a space including the camera image model and in which the vehicle image is arranged with respect to the current position as a virtual space. In the present embodiment, the display image generation unit 401 pastes a captured image obtained by capturing an image of the surroundings of the vehicle 1 with a single image capturing unit 15 to the camera image model. Thereby, unlike the case where a plurality of captured images obtained by imaging the periphery of the vehicle 1 by the plurality of imaging units 15 are pasted on the camera image model, it is possible to prevent the joint image from being generated on the captured image pasted on the camera image model. Therefore, it is possible to make it easier to confirm the positional relationship between the vehicle 1 and the surrounding obstacles. In the present embodiment, the display image generation unit 401 generates a display image based on the captured image of one imaging unit 15, but is obtained by imaging the surroundings of the vehicle 1 with the plurality of imaging units 15. A plurality of captured images may be pasted on a camera image model, and a space including the camera image model may be used as a virtual space.

  The display image output unit 402 outputs the display image generated by the display image generation unit 401 to the display device 8, and causes the display device 8 to display the display image (step S504). Thereafter, the display image generation unit 401 determines whether the vehicle 1 has stopped based on the detection result of the shift range by the shift sensor 21 (step S505). When it is determined that the shift range detected by the shift sensor 21 is not the parking range and the vehicle 1 is not stopped (step S505: No), the display image generating unit 401 rotates the wheel 3 by the wheel speed sensor 22. Based on the detection results of the number, it is determined whether or not the vehicle speed of the vehicle 1 is equal to or lower than a predetermined vehicle speed (step S506). When it is determined that the vehicle 1 is not stopped and the vehicle speed of the vehicle 1 is equal to or lower than the predetermined vehicle speed (step S506: Yes), the display image generation unit 401 uses the operation input unit 10 to create a new virtual It is determined whether or not a viewpoint has been input (step S508). When a preset time has elapsed without a new virtual viewpoint being input (step S508: No), the display image output unit 402 stops outputting the display image to the display device 8, and the display device 8 The display of the display image for is terminated (step S507).

  When a new virtual viewpoint is input (step S508: Yes), the display image generation unit 401 regenerates a display image in which a fixed gazing point in the virtual space is viewed from the new virtual viewpoint (step S508). S503). Then, the display image output unit 402 outputs the display image generated by the display image generation unit 401 to the display device 8, and causes the display device 8 to display the generated display image (step). S504). Thereby, until it is determined that the vehicle 1 has stopped, an occupant of the vehicle 1 simply inputs a virtual viewpoint via the operation input unit 10 and easily displays images of the gazing point from various angles. be able to.

  On the other hand, when it is determined that the shift range detected by the shift sensor 21 is the parking range and the vehicle 1 has stopped (step S505: Yes), and when the vehicle speed of the vehicle 1 is higher than the predetermined vehicle speed (step S506: No). The display image output unit 402 stops outputting the display image to the display device 8 and ends the display of the display image on the display device 8 (step S507). In the present embodiment, the display image generation unit 401 determines whether or not the vehicle 1 is stopped based on the detection result of the shift range by the shift sensor 21 in step S505, but the present invention is not limited to this. Instead, it may be determined whether or not the vehicle 1 is stopped based on the detection result of the rotation speed of the wheel 3 by the wheel speed sensor 22. Specifically, the display image generation unit 401 stops the vehicle 1 when a preset time (for example, 3 seconds) has elapsed while the rotation speed of the wheel 3 remains 0 by the wheel speed sensor 22. You may judge.

  Next, an example of display image generation processing by the vehicle 1 according to the present embodiment will be described with reference to FIGS. 6 and 7 are views for explaining an example of a camera image model used for generating a display image by the vehicle according to the present embodiment. 6 and 7, one direction parallel to the ground contact surface (road surface) of the tire of the vehicle 1 is defined as the Z direction, and a direction parallel to the ground contact surface of the tire of the vehicle 1 and perpendicular to the Z direction is defined as the X direction. A direction perpendicular to the ground is defined as a Y direction. FIG. 8 and FIG. 9 are diagrams for explaining an example of a camera image model and a vehicle image used for generating a display image in the vehicle according to the present embodiment.

  As shown in FIGS. 6 and 7, the display image generation unit 401 generates a camera image model S including a first surface S1 and a second surface S2. Here, the first surface S1 is a flat surface corresponding to the road surface on which the vehicle 1 is present. For example, the first surface S1 is an elliptical flat surface. The second surface S2 is a surface that is continuously provided on the outer side (outer edge) of the first surface S1 with respect to the first surface S1, and that gradually rises in the Y direction as the distance from the first surface S1 increases. For example, the second surface S2 is a surface that rises in an elliptical shape or a parabolic shape in the Y direction from the outside of the first surface S1. That is, the display image generation unit 401 generates a saddle-shaped or cylindrical three-dimensional surface as the camera image model S.

  In the present embodiment, the display image generation unit 401 generates a three-dimensional pasting surface having a flat first surface S1 and a curved second surface S2 as the camera image model S. As long as the sticking surface is generated as the camera image model S, the present invention is not limited to this. For example, the display image generation unit 401 includes a flat first surface S1, a second surface S2 of a flat surface that rises perpendicularly or gradually from the outside of the first surface S1 to the first surface S1, A three-dimensional affixing surface having a may be generated as the camera image model S.

  Next, the display image generation unit 401 pastes a captured image obtained by imaging the periphery of the vehicle 1 by the imaging unit 15 at the past position P1 to the camera image model S. In the present embodiment, the display image generation unit 401 includes coordinates (hereinafter referred to as three-dimensional coordinates) of points (hereinafter referred to as paste points) in the camera image model S represented by the world coordinate system with the past position P1 as the origin. And a coordinate table for associating the coordinates (hereinafter referred to as camera image coordinates) of the points in the captured image (hereinafter referred to as camera image points) to be pasted to the paste points of the three-dimensional coordinates. . Then, the display image generation unit 401 pastes the camera image point in the captured image to the attachment point of the three-dimensional coordinate associated with the camera image coordinate of the camera image point in the coordinate table. In the present embodiment, the display image generation unit 401 creates the coordinate table every time the internal combustion engine or electric motor of the vehicle 1 is started.

  Thereafter, the display image generation unit 401 arranges the camera image model S with the captured image pasted in the space around the vehicle 1 at the past position P1. Furthermore, the display image generation unit 401 generates, as a virtual space A, a space in which the vehicle image CG is arranged with respect to the current position P2 of the vehicle 1 in the space, as shown in FIGS. When the virtual space A is generated, the display image generation unit 401 starts from the front end of the vehicle image CG provided at the current position P2 of the vehicle 1 in the virtual space A, as shown in FIGS. 6, 8, and 9. A point dropped perpendicularly to the first surface S1 is set as the gazing point P3.

  Further, as shown in FIG. 6, the display image generation unit 401 is placed in a position away from the vehicle image CG in the virtual space A by a preset distance (for example, a distance corresponding to 1 to 2 m in the real space). The viewpoint P3 may be set. As a result, the gazing point P3 is set in advance at a position that becomes a blind spot when viewed from the occupant of the vehicle 1, so that the vehicle 1 can be operated even if the occupant of the vehicle 1 does not input the gazing point P3 via the operation input unit 10. The image at the position of 1 blind spot can be confirmed via the display device 8.

  Next, as shown in FIGS. 8 and 9, the display image generation unit 401 generates a display image in which the gazing point P3 is viewed from the virtual viewpoint P4 input from the operation input unit 10. As a result, a display image including an image under the floor of the current position P2 of the vehicle 1 can be generated. In addition, the positional relationship between the vehicle image CG included in the display image and the surrounding three-dimensional object can be made easier to understand. For example, as illustrated in FIG. 8, the display image generation unit 401 uses, as a display image, an image obtained by viewing the gazing point P3 in the virtual space A from the virtual viewpoint P4 that exists above the driver seat of the vehicle image CG. Generate. Alternatively, as shown in FIG. 9, the display image generation unit 401 has the gazing point P3 in the virtual space A above the vehicle image CG and on the right side when viewed from the vehicle image CG in the periphery of the vehicle image CG. An image viewed from the virtual viewpoint P4 is generated as a display image.

  In the present embodiment, the display image generation unit 401 arranges the three-dimensional model image of the vehicle 1 in the virtual space A as the vehicle image CG, but the two-dimensional model image of the vehicle 1 is the vehicle image. You may arrange | position in the virtual space A as CG. In the present embodiment, the display image generation unit 401 adjusts the transparency of the vehicle image CG so that the virtual space A can be seen through the vehicle image CG, and then displays the vehicle image CG in the virtual space. Place in A. At that time, the display image generation unit 401 makes the transmittance of the vehicle image CG other than the tire image TG higher than the transmittance of the tire image TG. Thereby, the position of the vehicle image CG in the virtual space A can be easily grasped from the tire image TG included in the vehicle image CG.

  In addition, a camera image model S to which a captured image obtained by imaging the periphery of the vehicle 1 at the past position P1 (for example, the front of the vehicle 1) with a wide-angle camera (for example, a camera having an angle of view of 180 °) is pasted. When the image of the virtual space A including the image is displayed on the display device 8 as it is, the image of the vehicle 1 (for example, the image of the front bumper of the vehicle 1) included in the captured image is reflected in the display image. May feel uncomfortable. On the other hand, in the present embodiment, the display image generation unit 401 provides a camera image model S with a gap from the past position P1 of the vehicle 1 toward the outside of the vehicle 1, thereby capturing a captured image. Since the image of the vehicle 1 included in the image can be prevented from being reflected in the display image, the passenger of the vehicle 1 can be prevented from feeling uncomfortable. Further, in the present embodiment, the display image generating unit 401 moves along with the movement of the current position P2 of the vehicle 1 (in other words, with the movement of the vehicle image CG corresponding to the movement of the vehicle 1). Move P3. As a result, when the current position P2 of the vehicle 1 moves, there is no need for the occupant of the vehicle 1 to input the gazing point P3 via the operation input unit 10, so that the display image can be displayed on the display device 8. The burden on the passenger of the vehicle 1 can be reduced.

  Next, an example of a display image generated in the vehicle 1 according to the present embodiment will be described with reference to FIGS. 10 and 11. 10 and 11 are diagrams illustrating an example of a display image generated in the vehicle according to the present embodiment.

  As shown in FIG. 10, the display image generation unit 401 generates a display image G1 when the fixed gazing point P3 in the virtual space is viewed from the virtual viewpoint P4 existing on the right side when viewed from the vehicle image CG. . At that time, as shown in FIG. 10, the display image generation unit 401 displays the display image G1 so that the position of the gazing point P3 in the display image G1 is positioned at the center in the display image G1. Generate. Then, the display image output unit 402 causes the display device 8 to display the generated display image G1. At this time, the display image generation unit 401 includes operation images SG1 to SG4 that are GUIs (Graphical User Interfaces) that allow the virtual viewpoint P4 to be input to the display image G1. Here, the operation image SG1 is a GUI that allows an instruction to move the virtual viewpoint P4 forward of the vehicle image CG. The operation image SG2 is a GUI that can instruct the movement of the virtual viewpoint P4 in the right direction when viewed from the vehicle image CG. The operation image SG3 is a GUI that allows an instruction to move the virtual viewpoint P4 to the rear of the vehicle image CG. The operation image SG4 is a GUI that can instruct the movement of the virtual viewpoint P4 in the left direction as viewed from the vehicle image CG.

  Thereafter, when the operation input unit 10 detects that the operation image SG3 is touch-operated, the display image generation unit 401 displays a fixed gazing point P3 in the virtual space as illustrated in FIG. A display image G2 viewed from the virtual viewpoint P4 moved rearward is generated. Thereby, since the virtual viewpoint P4 can be input simply by touching the operation image, the burden on the passenger of the vehicle 1 when the display image is displayed on the display device 8 can be further reduced. Then, the display image output unit 402 causes the display device 8 to display the generated display image G2. Also in this case, the display image generation unit 401 includes the operation images SG1 to SG4 with respect to the display image G2. Here, the virtual viewpoint P4 is input using the operation images SG1 to SG4. However, the present invention is not limited to this as long as the virtual viewpoint P4 is input using the operation input unit 10. For example, the display image generation unit 401 uses the operation input unit 10 to detect a flick input on the screen of the display device 8, and the virtual viewpoint P4 that has moved according to the detected flick input may be used as the input virtual viewpoint P4. good.

  As described above, according to the vehicle 1 according to the present embodiment, when the display image in which the gazing point around the vehicle 1 is viewed from the virtual viewpoint is displayed on the display device 8, the passenger of the vehicle 1 operates the operation input unit 10. Since the display image is displayed simply by inputting only the virtual viewpoint, the burden on the occupant of the vehicle 1 when the display image is displayed on the display device 8 can be reduced.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 8 ... Display apparatus, 14 ... ECU, 15 ... Imaging part, 14a ... CPU, 14b ... ROM, 14c ... RAM, 14d ... Display control part, 14f ... SSD, 401 ... Display image generation part, 402 ... Display image output unit.

Claims (5)

Based on a captured image obtained by imaging the surroundings of the vehicle by an imaging unit mounted on the vehicle, an operation input unit is used to display a fixed gazing point in a virtual space in which a vehicle image is provided in the space around the vehicle. A generation unit for generating a first image viewed from a virtual viewpoint input via
An output unit that outputs the first image generated by the generation unit to a display unit;
A peripheral monitoring device comprising:   The periphery monitoring device according to claim 1, wherein the generation unit uses a position away from a predetermined distance from the vehicle image as the gazing point.   The periphery monitoring device according to claim 1, wherein the generation unit moves the gazing point in accordance with the movement of the vehicle image corresponding to the movement of the vehicle.   The generation unit generates the first image including an operation image that allows the virtual viewpoint to be input, and when the new virtual viewpoint is input using the operation image, the generation unit starts from the new virtual viewpoint. The periphery monitoring device according to claim 1, wherein the first image viewed from a viewpoint is regenerated.   The periphery monitoring device according to any one of claims 1 to 4, wherein the vehicle image is a three-dimensional image of the vehicle.

Images