JP2018186387A - Display controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten confirmation time in the vicinity of a vehicle.SOLUTION: A display controller of an embodiment includes: an acquisition part for acquiring a plurality of captured image data from a plurality of imaging parts for imaging the vicinity of a vehicle; a generation part which, from the captured image data acquired by the acquisition part, generates the image data including an imaging region where the outside is imaged from at least two surfaces opposite to each other, out of surfaces representing the periphery of the vehicle, and a region where the vehicle exists; and a display processing part for displaying so that the imaging region and the region where the vehicle exists move in the direction perpendicular to the direction connecting two surfaces when switching display of the imaging data generated by the generation part with passage of time.SELECTED DRAWING: Figure 4

Description

  Embodiments described herein relate generally to a display control apparatus.

  Conventionally, a plurality of imaging units provided around the vehicle image the periphery of the vehicle, combine the plurality of captured image data, and display the combined image data. There is technology to make the driver recognize.

JP 2011-66763 A

  By the way, if the entire periphery of the vehicle is displayed on the display unit, it is difficult to grasp the detailed situation around the vehicle. Therefore, a technique for displaying an animation so as to go around the vehicle has been proposed.

  However, when the display is performed so as to go around the vehicle, in order to check the surroundings of the vehicle, the driver needs to watch until it makes a round, and there is a problem that it takes time.

  One embodiment of the present invention has been made in view of the above, and provides a display control device capable of shortening the time required to confirm the surroundings of a vehicle.

  As an example, the display control apparatus according to the embodiment includes an acquisition unit that acquires a plurality of captured image data from a plurality of imaging units that capture the periphery of the vehicle, and at least two opposing surfaces among the surfaces that represent the periphery of the vehicle. Generated by the generation unit with the passage of time, a generation unit that generates image data including the imaging region obtained by imaging the outside and the region where the vehicle exists, from the captured image data acquired by the acquisition unit A display processing unit configured to display the imaging region and the region where the vehicle exists so as to move in a direction perpendicular to a direction connecting the two surfaces when the display of the image data is switched. Therefore, since the time until all the surroundings of the vehicle are displayed can be shortened, the driver can quickly confirm the surroundings of the vehicle.

  As an example, the display control apparatus according to the embodiment projects a plurality of captured image data as an imaging region onto a virtual projection plane in a three-dimensional virtual space based on a position where the vehicle is present, and the vehicle is projected onto the virtual projection plane. Each time the gazing point is moved in the direction perpendicular to the direction connecting the two surfaces with the viewpoint fixed, the virtual projection plane is further provided. Then, the image data including the imaging region and the region where the vehicle exists is generated, and the display processing unit displays the image data generated by the generation unit, so that the imaging region and the vehicle exist. The displayed area is displayed so as to move in a direction perpendicular to the direction connecting the two surfaces. Therefore, since the time until all the surroundings of the vehicle are displayed can be shortened, the driver can quickly confirm the surroundings of the vehicle.

  As an example, the display control apparatus according to the embodiment projects a plurality of captured image data as an imaging region onto a virtual projection plane in a three-dimensional virtual space based on a position where the vehicle is present, and the vehicle is projected onto the virtual projection plane. Each time the viewpoint and the gazing point are moved in a direction perpendicular to the direction connecting the two planes, the generation unit starts from the virtual projection plane. And the region where the vehicle exists, and the display processing unit displays the image data generated by the generation unit, so that the imaging region and the region where the vehicle exists Is displayed so as to move in the direction perpendicular to the direction connecting the two surfaces. Therefore, since the time until all the surroundings of the vehicle are displayed can be shortened, the driver can quickly confirm the surroundings of the vehicle.

FIG. 1 is a perspective view illustrating an example of a state in which a part of a passenger compartment of a vehicle on which the display control device according to the embodiment is mounted is seen through. FIG. 2 is a plan view (bird's eye view) illustrating an example of a vehicle on which the display control device according to the embodiment is mounted. FIG. 3 is a block diagram illustrating an example of a configuration of a display control system including the display control device according to the embodiment. FIG. 4 is a block diagram illustrating a functional configuration of the ECU as the display control apparatus according to the embodiment. FIG. 5 is an exemplary schematic diagram illustrating projection of captured image data onto a virtual projection plane in the display control system according to the embodiment. FIG. 6 is a schematic and exemplary side view showing a vehicle and a virtual projection plane in the display control system according to the embodiment. FIG. 7 is a diagram illustrating movement of a gazing point after fixing the viewpoint according to the embodiment. FIG. 8 is a diagram illustrating movement of a gazing point after fixing the viewpoint according to the embodiment. FIG. 9 is a diagram illustrating changes in viewpoint image data displayed by the display processing unit according to the embodiment. FIG. 10 is a flowchart illustrating a procedure of display processing in the display control system according to the embodiment. FIG. 11 is a diagram illustrating movement of the viewpoint and the gazing point in the modification.

  Hereinafter, exemplary embodiments of the present invention are disclosed. The configuration of the embodiment shown below and the operations, results, and effects brought about by the configuration are examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and at least one of various effects based on the basic configuration and derivative effects can be obtained. .

  In the present embodiment, the vehicle 1 equipped with a display control device (display control system) may be, for example, an automobile using an internal combustion engine (not shown) as a drive source, that is, an internal combustion engine automobile, or an electric motor (not shown). It may be a vehicle as a drive source, that is, an electric vehicle, a fuel cell vehicle, or the like. Moreover, the hybrid vehicle which uses both of them as a drive source may be sufficient, and the vehicle provided with the other drive source may be sufficient. Further, the vehicle 1 can be mounted with various transmissions, and various devices necessary for driving the internal combustion engine and the electric motor, such as systems and components, can be mounted. As a driving system, a driving force can be transmitted to all four wheels 3 and all four wheels can be used as driving wheels. Various methods, numbers, layouts, and the like of devices related to driving of the wheel 3 can be set. Further, the driving method is not limited to the four-wheel driving method, and may be a front wheel driving method or a rear wheel driving method, for example.

  As illustrated in FIG. 1, the vehicle body 2 constitutes a passenger compartment 2 a in which a passenger (not shown) gets. In the passenger compartment 2a, a steering section 4, an acceleration operation section 5, a braking operation section 6, a shift operation section 7 and the like are provided in a state facing the driver's seat 2b as a passenger. The steering unit 4 is, for example, a steering wheel protruding from the dashboard 24, the acceleration operation unit 5 is, for example, an accelerator pedal positioned under the driver's feet, and the braking operation unit 6 is, for example, a driver's foot It is a brake pedal located under the foot, and the speed change operation unit 7 is, for example, a shift lever protruding from the center console. The steering unit 4, the acceleration operation unit 5, the braking operation unit 6, the speed change operation unit 7 and the like are not limited to these.

  A display device 8 and an audio output device 9 are provided in the passenger compartment 2a. The display device 8 is, for example, an LCD (liquid crystal display) or an OELD (organic electroluminescent display). The audio output device 9 is, for example, a speaker. The display device 8 is covered with a transparent operation input unit 10 such as a touch panel. The occupant can visually recognize an image displayed on the display screen of the display device 8 via the operation input unit 10. In addition, the occupant can execute an operation input by touching, pushing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8. . The display device 8, the audio output device 9, the operation input unit 10, and the like are provided, for example, in the monitor device 11 that is located in the vehicle width direction of the dashboard 24, that is, the central portion in the left-right direction. The monitor device 11 can have an operation input unit (not shown) such as a switch, a dial, a joystick, and a push button. Further, a sound output device (not shown) can be provided at another position in the passenger compartment 2a different from the monitor device 11, and sound is output from the sound output device 9 of the monitor device 11 and other sound output devices. be able to. Note that the monitor device 11 can be used also as, for example, a navigation system or an audio system.

  As illustrated in FIGS. 1 and 2, the vehicle 1 is, for example, a four-wheeled vehicle, and includes two left and right front wheels 3 </ b> F and two right and left rear wheels 3 </ b> R. All of these four wheels 3 can be configured to be steerable. As illustrated in FIG. 3, the vehicle 1 includes a steering system 13 that steers at least two wheels 3. The steering system 13 has a torque sensor 13b. The steering system 13 is electrically controlled by an ECU 14 (electronic control unit) or the like to operate an actuator. The steering system 13 is, for example, an electric power steering system, an SBW (steer by wire) system, or the like. The steering system 13 supplements the steering force by adding torque, that is, assist torque to the steering unit 4 by the actuator, or steers the wheel 3 by the actuator. In this case, the actuator may steer one wheel 3 or may steer a plurality of wheels 3. Moreover, the torque sensor 13b detects the torque which a driver | operator gives to the steering part 4, for example.

  Further, as illustrated in FIG. 2, for example, four imaging units 15 a to 15 d are provided in the vehicle body 2 as the plurality of imaging units 15. The imaging unit 15 is a digital camera including an imaging element such as a charge coupled device (CCD) or a CMOS image sensor (CIS). The imaging unit 15 can output moving image data (captured image data) at a predetermined frame rate. The imaging unit 15 has a wide-angle lens or a fisheye lens, respectively, and can image a range of, for example, 140 ° to 220 ° in the horizontal direction. Further, the optical axis of the imaging unit 15 may be set obliquely downward. Therefore, the imaging unit 15 sequentially captures the external environment around the vehicle 1 including the road surface on which the vehicle 1 is movable and surrounding objects (obstacles, rocks, depressions, puddles, dredging, etc.), and takes the captured image data. Output.

  The imaging unit 15a is located at, for example, the rear end 2e of the vehicle body 2, and is provided on a wall portion below the rear window of the rear hatch door 2h. The imaging unit 15b is located, for example, at the right end 2f of the vehicle body 2 and provided on the right door mirror 2g. The imaging unit 15c is located, for example, on the front side of the vehicle body 2, that is, the front end 2c in the vehicle front-rear direction, and is provided on a front bumper, a front grill, or the like. The imaging unit 15d is located, for example, at the left end 2d of the vehicle body 2 and is provided on the left door mirror 2g.

  As illustrated in FIG. 3, in the display control system 100 (display control device), in addition to the ECU 14, the monitor device 11, the steering system 13, etc., the brake system 18, the steering angle sensor 19, the accelerator sensor 20, the shift A sensor 21, a wheel speed sensor 22, and the like are electrically connected via an in-vehicle network 23 as an electric communication line. The in-vehicle network 23 is configured as a CAN (controller area network), for example. The ECU 14 can control the steering system 13, the brake system 18, and the like by sending a control signal through the in-vehicle network 23. The ECU 14 also detects the detection results of the torque sensor 13b, the brake sensor 18b, the rudder angle sensor 19, the accelerator sensor 20, the shift sensor 21, and the wheel speed sensor 22 through the in-vehicle network 23, and operation signals from the operation input unit 10 and the like. You can receive, etc.

  The ECU 14 includes, for example, a CPU 14a (central processing unit), a ROM 14b (read only memory), a RAM 14c (random access memory), a display control unit 14d, an audio control unit 14e, an SSD 14f (solid state drive, flash memory), and the like. ing. The CPU 14a reads a program stored (installed) in a non-volatile storage device such as the ROM 14b, and executes arithmetic processing according to the program. For example, the CPU 14a executes image processing related to an image displayed on the display device 8. For example, the CPU 14a performs arithmetic processing or image processing on the captured image data captured by the imaging unit 15 to detect whether or not there is a specific region to be noted on the predicted course of the vehicle 1, For example, the user (driver or passenger) is notified of the presence of a specific region by changing the display mode of a course indicator (predicted course line) indicating the estimated travel direction of the vehicle 1.

  The RAM 14c temporarily stores various types of data used in computations by the CPU 14a. The display control unit 14d mainly performs image processing using the captured image data obtained by the imaging unit 15 and image processing of image data displayed on the display device 8 (as an example) in the arithmetic processing performed by the ECU 14. Executes image composition). In addition, the voice control unit 14 e mainly executes processing of voice data output from the voice output device 9 among the calculation processes in the ECU 14. The SSD 14f is a rewritable nonvolatile storage unit, and can store data even when the ECU 14 is powered off. The CPU 14a, the ROM 14b, the RAM 14c, and the like can be integrated in the same package. Further, the ECU 14 may have a configuration in which another logical operation processor such as a DSP (digital signal processor) or a logic circuit is used instead of the CPU 14a. Further, an HDD (hard disk drive) may be provided instead of the SSD 14f, and the SSD 14f and the HDD may be provided separately from the peripheral monitoring ECU 14.

  The brake system 18 includes, for example, an anti-lock brake system (ABS) that suppresses brake locking, an anti-slip device (ESC: electronic stability control) that suppresses side slip of the vehicle 1 during cornering, and enhances braking force ( Electric brake system that executes brake assist, BBW (brake by wire), etc. The brake system 18 applies a braking force to the wheels 3 and thus to the vehicle 1 via the actuator. The brake sensor 18b is a sensor that detects the position of the movable part of the braking operation unit 6, for example. The brake sensor 18b can detect the position of a brake pedal as a movable part. The brake sensor 18b includes a displacement sensor.

  The steering angle sensor 19 is, for example, a sensor that detects the steering amount of the steering unit 4 such as a steering wheel. The rudder angle sensor 19 is configured using, for example, a hall element. The ECU 14 obtains the steering amount of the steering unit 4 by the driver, the steering amount of each wheel 3 during automatic steering, and the like from the steering angle sensor 19 and executes various controls. The rudder angle sensor 19 detects the rotation angle of the rotating part included in the steering unit 4. The rudder angle sensor 19 is an example of an angle sensor.

  The accelerator sensor 20 is, for example, a sensor that detects the position of the movable part of the acceleration operation unit 5. The accelerator sensor 20 can detect the position of an accelerator pedal as a movable part. The accelerator sensor 20 includes a displacement sensor.

  The shift sensor 21 is, for example, a sensor that detects the position of the movable part of the speed change operation unit 7. The shift sensor 21 can detect the position of a lever, arm, button, or the like as a movable part. The shift sensor 21 may include a displacement sensor or may be configured as a switch.

  The wheel speed sensor 22 is a sensor that detects the amount of rotation of the wheel 3 and the number of rotations per unit time. The wheel speed sensor 22 outputs a wheel speed pulse number indicating the detected rotation speed as a sensor value. The wheel speed sensor 22 may be configured using, for example, a hall element. The ECU 14 calculates the amount of movement of the vehicle 1 based on the sensor value acquired from the wheel speed sensor 22 and executes various controls. Note that the wheel speed sensor 22 may be provided in the brake system 18. In that case, the ECU 14 acquires the detection result of the wheel speed sensor 22 via the brake system 18.

  The configuration, arrangement, electrical connection form, and the like of the various sensors and actuators described above are examples, and can be set (changed) in various ways.

  The CPU 14a included in the ECU 14 displays the environment around the vehicle 1 based on the captured image data as described above. In order to realize this function, the ECU 14 includes various modules as shown in FIG. The ECU 14 includes, for example, an acquisition unit 401, an image composition unit 402, a three-dimensional processing unit 403, a viewpoint image generation unit 404, and a display processing unit 405. These modules can be realized by reading a program installed and stored in a storage device such as the ROM 14b and executing it.

  The SSD 14f stores, for example, vehicle shape data that represents the shape of the vehicle 1 in a plane. The vehicle shape data holds the shape of the interior of the vehicle 1 as well as the contour of the vehicle 1 including the wheels 3.

  The acquisition unit 401 acquires information necessary for displaying the surroundings of the vehicle 1. For example, the acquisition unit 401 acquires a plurality of captured image data from the plurality of imaging units 15 that capture the periphery of the vehicle 1.

  The acquisition unit 401 acquires operation data representing an operation performed by the driver via the operation input unit 10. As the operation data, for example, a display start operation of the screen displayed on the display device 8 or a viewpoint change operation for the screen displayed on the display device 8 can be considered.

  The image composition unit 402 connects the data of a plurality of captured images acquired by the acquisition unit 401, that is, the data of the plurality of captured images captured by the plurality of image capturing units 15 by combining their boundary portions, Data of one captured image is generated.

  The three-dimensional processing unit 403 includes a virtual projection unit 411 and an area setting unit 412, and is synthesized by the image synthesis unit 402 on a virtual projection plane in a three-dimensional virtual space with reference to the position where the vehicle 1 exists. The captured image data is projected, and an area where the vehicle 1 exists is set on the virtual projection plane. As a result, a plurality of captured image data acquired by the acquisition unit 401 is projected onto the virtual projection plane.

  The virtual projection unit 411 generates virtual projection image data obtained by projecting the captured image data on a virtual projection plane surrounding the periphery of the vehicle 1, which is determined based on the position where the vehicle 1 exists. FIG. 5 is an exemplary schematic diagram illustrating projection of the captured image Ic onto the virtual projection plane Sp in the display control system 100. In the example of FIG. 5, the virtual projection plane Sp has a bottom surface Spg along the ground Gr and a side surface Sps rising from the bottom surface Spg, that is, the ground Gr. The ground Gr is a horizontal plane orthogonal to the vertical direction Z of the vehicle 1 and is also a ground contact surface of the wheel 3. The bottom surface Spg is a substantially circular flat surface and is a horizontal surface with the vehicle 1 as a reference. The side surface Sps is a curved surface in contact with the bottom surface Spg. As shown in FIG. 5, the shape of the virtual cross section of the side surface Sps passing through the center Gc of the vehicle 1 and perpendicular to the vehicle 1 is, for example, an ellipse or a parabola. The side surface Sps is configured as, for example, a rotating surface around the center line CL along the vertical direction (Z direction) of the vehicle 1 through the center Gc of the vehicle 1 and surrounds the periphery of the vehicle 1. The virtual projection unit 411 calculates a virtual projection image Ip obtained by projecting the captured image Ic onto the virtual projection plane Sp. When the captured image Ic is projected onto the ground Gr, the image becomes longer as the distance from the imaging unit 15 increases, and may appear longer than the actual length in the output image. As can be seen from FIG. 5, the virtual projection image Ip projected onto the side surface Sps rising from the ground Gr (bottom surface Spg) is shorter than that projected onto the ground Gr. It is suppressed that the image is projected longer than the length. In the present embodiment, the imaging unit 15 cannot capture an area 501 where the vehicle 1 exists.

  Therefore, the region setting unit 412 sets the region 501 as a region where the vehicle 1 exists, and superimposes two-dimensional vehicle shape data on the region 501.

  The viewpoint image generation unit 404 generates viewpoint image data obtained by viewing the virtual projection image Ip and vehicle shape data projected onto the virtual projection plane Sp from a predetermined virtual viewpoint. FIG. 6 is a schematic and exemplary side view showing the vehicle 1 and the virtual projection plane Sp (Sp0) in the display control system 100. FIG. As shown in FIG. 6, the viewpoint image generation unit 404 displays a virtual projection image Ip (not shown in FIG. 6) and vehicle shape data 601 projected on the virtual projection plane Sp obliquely downward from a predetermined viewpoint Ep. It is converted into viewed viewpoint image data (not shown in FIG. 6). The viewpoint Ep is set near the ceiling of the vehicle 1 and at the center of the vehicle 1. Note that the vehicle body 2 shown in FIG. 6 is shown for explanation, and is not included in the viewpoint image data.

  As described above, the viewpoint image generation unit 404 of the present embodiment represents the surroundings of the vehicle 1 from the virtual projection image Ip and the vehicle shape data 601 projected on the virtual projection plane Sp by setting the viewpoint and the gazing point. The viewpoint image data including an imaging region in which the outside is imaged from at least two opposite side surfaces (the left side surface and the right side surface) and a part of the region 501 where the vehicle 1 exists is generated.

  The surface representing the periphery of the vehicle 1 may be, for example, the exterior surface of the vehicle 1 or the body surface of the vehicle 1 that is the outer surface of the vehicle 1. In the present embodiment, an example in which the outside is photographed from each of the left side surface and the right side surface, which are exterior surfaces of the vehicle 1, will be described. However, this embodiment does not limit at least two opposing surfaces to the left side surface and the right side surface, and may be, for example, the front surface and the rear surface. That is, conventionally, since the outside is displayed from each of the front surface, the rear surface, the right side surface, and the left side surface, it takes time to check the periphery of the vehicle 1. On the other hand, in the present embodiment, the range that can be confirmed is widened by displaying the outside from at least two opposing faces at a time, so that the time required for confirmation can be shortened.

  Next, the viewpoint Ep and the gaze point used for generating viewpoint image data will be described. 7 and 8 are diagrams illustrating the movement of the gazing point after fixing the viewpoint. The example illustrated in FIG. 7 is a diagram illustrating the movement of the gazing point in the side surface direction of the vehicle body 2, and the example illustrated in FIG. 8 is a diagram illustrating the movement of the gazing point from the upper surface direction of the vehicle body 2. It is.

  As shown in FIGS. 7 and 8, the viewpoint image generation unit 404 of the present embodiment fixes the viewpoint Ep and moves it to the virtual projection plane Sp every time the gazing point is moved in the backward direction 711 of the vehicle 1. From the projected virtual projection image Ip and the vehicle shape data 601, viewpoint image data including an imaging region and a part of the region 501 where the vehicle 1 exists is generated.

  In the example shown in FIG. 7, the gazing point moves in the backward direction 711, in other words, from the coordinate 701 to the coordinate 703 via the coordinate 702. The viewpoint image generation unit 404 generates viewpoint image data every time the gazing point is moved in the backward direction 711.

  In other words, in the present embodiment, at least the imaging region obtained by imaging the outside from the left side surface and the right side surface of the vehicle 1 is displayed at a time, and therefore, the rear direction 711 is perpendicular to the direction connecting the left side surface and the right side surface. By switching the display in the forward direction 712, the entire periphery of the vehicle 1 can be displayed.

  Thereafter, the gazing point moves in the forward direction 712, in other words, from the coordinate 703 to the coordinate 701 via the coordinate 702. The viewpoint image generation unit 404 generates viewpoint image data every time the gazing point is moved in the forward direction 712.

  Conventionally, there has been a technique for displaying the surroundings of the vehicle so that the imaging unit 15 circulates in order to make the driver recognize the situation around the vehicle. However, for example, when displaying the surroundings of the vehicle clockwise from the front direction of the vehicle, the display is performed in the order of the front direction of the vehicle, the right direction of the vehicle, the rear direction of the vehicle, and the left direction of the vehicle. Become. In this case, it takes time for the driver to grasp the situation around the vehicle. In this example, since the situation around the vehicle is displayed from the right direction, it takes time to recognize the situation on the left side of the vehicle.

  On the other hand, the viewpoint image generation unit 404 of the present embodiment does not generate a plurality of viewpoint image data so that the imaging unit circulates around the vehicle 1, but displays the displayed area like a pendulum. In other words, a plurality of viewpoint image data are generated so as to move the gazing point in a straight line direction. Thereby, the time until all the conditions around the vehicle 1 are displayed can be shortened.

  Returning to FIG. 4, the display processing unit 405 of the present embodiment switches the display of the viewpoint image data generated by the viewpoint image generation unit 404 as time passes. The display processing unit 405 switches the display so that the imaging region and the region where the vehicle 1 exists are backward or forward (parallel to the ground contact surface of the vehicle 1 and connect the right side surface and the left side surface. Display to move in the vertical direction).

  FIG. 9 is a diagram illustrating changes in viewpoint image data displayed by the display processing unit 405. The viewpoint image data 901 to 906 shown in FIG. 9 is generated by the viewpoint image generation unit 404, and is an image in which the displayed area is changed based on the fixed viewpoint Ep and the movement of the gazing point. A group.

  In the viewpoint image data 901, the area 911 is synthesized from the captured image data captured by the imaging unit 15c, the area 912 is synthesized from the captured image data captured by the imaging unit 15d, and the area 913 is synthesized by the imaging unit 15b. It is synthesized from the captured image data. In the area 914, two-dimensional vehicle shape data is superimposed by the area setting unit 412. In the region 914, the current traveling direction of the vehicle 1 can be recognized by the tread pattern represented on the wheels 931 and 932 of the vehicle shape data.

  Further, the shape of the transmitted vehicle 1 may be superimposed on the areas 912 and 913. In the example illustrated in FIG. 9, the shape data 921, 922, 923, and 924 of the transmitted wheel 3 is displayed in a superimposed manner. Thereby, the driver | operator can recognize the positional relationship of the wheel 3 and the circumference | surroundings.

  In the example shown in FIG. 9, the region 912 is made wider than the region 913. Thereby, the driver | operator can confirm the condition on the opposite side to a driver's seat. Note that which of the region 913 and the region 912 is set wider is determined according to the embodiment.

  The viewpoint image data 902 is image data generated by the viewpoint image generation unit 404 from the viewpoint image data 901 based on the gazing point moved to the rear of the vehicle 1.

  Similarly, the viewpoint image data 903 to 906 are image data generated by the viewpoint image generation unit 404 based on the gazing point moved to the rear of the vehicle from the previously displayed viewpoint image data. The viewpoint image data 901 to viewpoint image data 906 are image data generated at a reduction / enlargement ratio that always includes some of the wheels of the vehicle 1 when displayed on the display device 8. Thereby, when the driver refers to the viewpoint image data, the traveling direction of the vehicle 1 and the displayed position can be specified from the wheels shown in the viewpoint image data.

  In the viewpoint image data 906, the area 915 is synthesized from the captured image data captured by the imaging unit 15a. Similar to the viewpoint image data 901, the regions 912 and 913 are synthesized from the captured image data captured by the imaging units 15d and 15b.

  The display from the viewpoint image data 901 to the viewpoint image data 906 corresponds to the display according to the backward direction 711 in FIG. In this embodiment, since all the surrounding conditions of the vehicle 1 are displayed in the display from the viewpoint image data 901 to the viewpoint image data 906, the driver can immediately grasp the surrounding conditions of the vehicle.

  Thereafter, as display according to the forward direction (the reverse direction to the backward direction 711) 712, viewpoint image data 906, viewpoint image data 905, viewpoint image data 904, viewpoint image data 903, viewpoint image data 902, and viewpoint image data 901 are sequentially displayed. Is displayed. Note that description of the display is omitted.

  Next, display processing in the display control system 100 (ECU 14) of the present embodiment will be described. FIG. 10 is a flowchart showing a procedure of the above-described processing in the display control system 100 of the present embodiment.

  First, the acquisition unit 401 of the ECU 14 acquires information indicating that a display operation has been received from the operation input unit 10 (S1001). By acquiring the information, a process for displaying is started.

  Next, the acquisition part 401 acquires the captured image data which imaged the circumference | surroundings of the vehicle 1 from the four imaging parts 15a-15d (S1002).

  Then, the image composition unit 402 synthesizes a plurality of captured image data acquired in S1002 to generate one captured image data (S1003).

  The three-dimensional processing unit 403 projects the combined captured image data on a three-dimensional virtual projection plane, and sets an area where the vehicle 1 exists on the three-dimensional virtual projection plane (S1004). In this embodiment, two-dimensional vehicle shape data is superimposed as the setting.

  Then, the viewpoint image generation unit 404 fixes the viewpoint Ep and captures the outside from the vehicle 1 from the virtual projection plane every time the gazing point is moved in the backward direction (or forward direction) of the vehicle 1. And viewpoint image data including the region where the vehicle exists (S1005).

  The display processing unit 405 displays the viewpoint image data generated by the viewpoint image generation unit 404 (S1006).

  Then, the display processing unit 405 determines whether to end the display of the viewpoint image data (S1007). Whether or not to end the display is, for example, whether or not a display stop operation has been received from the driver, or whether or not the movement of the gazing point according to the rear direction 711 and the front direction 712 in FIG. 7 has ended. Judgment based on.

  When the display processing unit 405 determines not to end the display (S1007: No), the process starts from S1005, and the viewpoint image generation unit 404 starts from the displayed viewpoint image data in the backward direction (or forward direction) of the vehicle 1. The viewpoint image data in which the gazing point is moved is generated.

  On the other hand, when the display processing unit 405 determines to end the display (S1007: Yes), the display of the viewpoint image data is ended. Thereafter, the display processing unit 405 may display overhead image data or the like showing the surroundings of the vehicle 1.

(Modification)
In the embodiment described above, the example in which the viewpoint Ep is fixed has been described. However, the method is not limited to the method of fixing the viewpoint Ep, and the viewpoint may be moved together with the movement of the gazing point. Therefore, in the modification, an example in which the gazing point and the viewpoint are moved will be described. Note that the configuration of the display control system is the same as that of the embodiment, and a description thereof is omitted.

  FIG. 11 is a diagram illustrating movement of the viewpoint and the gazing point in the modification. The example shown in FIG. 11 is a diagram illustrating the point of sight and the movement of the viewpoint in the side surface direction of the vehicle body 2. In the example shown in FIG. 11, as in the above-described embodiment, after moving the gazing point in the backward direction 1111, the gazing point is moved in the forward direction 1112. Different from the embodiment described above.

  As shown in FIG. 11, in this modification, the viewpoint Ep <b> 1 is fixed while moving from the gazing point 1101 to the gazing point 1102. Then, while moving from the gazing point 1102 to the gazing point 1103, the movement is performed in the order of the viewpoints Ep1, Ep2, Ep3, Ep4, Ep5, and Ep6. Thereafter, the camera moves from the gazing point 1103 to the gazing point 1104 with the viewpoint Ep6 fixed.

  As described above, the viewpoint image generation unit 404 is forward or rearward of the vehicle 1 (a direction parallel to the ground contact surface of the vehicle 1 and perpendicular to a direction connecting the right side surface and the left side surface of the vehicle 1). Each time the viewpoint and the gazing point are moved, viewpoint image data including the imaging area and the area where the vehicle exists is generated from the virtual projection plane generated by the three-dimensional processing unit 403.

  Then, the display processing unit 405 displays the viewpoint image data generated in the viewpoint image generation unit 404 so that the imaging region and the region where the vehicle 1 exists are in the front direction or the rear direction of the vehicle 1. It will be displayed to move.

  In this modification, the same effect as that of the embodiment can be obtained by performing the above-described processing.

  In the embodiment and the modification described above, the imaging region and the vehicle shape data (the region where the vehicle 1 exists) whose outside is represented from the right side surface and the left side surface of the vehicle 1 are displayed, and the imaging region and the vehicle shape are displayed. The example in which the data is displayed so as to move in the forward direction or the backward direction of the vehicle 1 has been described. However, the embodiment and the modification are not limited to such a display method. For example, the imaging region and the vehicle shape data that are represented from the front and rear surfaces of the vehicle 1 are displayed, and the imaging region and the vehicle shape data are displayed so as to move rightward or leftward of the vehicle 1. May be.

  As described above, it is only necessary to display an imaging region obtained by photographing the outside from at least two opposing surfaces among the surfaces representing the periphery of the vehicle 1 so as to move in a direction perpendicular to the direction connecting the two surfaces. . By performing the process, the time for the driver to check the vicinity of the vehicle can be shortened.

  In the embodiment and the modification described above, the example in which the vehicle shape data is superimposed on the area where the vehicle exists in the viewpoint image data has been described. However, the method is not limited to the method of superimposing the vehicle shape data. For example, nothing may be displayed or other image data may be displayed.

  In the embodiment and the modification described above, the display control system displays the viewpoint image data with the above-described configuration, so that the driver can shorten the time until the entire periphery of the vehicle is displayed. Confirmation of the periphery of the can be realized.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 14 ... ECU, 100 ... Display control system, 401 ... Acquisition part, 402 ... Image composition part, 403 ... Three-dimensional processing part, 404 ... Viewpoint image generation part, 405 ... Display processing part, 411 ... Virtual projection part 412... Area setting unit.

Claims (3)

An acquisition unit that acquires a plurality of captured image data from a plurality of imaging units that capture the periphery of the vehicle;
The imaging in which the acquisition unit has acquired image data including an imaging area in which the outside is captured from at least two opposing faces among the surfaces representing the periphery of the vehicle and an area in which the vehicle is present A generating unit that generates image data;
When the display of the image data generated by the generation unit is switched over time, the imaging region and the region where the vehicle exists are in a direction perpendicular to the direction connecting the two surfaces. A display processing unit for displaying to move,
A display control device. An area in which the plurality of captured image data is projected as the imaging area onto a virtual projection plane in a three-dimensional virtual space based on a position where the vehicle exists, and the vehicle exists on the virtual projection plane A three-dimensional processing unit for setting
An area where the imaging area and the vehicle exist from the virtual projection plane each time the generation unit moves the gazing point in a direction perpendicular to a direction connecting the two surfaces with the viewpoint fixed. And generating the image data including
The display processing unit displays the image data generated by the generation unit so that the imaging region and the region where the vehicle exists are in a direction perpendicular to the direction connecting the two surfaces. Display to move,
The display control apparatus according to claim 1. An area in which the plurality of captured image data is projected as the imaging area onto a virtual projection plane in a three-dimensional virtual space based on a position where the vehicle exists, and the vehicle exists on the virtual projection plane A three-dimensional processing unit for setting
The generation unit includes, from the virtual projection plane, the imaging region and the region where the vehicle exists each time the viewpoint and the gazing point are moved in a direction perpendicular to the direction connecting the two surfaces. Generating the image data,
The display processing unit displays the image data generated by the generation unit so that the imaging region and the region where the vehicle exists are in a direction perpendicular to the direction connecting the two surfaces. Display to move,
The display control apparatus according to claim 1.

Images