JP2018186432A - Display controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an intuitive operation.SOLUTION: A display controller of an embodiment includes: as an example, an acquisition part for acquiring a plurality of captured image data from a plurality of imaging parts for capturing an image around a vehicle; a display processing part which displays a three-dimensional shape model representing the vehicle in a first display region and displays the peripheral image data representing the vicinity of the vehicle based on the captured image data acquired by the acquisition part according to the direction of the vehicle of the three-dimensional shape mode displayed in the first display region; and an operation receiving part for receiving operation for changing the direction of the three-dimensional shape model displayed in the first display region displayed by the display processing part. The display processing part switches the peripheral image data displayed in the second display region according to the operation for changing the direction received by the operation receiving part.SELECTED DRAWING: Figure 4

Description

  Embodiments described herein relate generally to a display control apparatus.

  Conventionally, a plurality of imaging units provided around the vehicle image the periphery of the vehicle, synthesize the plurality of captured image data, and display the synthesized captured image data. There is a technology that makes the driver recognize.

  At that time, there is a technique of switching the periphery of the vehicle to be displayed by superimposing display data (for example, a camera icon) for changing the viewpoint on the image data and receiving an operation such as pressing the display data.

International Publication No. 2013/053590

  However, in the prior art, it is often difficult to grasp what kind of switching has been performed by pressing the display data, and thus it has been desired to realize an intuitive display switching operation.

  As an example, the display control device of the embodiment displays an acquisition unit that acquires a plurality of captured image data from a plurality of imaging units that capture the periphery of the vehicle, and a three-dimensional shape model representing the vehicle in the first display area. Peripheral image data representing the periphery of the vehicle based on the captured image data acquired by the acquisition unit according to the orientation of the vehicle of the three-dimensional shape model displayed in the first display region is displayed in the second display region. And a display processing unit that receives an operation for changing the orientation of the three-dimensional shape model displayed in the first display area by the display processing unit. The peripheral image data displayed in the second display area is switched in accordance with the operation of changing the orientation accepted by the user. Therefore, as an example, in order to switch the display of the peripheral image data displayed in the second display area in conjunction with the change of the orientation of the three-dimensional shape model displayed in the first display area, an intuitive operation is performed. realizable.

  In the display control apparatus of the embodiment, for example, the acquisition unit further acquires inclination information indicating the inclination of the vehicle, and the display processing unit further includes an inclination corresponding to the inclination information in the first display area. In addition to displaying the three-dimensional model of the vehicle and the scale information indicating the degree of the inclination, the scale information rotated according to the direction according to the operation of changing the direction received by the operation receiving unit Change the position where is displayed. Therefore, as an example, since the inclination of the vehicle is displayed in the first display area, the situation around the vehicle can be recognized in more detail.

  In addition, as an example, the display control apparatus according to the embodiment is configured so that, based on a viewpoint and a gazing point set in the three-dimensional virtual space, from a virtual projection plane in the three-dimensional virtual space on which the captured image data is projected. A generation unit that generates one peripheral image data, the operation reception unit receives an operation to change the angle of the yaw direction as the direction of the three-dimensional shape model, and the generation unit is provided in the three-dimensional virtual space Based on the position of the vehicle, the second peripheral image data is generated by rotating the viewpoint and the gazing point in the yaw direction, and the display processing unit responds to the operation for changing the angle in the yaw direction received by the operation receiving unit. The second peripheral image data generated by the generation unit is switched to. Therefore, as an example, since it is possible to easily display the peripheral image data in which the angle is switched in the yaw direction with respect to the vehicle, an intuitive operation can be realized.

  In addition, as an example, the display control apparatus according to the embodiment is configured so that, based on a viewpoint and a gazing point set in the three-dimensional virtual space, from a virtual projection plane in the three-dimensional virtual space on which the captured image data is projected. A generation unit that generates one peripheral image data, the operation reception unit receives an operation of changing an angle in the pitch direction as the direction of the three-dimensional shape model, and the generation unit is a virtual projection plane in the three-dimensional virtual space From the first peripheral image data generated last time, second peripheral image data whose viewpoint is moved upward is generated, and the display processing unit changes the angle in the pitch direction received by the operation reception unit. In accordance with the operation to be performed, the second peripheral image data generated by the generation unit is switched. Therefore, as an example, since it is possible to easily display the peripheral image data in which the angle is switched in the pitch direction with reference to the vehicle, an intuitive operation can be realized.

  In the display control apparatus according to the embodiment, for example, the operation receiving unit further receives an enlargement or reduction operation of the image displayed in the second display area, and the display processing unit receives the image received by the operation receiving unit. The peripheral image data displayed in the second display area is switched according to the enlargement or reduction operation. Therefore, as an example, display of peripheral image data corresponding to an enlargement or reduction operation can be easily realized, so that operability can be improved.

  In addition, as an example, the display control device according to the embodiment has a display processing unit that is a three-dimensional vehicle that is displayed in the first display area when the operation receiving unit receives an image enlargement or reduction operation. Predetermined display information is expressed at a position on the shape model which is a reference for enlargement or reduction. Therefore, as an example, since the enlarged position can be recognized by referring to the first display area, the operability can be improved.

FIG. 1 is a perspective view illustrating an example of a state in which a part of a passenger compartment of a vehicle on which the display control device according to the embodiment is mounted is seen through. FIG. 2 is a plan view (bird's eye view) illustrating an example of a vehicle on which the display control device according to the embodiment is mounted. FIG. 3 is a block diagram illustrating an example of a configuration of a display control system including the display control device according to the embodiment. FIG. 4 is a block diagram illustrating a functional configuration of the ECU as the display control apparatus according to the embodiment. FIG. 5 is an exemplary schematic diagram illustrating projection of a captured image onto a virtual projection plane in the display control system according to the embodiment. FIG. 6 is a schematic and exemplary side view showing a vehicle shape model and a virtual projection plane in the display control system according to the embodiment. FIG. 7 is a diagram illustrating an example of a display screen displayed by the display processing unit according to the embodiment. FIG. 8 is a diagram illustrating a display icon of the embodiment. FIG. 9 is a diagram illustrating display icons according to the embodiment. FIG. 10 is a diagram illustrating a screen change when a right swipe operation is accepted for the display icon displayed in the first display area of the embodiment. FIG. 11 is a diagram illustrating an example of a screen change when a downward swipe operation is received for the display icon displayed in the first display area of the embodiment. FIG. 12 is a diagram exemplifying a change in the display mode of the first display area displayed by the display processing unit of the embodiment. FIG. 13 is a diagram illustrating a change in the display mode of the first display area displayed by the display processing unit of the embodiment. FIG. 14 is a diagram illustrating an example of a screen that is displayed when the acquisition unit of the embodiment acquires inclination information (roll angle) “30 °” in the left-right direction of the vehicle. FIG. 15 is a diagram illustrating an example of a screen displayed when the operation reception unit according to the embodiment receives a downward swipe operation. FIG. 16 is a diagram illustrating an example of a screen displayed before the operation receiving unit of the embodiment receives an enlargement operation of the peripheral image displayed in the second display area. FIG. 17 is a diagram illustrating an example of a screen displayed when the operation receiving unit of the embodiment receives an enlargement operation of the peripheral image displayed in the second display area. FIG. 18 is a diagram illustrating an example of a screen displayed when the operation reception unit of the embodiment receives an enlargement operation of the peripheral image displayed in the second display area. FIG. 19 is a flowchart illustrating a procedure of display processing in the display control system of the embodiment.

  Hereinafter, exemplary embodiments of the present invention are disclosed. The configuration of the embodiment shown below and the operations, results, and effects brought about by the configuration are examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and at least one of various effects based on the basic configuration and derivative effects can be obtained. .

  In the present embodiment, the vehicle 1 equipped with a display control device (display control system) may be, for example, an automobile using an internal combustion engine (not shown) as a drive source, that is, an internal combustion engine automobile, or an electric motor (not shown). It may be a vehicle as a drive source, that is, an electric vehicle, a fuel cell vehicle, or the like. Moreover, the hybrid vehicle which uses both of them as a drive source may be sufficient, and the vehicle provided with the other drive source may be sufficient. Further, the vehicle 1 can be mounted with various transmissions, and various devices necessary for driving the internal combustion engine and the electric motor, such as systems and components, can be mounted. As a drive system, it is possible to provide a four-wheel drive vehicle in which a drive force is transmitted to all four wheels 3 and all four wheels are used as drive wheels. Various methods, numbers, layouts, and the like of devices related to driving of the wheel 3 can be set. Further, the driving method is not limited to the four-wheel driving method, and may be a front wheel driving method or a rear wheel driving method, for example.

  As illustrated in FIG. 1, the vehicle body 2 constitutes a passenger compartment 2 a in which a passenger (not shown) gets. In the passenger compartment 2a, a steering section 4, an acceleration operation section 5, a braking operation section 6, a shift operation section 7 and the like are provided in a state facing the driver's seat 2b as a passenger. The steering unit 4 is, for example, a steering wheel protruding from the dashboard 25, the acceleration operation unit 5 is, for example, an accelerator pedal positioned under the driver's feet, and the braking operation unit 6 is, for example, a driver's It is a brake pedal located under the foot, and the speed change operation unit 7 is, for example, a shift lever protruding from the center console. The steering unit 4, the acceleration operation unit 5, the braking operation unit 6, the speed change operation unit 7 and the like are not limited to these.

  A display device 8 and an audio output device 9 are provided in the passenger compartment 2a. The display device 8 is, for example, an LCD (liquid crystal display) or an OELD (organic electroluminescent display). The audio output device 9 is, for example, a speaker. The display device 8 is covered with a transparent operation input unit 10 such as a touch panel. The occupant can visually recognize an image displayed on the display screen of the display device 8 via the operation input unit 10. In addition, the occupant can execute an operation input by touching, pushing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8. . The display device 8, the audio output device 9, the operation input unit 10, and the like are provided, for example, in the monitor device 11 that is located in the vehicle width direction of the dashboard 25, that is, in the center in the left-right direction. The monitor device 11 can have an operation input unit (not shown) such as a switch, a dial, a joystick, and a push button. Further, a sound output device (not shown) can be provided at another position in the passenger compartment 2a different from the monitor device 11, and sound is output from the sound output device 9 of the monitor device 11 and other sound output devices. be able to. Note that the monitor device 11 can be used also as, for example, a navigation system or an audio system.

  As illustrated in FIGS. 1 and 2, the vehicle 1 is, for example, a four-wheeled vehicle, and includes two left and right front wheels 3 </ b> F and two right and left rear wheels 3 </ b> R. All of these four wheels 3 can be configured to be steerable. As illustrated in FIG. 3, the vehicle 1 includes a steering system 13 that steers at least two wheels 3. The steering system 13 has a torque sensor 13b. The steering system 13 is electrically controlled by an ECU 14 (electronic control unit) or the like to operate an actuator. The steering system 13 is, for example, an electric power steering system, an SBW (steer by wire) system, or the like. The steering system 13 supplements the steering force by adding torque, that is, assist torque to the steering unit 4 by the actuator, or steers the wheel 3 by the actuator. In this case, the actuator may steer one wheel 3 or may steer a plurality of wheels 3. Moreover, the torque sensor 13b detects the torque which a driver | operator gives to the steering part 4, for example.

  Further, as illustrated in FIG. 2, for example, four imaging units 15 a to 15 d are provided in the vehicle body 2 as the plurality of imaging units 15. The imaging unit 15 is a digital camera including an imaging element such as a charge coupled device (CCD) or a CMOS image sensor (CIS). The imaging unit 15 can output moving image data (captured image data) at a predetermined frame rate. The imaging unit 15 has a wide-angle lens or a fish-eye lens, respectively, and can capture a range of, for example, 140 ° to 220 ° in the horizontal direction. Further, the optical axis of the imaging unit 15 may be set obliquely downward. Accordingly, the imaging unit 15 sequentially captures the external environment around the vehicle 1 including the road surface on which the vehicle 1 is movable and surrounding objects (obstacles, rocks, dents, puddles, dredging, etc.), and as captured image data. Output.

  The imaging unit 15a is located at, for example, the rear end 2e of the vehicle body 2, and is provided on a wall portion below the rear window of the rear hatch door 2h. The imaging unit 15b is located, for example, at the right end 2f of the vehicle body 2 and provided on the right door mirror 2g. The imaging unit 15c is located, for example, on the front side of the vehicle body 2, that is, the front end 2c in the vehicle front-rear direction, and is provided on a front bumper, a front grill, or the like. The imaging unit 15d is located, for example, at the left end 2d of the vehicle body 2 and is provided on the left door mirror 2g.

  As illustrated in FIG. 3, in the display control system 100 (display control device), in addition to the ECU 14, the monitor device 11, the steering system 13, etc., the brake system 18, the steering angle sensor 19, the accelerator sensor 20, the shift A sensor 21, a wheel speed sensor 22, an acceleration sensor 24, and the like are electrically connected via an in-vehicle network 23 as an electric communication line. The in-vehicle network 23 is configured as a CAN (controller area network), for example. The ECU 14 can control the steering system 13, the brake system 18, and the like by sending a control signal through the in-vehicle network 23. The ECU 14 also detects the detection results of the torque sensor 13b, the brake sensor 18b, the rudder angle sensor 19, the accelerator sensor 20, the shift sensor 21, the wheel speed sensor 22, and the acceleration sensor 24 via the in-vehicle network 23, and the operation input unit 10. An operation signal such as can be received.

  The ECU 14 includes, for example, a CPU 14a (central processing unit), a ROM 14b (read only memory), a RAM 14c (random access memory), a display control unit 14d, an audio control unit 14e, an SSD 14f (solid state drive, flash memory), and the like. ing. The CPU 14a reads a program stored (installed) in a non-volatile storage device such as the ROM 14b, and executes arithmetic processing according to the program. For example, the CPU 14a executes image processing related to an image displayed on the display device 8. For example, the CPU 14a performs arithmetic processing or image processing on the captured image data captured by the imaging unit 15 to detect whether or not there is a specific region to be noted on the predicted course of the vehicle 1, For example, the user (driver or passenger) is notified of the presence of a specific region by changing the display mode of a course indicator (predicted course line) indicating the estimated travel direction of the vehicle 1.

  The RAM 14c temporarily stores various types of data used in computations by the CPU 14a. The display control unit 14d mainly performs image processing using the captured image data obtained by the imaging unit 15 and image processing of image data displayed on the display device 8 (as an example) in the arithmetic processing performed by the ECU 14. Executes image composition). In addition, the voice control unit 14 e mainly executes processing of voice data output from the voice output device 9 among the calculation processes in the ECU 14. The SSD 14f is a rewritable nonvolatile storage unit, and can store data even when the ECU 14 is powered off. The CPU 14a, the ROM 14b, the RAM 14c, and the like can be integrated in the same package. Further, the ECU 14 may have a configuration in which another logical operation processor such as a DSP (digital signal processor) or a logic circuit is used instead of the CPU 14a. Further, an HDD (hard disk drive) may be provided instead of the SSD 14f, and the SSD 14f and the HDD may be provided separately from the peripheral monitoring ECU 14.

  The brake system 18 includes, for example, an anti-lock brake system (ABS) that suppresses brake locking, an anti-slip device (ESC: electronic stability control) that suppresses side slip of the vehicle 1 during cornering, and enhances braking force ( Electric brake system that executes brake assist, BBW (brake by wire), etc. The brake system 18 applies a braking force to the wheels 3 and thus to the vehicle 1 via the actuator. The brake sensor 18b is a sensor that detects the position of the movable part of the braking operation unit 6, for example. The brake sensor 18b can detect the position of a brake pedal as a movable part. The brake sensor 18b includes a displacement sensor.

  The steering angle sensor 19 is, for example, a sensor that detects the steering amount of the steering unit 4 such as a steering wheel. The rudder angle sensor 19 is configured using, for example, a hall element. The ECU 14 obtains the steering amount of the steering unit 4 by the driver, the steering amount of each wheel 3 during automatic steering, and the like from the steering angle sensor 19 and executes various controls. The rudder angle sensor 19 detects the rotation angle of the rotating part included in the steering unit 4. The rudder angle sensor 19 is an example of an angle sensor.

  The accelerator sensor 20 is, for example, a sensor that detects the position of the movable part of the acceleration operation unit 5. The accelerator sensor 20 can detect the position of an accelerator pedal as a movable part. The accelerator sensor 20 includes a displacement sensor.

  The shift sensor 21 is, for example, a sensor that detects the position of the movable part of the speed change operation unit 7. The shift sensor 21 can detect the position of a lever, arm, button, or the like as a movable part. The shift sensor 21 may include a displacement sensor or may be configured as a switch.

  The wheel speed sensor 22 is a sensor that detects the amount of rotation of the wheel 3 and the number of rotations per unit time. The wheel speed sensor 22 outputs a wheel speed pulse number indicating the detected rotation speed as a sensor value. The wheel speed sensor 22 may be configured using, for example, a hall element. The ECU 14 calculates the amount of movement of the vehicle 1 based on the sensor value acquired from the wheel speed sensor 22 and executes various controls. Note that the wheel speed sensor 22 may be provided in the brake system 18. In that case, the ECU 14 acquires the detection result of the wheel speed sensor 22 via the brake system 18.

  The acceleration sensor 24 is provided in the vehicle 1, for example. Based on the signal from the acceleration sensor 24, the ECU 14 calculates the front-rear direction inclination (pitch angle) and the left-right direction inclination (roll angle) of the vehicle 1. The pitch angle is an angle indicating the inclination of the vehicle 1 around the left and right axis, and the pitch angle is 0 ° when the vehicle 1 is present on a horizontal surface (ground, road surface). Further, the roll angle is an angle indicating the inclination around the longitudinal axis of the vehicle 1, and when the vehicle 1 is present on a horizontal surface (ground, road surface), the roll angle is 0 °. That is, it is possible to detect whether the vehicle 1 is present on a horizontal road surface, whether it is present on an inclined surface (an uphill road surface or a downhill road surface), and the like. In addition, when the vehicle 1 mounts ESC, the acceleration sensor 24 conventionally mounted in ESC is used. In the present embodiment, the acceleration sensor 24 is not limited, and any sensor that can detect the acceleration in the front-rear and left-right directions of the vehicle 1 may be used.

  The configuration, arrangement, electrical connection form, and the like of the various sensors and actuators described above are examples, and can be set (changed) in various ways.

  The CPU 14a included in the ECU 14 displays the environment around the vehicle 1 based on the captured image data as described above. In order to realize this function, the ECU 14 includes various modules as shown in FIG. The SSD 14f of the ECU 14 includes a three-dimensional shape model storage unit 401, for example. Further, the ECU 14 includes, for example, an operation reception unit 402, an acquisition unit 403, an image composition unit 404, a three-dimensional processing unit 405, a peripheral image generation unit 406, and a display processing unit 407. These modules can be realized by reading a program installed and stored in a storage device such as the ROM 14b and executing it.

  In the three-dimensional shape model storage unit 401 of the SSD 14f, for example, a three-dimensional vehicle shape model representing the three-dimensional shape of the vehicle 1 is stored.

  The operation reception unit 402 acquires a signal generated by an operation input from the operation input unit 10. The operation input unit 10 is a touch panel or the like provided on the display screen of the display device 8, and the operation reception unit 402 can recognize an operation performed on the operation input unit 10. Recognizable operations include operations such as tapping, flicking, and swiping.

  The acquisition unit 403 acquires information necessary for displaying the surroundings of the vehicle 1. For example, the acquisition unit 403 acquires a plurality of captured image data from the plurality of imaging units 15 that capture the periphery of the vehicle 1.

  Further, the acquisition unit 403 acquires the tilt information (pitch angle) in the front-rear direction and the tilt information (roll angle) in the left-right direction of the vehicle 1 by calculation based on the signal from the acceleration sensor 24.

  The image composition unit 404 connects the data of the plurality of captured images acquired by the acquisition unit 403, that is, the data of the plurality of captured images captured by the plurality of image capturing units 15 by combining their boundary portions, Data of one captured image is generated.

  The three-dimensional processing unit 405 projects the captured image data synthesized by the image synthesizing unit 404 onto a virtual projection plane in the three-dimensional virtual space with the position where the vehicle 1 is present as a reference.

  The three-dimensional processing unit 405 generates virtual projection image data obtained by projecting captured image data on a virtual projection plane surrounding the periphery of the vehicle 1, which is determined based on the position where the vehicle 1 exists. FIG. 5 is an exemplary schematic diagram illustrating projection of the captured image Ic onto the virtual projection plane Sp in the display control system 100. In the example of FIG. 5, the virtual projection plane Sp has a bottom surface Spg along the ground Gr and a side surface Sps rising from the bottom surface Spg, that is, the ground Gr. The ground Gr is a horizontal plane orthogonal to the vertical direction Z of the vehicle 1 and is also a tire contact surface. The bottom surface Spg is a substantially circular flat surface and is a horizontal surface with the vehicle 1 as a reference. The side surface Sps is a curved surface in contact with the bottom surface Spg. As shown in FIG. 5, the shape of the virtual cross section of the side surface Sps passing through the center Gc of the vehicle 1 and perpendicular to the vehicle 1 is, for example, an ellipse or a parabola. The side surface Sps is configured as, for example, a rotating surface around the center line CL that passes through the center Gc of the vehicle 1 and extends in the vertical direction of the vehicle 1, and surrounds the periphery of the vehicle 1. The three-dimensional processing unit 405 calculates a virtual projection image Ip obtained by projecting the captured image Ic onto the virtual projection plane Sp. When the captured image Ic is projected onto the ground Gr, the image becomes longer as the distance from the imaging unit 15 increases, and may appear longer than the actual length in the output image. As can be seen from FIG. 5, the virtual projection image Ip projected onto the side surface Sps rising from the ground Gr (bottom surface Spg) is shorter than that projected onto the ground Gr. It is suppressed that the image is projected longer than the length.

  The three-dimensional processing unit 405 arranges the vehicle shape model stored in the three-dimensional shape model storage unit 401 in the region 501 of the three-dimensional virtual space. The three-dimensional processing unit 405 sets a predetermined transmittance for the vehicle shape model arranged in the region 501. Thereby, the virtual projection image Ip on the opposite side can be visually recognized through the vehicle shape model in the three-dimensional virtual space.

  The peripheral image generation unit 406 generates a viewpoint set in the three-dimensional virtual space from the virtual projection image Ip and the vehicle shape model projected on the virtual projection plane Sp in the three-dimensional virtual space on which the captured image data is projected. The peripheral image data is generated based on the gazing point.

  FIG. 6 is a schematic and exemplary side view showing the vehicle shape model 601 and the virtual projection plane Sp (Sp0) in the display control system 100. In the example illustrated in FIG. 6, the peripheral image generation unit 406 looks at the virtual gazing point En from the virtual viewpoint Ep on the virtual projection image Ip (not illustrated in FIG. 6) projected on the virtual projection plane Sp. It is converted into peripheral image data (not shown in FIG. 6). In the example shown in FIG. 6, the viewpoint Ep is set behind the vehicle shape model 601, and the gazing point En is set in front of the vehicle shape model 601 via the vehicle shape model 601.

  As described above, the peripheral image generation unit 406 according to the present embodiment uses the vehicle shape model from the virtual projection plane in the three-dimensional virtual space on the basis of the viewpoint Ep and the gazing point En set in the three-dimensional virtual space. Thus, peripheral image data including an imaging region in which a situation around the vehicle 1 in the traveling direction is represented is generated.

  The display processing unit 407 of the present embodiment displays a display icon including a three-dimensional shape model representing the vehicle 1 in the first display area, and the three-dimensional shape model displayed in the first display area of the display device 8. The peripheral image data generated by the peripheral image generation unit 406 is displayed in the second display area of the display device 8 in accordance with the direction of the vehicle 1. The display icons displayed in the first display area are arranged based on the viewpoint Ep and the gazing point En set in the three-dimensional virtual space.

  FIG. 7 is a diagram illustrating an example of a display screen displayed by the display processing unit 407. In the display screen example shown in FIG. 7, a display icon for operation is displayed in the first display area 701, and peripheral image data is displayed in the second display area 702. Since the vehicle shape model 703 displayed in the peripheral image data of the second display area 702 has a predetermined transmittance, the situation in the vicinity of the traveling direction of the vehicle 1 is determined via the vehicle shape model 703. Is displayed. The “−” button 704 and the “+” button 705 are icons for enlarging and reducing the peripheral image data displayed in the second display area 702. The “−” button 704 and the “+” button 705 may be translucent. Accordingly, it is possible to grasp the situation of the portion where the “−” button 704 and the “+” button 705 are superimposed.

  The display icon displayed in the first display area 701 displays a three-dimensional vehicle shape model of the vehicle 1 and scale information for representing the tilt angle of the vehicle 1. The orientation of the vehicle shape model shown as the display icon is linked to the vehicle shape model 703 displayed in the second display area 702. That is, in the present embodiment, when the operation receiving unit 402 receives an operation for changing the orientation of the vehicle shape model displayed on the display icon, the display processing unit 407 is displayed in the second display area 702. The direction of the vehicle shape model 703 to be displayed and the surrounding situation to be displayed are switched. Next, display icons displayed in the first display area 701 will be described.

  8 and 9 are diagrams illustrating display icons of the present embodiment. The example shown in FIG. 8 is a view when the three-dimensional shape model of the vehicle 1 displayed as a display icon is referred from the rear, and the example shown in FIG. 9 is the display of the vehicle 1 displayed as a display icon. It is assumed that the three-dimensional shape model is referenced from the left side. Note that the three-dimensional shape model of the vehicle 1 displayed as the display icon is read from the three-dimensional shape model storage unit 401.

  As shown in FIGS. 8 and 9, the display icon represents a disk 801 for representing the vertical direction of the three-dimensional shape model, in addition to the three-dimensional shape model of the vehicle 1, and the horizontal direction of the three-dimensional shape model. A disk 802 is shown.

  Furthermore, areas 811 to 822 for representing the tilt information of the vehicle 1 are arranged on the circumference around the display icon. In the regions 811 to 822 of the present embodiment, as shown in FIG. 7, “90”, “60”, “30”, “0”, “−30”, “−60” are numerical values indicating the inclination. , “−90”, “−60”, “−30”, “0”, “30”, and “60” are set. In the present embodiment, the current inclination of the vehicle 1 can be expressed by the numerical values shown in the regions 811 to 822 and the positions of the three-dimensional shape model and the disks 801 and 802.

  The display icon of the present embodiment is displayed so as to be rotatable with respect to the center point of the disks 801 and 802. The operation accepting unit 402 according to the present embodiment accepts an operation for changing the orientation of the three-dimensional shape model of the vehicle 1 displayed in the first display area 701. In the present embodiment, the swipe operation is accepted as an operation for changing the direction of the display icon (in other words, rotating the three-dimensional shape model).

  In the present embodiment, when the operation receiving unit 402 receives a swipe operation in the left-right direction via the display icon, the operation receiving unit 402 receives an operation for changing the angle (orientation) in the yaw direction for the three-dimensional shape model of the vehicle 1. . On the other hand, when the operation reception unit 402 receives a swipe operation in the vertical direction via the display icon, the operation reception unit 402 receives the operation of changing the angle (orientation) in the pitch direction for the three-dimensional model of the vehicle 1.

  Then, the display processing unit 407 switches the peripheral image data displayed in the second display area in accordance with the operation for changing the orientation received by the operation receiving unit 402. The peripheral image data to be switched will be described later.

  FIG. 10 is a diagram exemplifying a change in the screen when a right swipe operation is accepted for the display icon displayed in the first display area 701 of the present embodiment. As illustrated in FIG. 10, the image 1011 is an example in which the three-dimensional shape model of the vehicle 1 is displayed so that the rear side of the vehicle is the front. In the example illustrated in FIG. 10, the regions 811 to 822 and the regions 1001 to 1005 are regions on which scale information representing inclination is superimposed. In addition, the numerical value shown as scale information is the same as the numerical value mentioned above, and description is abbreviate | omitted.

  Of the scale information displayed in the areas 811 to 822 and the areas 1001 to 1005, the scale information arranged to overlap the three-dimensional shape model or the disks 801 and 802 (for example, the areas 1001 to 1001 in the image 1011). The scale information displayed in the area 1005) may be omitted in order to improve visibility.

  When the operation reception unit 402 receives a right swipe operation on the first display area 701 displayed on the display device 8 by the driver's or passenger's finger 1050, the display processing unit 407 The images displayed in one display area 701 are switched in the order of image 1011, image 1012, and image 1013. In response to the switching, the display processing unit 407 switches the peripheral image data displayed in the first display area 701. For example, when the image 1013 is displayed, the display processing unit 407 displays peripheral image data arranged in the second display area 702 so that the left side surface of the three-dimensional shape model of the vehicle 1 is in front. To do. In the image 1013, display of the scale information displayed in the areas 812 to 816 may be omitted.

  Then, the peripheral image generation unit 406 displays the orientation of the three-dimensional shape model of the vehicle 1 in the first display area 701 displayed after the right or left swipe operation, and 3 of the vehicle 1 in the peripheral image data. A peripheral image obtained by rotating the viewpoint and the gazing point in the yaw direction in the three-dimensional virtual space with reference to the position of the vehicle 1 provided in the three-dimensional virtual space so that the directions of the three-dimensional shape model are the same. Generate data.

  Then, the display processing unit 407 switches to the peripheral image data generated by the peripheral image generation unit 406 in accordance with the operation for changing the angle in the yaw direction received by the operation reception unit 402.

  FIG. 11 is a diagram illustrating a screen change when a downward swipe operation is accepted for the display icon displayed in the first display area 701 of the present embodiment. As shown in FIG. 11, the image 1111 is an example in which the three-dimensional shape model of the vehicle 1 is displayed so that the rear side of the vehicle 1 is the front.

  When the operation reception unit 402 receives a downward swipe operation on the first display area 701 displayed on the display device 8 with the driver's or passenger's finger 1150, the display processing unit 407 displays an image. The display is switched in the order of 1111, image 1112, and image 1113. In response to the switching, the display processing unit 407 switches the peripheral image data displayed in the first display area 701. For example, when the image 1113 is displayed, the display processing unit 407 displays the peripheral image data arranged so that the top surface of the three-dimensional shape model of the vehicle 1 is in front also in the second display area 702. .

  Then, the peripheral image generation unit 406 displays the orientation of the three-dimensional shape model of the vehicle 1 in the first display area 701 displayed after the upward or downward swipe operation, and 3 of the vehicle 1 in the peripheral image data. Compared to the previously generated peripheral image data so that the direction of the three-dimensional shape model is the same, at least the viewpoint is determined in the three-dimensional virtual space with reference to the position of the vehicle 1 provided in the three-dimensional virtual space. Peripheral image data moved upward or downward is generated. Note that the gazing point may be fixed, or may be moved in the same manner as the viewpoint.

  Then, the display processing unit 407 switches to the peripheral image data generated by the peripheral image generation unit 406 according to the operation of changing the pitch direction angle received by the operation reception unit 402.

  Further, in the present embodiment, the information is displayed in the first display area 701 according to the tilt information (pitch angle) in the front-rear direction and the tilt information (roll angle) in the left-right direction acquired by the acquisition unit 403. The display mode of the three-dimensional shape model of the vehicle 1 is changed.

  FIG. 12 is a diagram illustrating a change in the display mode of the first display area 701 displayed by the display processing unit 407 of the present embodiment. In the example shown in FIG. 12, the image 1211 is a case where the front-rear direction inclination (pitch angle) of the vehicle 1 acquired by the acquisition unit 403 is “0 °”. The image 1212 is obtained when the inclination (pitch angle) in the front-rear direction of the vehicle 1 acquired by the acquisition unit 403 is “30 °”. In the image 1211, the three-dimensional shape model 1201 is displayed in a horizontal state (pitch angle “0 °”). Since the disk 802 is also in a horizontal state, the end of the disk 802 is disposed so as to overlap the region 1003.

  In the image 1212, the three-dimensional shape model 1202 is displayed in a state of being inclined in the front-rear direction (pitch angle “30 °”). Since the disk 802 is also tilted together with the three-dimensional shape model 1202, the end of the disk 802 is disposed so as to overlap the region 1004.

  FIG. 13 is a diagram illustrating a change in the display mode of the first display area 701 displayed by the display processing unit 407 of the present embodiment. In the example illustrated in FIG. 13, the image 1311 is a case where the horizontal inclination (roll angle) of the vehicle 1 acquired by the acquisition unit 403 is “0 °”. The image 1312 is a case where the horizontal inclination (roll angle) of the vehicle 1 acquired by the acquisition unit 403 is “30 °”. In the image 1311, the three-dimensional shape model 1301 is displayed in a horizontal state (roll angle “0 °”). Since the disk 802 is also in a horizontal state, the end of the disk 802 is disposed so as to overlap the regions 820 and 814.

  In the image 1312, the three-dimensional shape model 1301 is displayed in a state inclined in the left-right direction (roll angle “30 °”). Since the disks 801 and 802 are also tilted together with the three-dimensional shape model 1302, the ends of the disks 802 are arranged so as to overlap the regions 819 and 813.

  Then, the display processing unit 407 displays the three-dimensional shape model of the vehicle 1 and the scale information representing the degree of inclination in the first display area 701 with an inclination corresponding to the inclination information. When the operation accepting unit 402 accepts an operation for changing the orientation, the display processing unit 407 displays the three-dimensional shape model displayed in the first display area 701 in accordance with the operation for changing the accepted orientation. And the position where the scale information is displayed is changed corresponding to the direction. The position changed by the scale information is the same as in FIG. 10 and FIG.

  FIG. 14 is a diagram illustrating an example of a screen that is displayed when the acquisition unit 403 acquires tilt information (roll angle) “30 °” in the left-right direction of the vehicle 1. In the example illustrated in FIG. 14, the first display area 1401 displays a three-dimensional shape model corresponding to the left-right direction tilt information (roll angle) of the vehicle 1 acquired by the acquisition unit 403. On the other hand, the second display area 1402 displays peripheral image data in which the same viewpoint and gazing point as the first display area 1401 are set. However, rotation correction according to the tilt information (roll angle) “30 °” is not performed on the captured image data. Therefore, the peripheral image data captured by the vehicle 1 in which the three-dimensional shape model is in a horizontal state and is tilted by the tilt information (roll angle) “30 °” is displayed.

  FIG. 15 is a diagram illustrating an example of a screen displayed when the operation reception unit 402 receives a downward swipe operation. In the example shown in FIG. 15, the first display area 1501 displays a three-dimensional shape model in which the vehicle 1 is rotated in the pitch direction according to the swipe operation accepted by the operation accepting unit 402. On the other hand, the second display area 1502 displays peripheral image data in which a viewpoint and a gazing point are set so that the first display area 1501 and the three-dimensional shape model have the same display mode.

  Furthermore, the operation reception unit 402 according to the present embodiment receives an enlargement or reduction operation of the peripheral image data displayed in the second display area. In the present embodiment, when the pressing of the “−” button or the “+” button displayed in the second display area is received, the display processing unit 407 performs an enlargement operation (“+” received by the operation receiving unit 402. The peripheral image data displayed in the second display area is switched in response to a “button press” or a reduction operation (“−” button).

  In this embodiment, when an enlargement operation is received, the peripheral image generation unit 406 generates peripheral image data in which the viewpoint is closer to the gazing point than the previously generated peripheral image data. Accordingly, it is possible to generate the peripheral image data in which the three-dimensional shape model and the surrounding situation are enlarged as compared with the peripheral image data generated last time.

  On the other hand, when a reduction operation is received, the peripheral image generation unit 406 generates peripheral image data with the viewpoint farther from the gazing point than the previously generated peripheral image data. Accordingly, it is possible to generate the peripheral image data in which the three-dimensional shape model and the surrounding situation are reduced as compared with the peripheral image data generated last time.

  16 to 18 are diagrams illustrating screen examples displayed when the operation receiving unit 402 according to the present embodiment receives an operation for enlarging or reducing the surrounding image data displayed in the second display area. FIG. 16 shows an example of a screen before accepting an enlargement operation. In the example illustrated in FIG. 16, when the operation reception unit 402 receives a press of the “+” button 1611 in the second display area 1602 from the driver or the passenger, FIG. 17 is displayed. Note that when the “-” button 1612 is pressed, the reduced peripheral image data is displayed in the second display area 1602. Note that description of the first display area 1601 is omitted.

  FIG. 17 shows an example of a screen after receiving an enlargement operation once. In the example illustrated in FIG. 17, the display processing unit 407 displays the enlarged peripheral image data in the second display area 1702. In addition, the display processing unit 407 displays a mark 1713 at a position that is a reference for enlargement on the three-dimensional shape model of the vehicle 1 in the first display area 1701. As a result, the driver or the passenger f can recognize which position is displayed in the enlarged peripheral image data. In the present embodiment, an example in which the mark 1713 is displayed at a position that has become a reference for enlargement will be described. Similarly, when a reduction operation is received, the display processing unit 407 is positioned at a position that has become a reference for reduction. Display the mark.

  In the example illustrated in FIG. 17, when the operation reception unit 402 receives a press of the “+” button 1711 in the second display area 1702 from the driver or the passenger, FIG. 18 is displayed.

  FIG. 18 shows an example of the screen after receiving the enlargement operation twice. In the example illustrated in FIG. 18, the display processing unit 407 displays the further enlarged peripheral image data in the second display area 1802. In addition, the display processing unit 407 displays a mark 1813 on the three-dimensional shape model of the vehicle 1 in the first display area 1801 at a position that is a reference for enlargement. In the example shown in FIG. 18, by repeating the enlargement operation, it is possible to grasp the shape of the obstacle 1821 existing on the opposite side of the vehicle 1 through the three-dimensional shape model. .

  In the example shown in FIG. 18, by receiving a press of a “+” button 1811 or a “−” button 1812, it is possible to display further enlarged or reduced peripheral image data.

  Next, display processing in the display control system (ECU 14) of the present embodiment will be described. FIG. 19 is a flowchart showing a procedure of the above-described processing in the display control system of the present embodiment.

  First, the acquisition unit 403 of the ECU 14 acquires the tilt information (pitch angle) in the front-rear direction and the tilt information (roll angle) in the left-right direction of the vehicle 1 based on the signal from the acceleration sensor 24 (S1901).

  Next, the acquisition unit 403 acquires captured image data obtained by imaging the periphery of the vehicle 1 from the four imaging units 15a to 15d (S1902).

  Then, the image composition unit 404 synthesizes the plurality of captured image data acquired in S1902 to generate one captured image data (S1903).

  The three-dimensional processing unit 405 projects the combined captured image data on the virtual projection plane in the three-dimensional virtual space (S1904). At that time, a three-dimensional shape model of the vehicle 1 in which a predetermined transmittance is set is also arranged in the three-dimensional virtual space.

  Then, the peripheral image generation unit 406 generates peripheral image data representing the captured image data projected on the virtual projection plane via the three-dimensional shape model of the vehicle 1 based on the predetermined viewpoint and the gazing point. Generate (S1905).

  The display processing unit 407 displays the three-dimensional shape model of the vehicle 1 tilted according to the tilt information acquired by the acquiring unit 403 in the first display area, and displays the peripheral image data in the second display area (S1906). ).

  Then, it is determined whether the operation reception unit 402 has received an operation (for example, a swipe operation) for changing the orientation of the three-dimensional shape model displayed in the first display area (S1907).

  When it is determined that the operation reception unit 402 has received an operation for changing the direction (for example, a swipe operation) (S1907: Yes), the peripheral image generation unit 406 is based on the viewpoint and the gaze point corresponding to the direction in which the operation is received. Then, peripheral image data representing the captured image data projected on the virtual projection plane is generated through the three-dimensional shape model of the vehicle 1 (S1908).

  The display processing unit 407 displays the three-dimensional shape model of the vehicle 1 in the changed orientation in the first display area, and displays the peripheral image data generated in S1908 as corresponding to the orientation in the second display area. (S1909). Thereafter, the processing is again performed from S1907.

  On the other hand, if the operation accepting unit 402 does not accept an operation for changing the direction (for example, a swipe operation) (S1907: No), the display ends in S1906 and the process ends.

  In the above-described embodiment, the display control system has the above-described configuration, and the three-dimensional shape model of the vehicle 1 and the surrounding situation displayed in the second display area in conjunction with the operation on the first display area. Since switching is performed, an intuitive display switching operation can be provided.

  In the embodiment described above, the scale information indicating the inclination of the vehicle 1 is displayed together with the display icon of the first display area, so that the inclination of the vehicle 1 can be recognized.

  In the embodiment described above, visibility can be improved by switching the display of the peripheral image data when an enlargement operation or a reduction operation is accepted.

  In the above-described embodiment, since an icon for switching the viewpoint is not displayed in the second display area, it is possible to suppress a situation in which the situation of the portion where the icon is superimposed cannot be recognized.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 14 ... ECU, 100 ... Display control system, 401 ... Three-dimensional shape model memory | storage part, 402 ... Operation reception part, 403 ... Acquisition part, 404 ... Image composition part, 405 ... Three-dimensional processing part, 406 ... Periphery Image generation unit, 407... Display processing unit.

Claims (6)

An acquisition unit that acquires a plurality of captured image data from a plurality of imaging units that capture the periphery of the vehicle;
Captured image data acquired by the acquisition unit according to the orientation of the vehicle of the three-dimensional shape model displayed in the first display region, displaying a three-dimensional shape model representing the vehicle in the first display region A display processing unit for displaying peripheral image data representing the periphery of the vehicle based on the second display area;
An operation receiving unit that receives an operation of changing the orientation of the three-dimensional shape model displayed in the first display area by the display processing unit;
The display processing unit switches the peripheral image data displayed in the second display area according to an operation of changing the orientation received by the operation receiving unit.
Display control device. The acquisition unit further acquires inclination information indicating the inclination of the vehicle,
The display processing unit further displays, in the first display area, a three-dimensional shape model of the vehicle with a tilt corresponding to the tilt information, and scale information representing the degree of the tilt, in the first display area. In response to an operation to change the orientation received by the reception unit, the position where the scale information rotated according to the orientation is displayed is changed.
The display control apparatus according to claim 1. A generating unit configured to generate first peripheral image data based on a viewpoint and a gazing point set in the three-dimensional virtual space from a virtual projection plane in the three-dimensional virtual space on which the captured image data is projected; ,
The operation accepting unit accepts an operation to change the angle of the yaw direction as the direction of the three-dimensional shape model,
The generating unit generates second peripheral image data in which the viewpoint and the gazing point are rotated in the yaw direction with reference to a position of a vehicle provided in the three-dimensional virtual space;
The display processing unit switches to the second peripheral image data generated by the generation unit according to an operation of changing the angle of the yaw direction received by the operation reception unit.
The display control apparatus according to claim 1 or 2. A generating unit configured to generate first peripheral image data based on a viewpoint and a gazing point set in the three-dimensional virtual space from a virtual projection plane in the three-dimensional virtual space on which the captured image data is projected; ,
The operation receiving unit receives an operation of changing an angle in the pitch direction as the direction of the three-dimensional shape model,
The generation unit generates second peripheral image data in which the viewpoint is moved upward compared to the first peripheral image data generated last time from the virtual projection plane in the three-dimensional virtual space,
The display processing unit switches to the second peripheral image data generated by the generation unit in accordance with an operation of changing the angle in the pitch direction received by the operation reception unit.
The display control apparatus according to claim 1 or 2. The operation accepting unit further accepts an enlargement or reduction operation of the image displayed in the second display area,
The display processing unit switches the peripheral image data displayed in the second display area in accordance with an enlargement or reduction operation of the image received by the operation reception unit.
The display control apparatus according to claim 1 or 2. The display processing unit enlarges or reduces on the three-dimensional shape model representing the vehicle displayed in the first display area when the operation accepting unit accepts an enlargement or reduction operation of the image. Predetermined display information is displayed at the reference position of
The display control apparatus according to claim 5.

Images