JP2017094922A - Periphery monitoring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a periphery monitoring device which can provide a peripheral image in a state similar to usual visible sensation of a user.SOLUTION: A periphery monitoring device comprises: an image acquisition part which acquires picked-up image data outputted from an imaging part which is provided on a vehicle and picks up an image of a periphery of the vehicle; a display unit which is provided in a cabin of the vehicle and displays an image; and a control part which acquires first sight line image data which faces on one first lateral region with at least the vehicle as a reference based on the picked-up image data, and second sight line image data which faces on the other second lateral region with the vehicle as a reference, causes the display unit to display an image based on the first sight line image data, and thereafter, to display an image based on the second sight line image data instead of the image based on said first sight line image data.SELECTED DRAWING: Figure 5

Description

  Embodiments described herein relate generally to a periphery monitoring device.

  2. Description of the Related Art Conventionally, a technique for displaying a peripheral image of a vehicle imaged by an in-vehicle camera on a display device arranged in the vehicle interior is known. For example, a technique for displaying the status of another traveling path on a display device when entering the other traveling path intersecting at an intersection from the traveling path of the own vehicle is known. When displaying such a peripheral image, the display area of the display device is divided into a plurality of areas so that more information can be recognized, and images taken in various directions are displayed at the same time, or a wide-angle lens or fish-eye lens is used. A wide-angle image taken with the camera used was displayed or displayed with distortion correction.

JP 2013-136258 A

  However, a display obtained by dividing a display area as described above or a display using a wide-angle image is different from a real-world landscape that a user (driver) usually sees (views). You need to get used to it. In addition, the display area is divided to reduce the display area in each direction, and images in various directions are arranged in a matrix, making it look different from the actual situation. There is a problem that there is a possibility that the possibility of oversight is high.

  Accordingly, one of the objects of the present invention is to provide a periphery monitoring device that can provide a peripheral image on a display device in a state close to a user's usual visual sense.

  The periphery monitoring device according to the embodiment of the present invention is provided in, for example, an image acquisition unit that is provided in a vehicle and acquires captured image data output from an imaging unit that captures the periphery of the vehicle, and the vehicle interior of the vehicle. A display device that displays the image, first visual line image data that faces one first side area based on at least the vehicle based on the captured image data, and the other second side area that faces the first vehicle. After obtaining the second line-of-sight image data and displaying the image based on the first line-of-sight image data on the display device, the image based on the second line-of-sight image data is displayed instead of the image based on the first line-of-sight image data A control unit. According to this configuration, for example, it is possible to provide an image display that performs “right / left confirmation” that is generally performed, and thus it is easily accepted by the user. In addition, since the image of the first side area (for example, the right side area of the vehicle) and the image of the second side area (for example, the left side area of the vehicle) are displayed at different timings, the images are individually displayed in a limited display area. Can be displayed as large as possible. As a result, the possibility of missing noticeable information can be reduced.

  In the periphery monitoring device, for example, the captured image data is wide-angle image data including the traveling direction of the vehicle, and the control unit extracts the captured image data from a vehicle front end position in the traveling direction of the vehicle. Image conversion for obtaining the first line-of-sight image data by converting to a plane image facing the first side area and for obtaining the upper second line-of-sight image data by converting to a plane image facing the second side area Part may be included. According to this configuration, for example, the image displayed on the display device can display an image that provides the same information as when the user stands at the front end (rear end) of the vehicle and visually recognizes the left and right. In this case, the confirmation of the first side region and the second side region can be confirmed in a state similar to the actual state without having to advance the vehicle to a position where the driver's seat exists (a position where the user can actually look around the side). A display mode can be provided. In addition, a wide range of image data including the traveling direction of the vehicle is efficiently acquired by the image acquisition unit. In addition, since the image conversion unit converts, for example, a wide-angle image that may contain distortion into a line-of-sight image with less distortion, the first line-of-sight image data and the second line-of-sight image data that make it easier to recognize the display content can be provided.

  Further, the control unit of the periphery monitoring device includes, for example, a front surface including a traveling direction of the vehicle during a transition from an image display based on the first line-of-sight image data to an image display based on the second line-of-sight image data. An image based on the third line-of-sight image data facing the area may be displayed. According to this configuration, for example, an image based on the third line-of-sight image data indicating a front area that the user can actually visually recognize is interposed between the image based on the first line-of-sight image data and the image based on the second line-of-sight image data. Therefore, it is possible to easily identify the transition state of the displayed image, that is, whether the image to be displayed is an image based on the first visual line image data or an image based on the second visual line image data.

  In addition, the control unit of the periphery monitoring device may, for example, display an image based on the third line-of-sight image data from an image display period based on the first line-of-sight image data and an image display period based on the second line-of-sight image data. The display period may be shortened. According to this configuration, for example, it is difficult for the user himself to visually recognize (only visible via the image), and a long confirmation time for the content of the image based on the first visual line image data and the content of the image based on the second visual line image data is secured. can do. As a result, the possibility of missing noticeable information can be reduced. In addition, the image display period based on the third line-of-sight image data is shorter than the image display period based on the first line-of-sight image data and the image based on the second line-of-sight image data. The possibility of being distracted can be reduced, and the image can be more easily concentrated by the image based on the first line-of-sight image data or the image based on the second line-of-sight image data.

  In addition, the image acquisition unit of the periphery monitoring device further acquires object image data output from an object detection unit that is provided in a vehicle and detects a moving object around the vehicle, for example. The presence of the moving object may be notified by superimposing an image based on the object image data on an image based on the captured image data. According to this configuration, for example, the presence of a moving object such as a pedestrian or another vehicle existing in the first lateral region or the second lateral region can be easily recognized by the user.

  Further, the control unit of the periphery monitoring device, for example, displays an image based on the first line-of-sight image data, displays an image based on the second line-of-sight image data, and then again displays an image based on the first line-of-sight image data. May be displayed. According to this configuration, for example, the display is closer to the right and left confirmation operations that are generally performed, and the user's sense of security can be further improved, and the confirmation accuracy can be further improved.

FIG. 1 is a perspective view illustrating an example of a state in which a part of a passenger compartment of a vehicle on which the periphery monitoring device according to the embodiment is mounted is seen through. FIG. 2 is a plan view illustrating an example of a vehicle on which the periphery monitoring device according to the embodiment is mounted. FIG. 3 is an example of a dashboard of a vehicle on which the periphery monitoring device according to the embodiment is mounted, and is a diagram in a view from the rear of the vehicle. FIG. 4 is a block diagram illustrating an example of an image control system including the periphery monitoring device according to the embodiment. FIG. 5 is a block diagram illustrating an example of a configuration of a control unit for displaying an image realized in the ECU of the periphery monitoring device according to the embodiment. FIG. 6 is an explanatory diagram illustrating an example of transition of a display image displayed on the display device of the periphery monitoring device according to the embodiment. FIG. 7 is a flowchart illustrating an example of display processing of the periphery monitoring device according to the embodiment. FIG. 8 is an explanatory diagram illustrating an example of an image displayed on the display device of the periphery monitoring device according to the embodiment. FIG. 9 is an explanatory diagram illustrating another image example displayed on the display device of the periphery monitoring device according to the embodiment.

  Hereinafter, exemplary embodiments of the present invention are disclosed. The configuration of the embodiment shown below and the operations, results, and effects brought about by the configuration are examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and at least one of various effects based on the basic configuration and derivative effects can be obtained. .

  In the present embodiment, the vehicle 1 equipped with the periphery monitoring device (perimeter monitoring system) may be, for example, an automobile using an internal combustion engine (not shown) as a drive source, that is, an internal combustion engine automobile, or an electric motor (not shown). It may be a vehicle as a drive source, that is, an electric vehicle, a fuel cell vehicle, or the like. Moreover, the hybrid vehicle which uses both of them as a drive source may be sufficient, and the vehicle provided with the other drive source may be sufficient. Further, the vehicle 1 can be mounted with various transmissions, and various devices necessary for driving the internal combustion engine and the electric motor, such as systems and components, can be mounted. In addition, the method, number, layout, and the like of the device related to driving of the wheels 3 in the vehicle 1 can be variously set.

  As illustrated in FIG. 1, the vehicle body 2 constitutes a passenger compartment 2 a in which a passenger (not shown) gets. In the passenger compartment 2a, a steering section 4, an acceleration operation section 5, a braking operation section 6, a shift operation section 7 and the like are provided in a state facing the driver's seat 2b as a passenger. The steering unit 4 is, for example, a steering wheel protruding from the dashboard 24, the acceleration operation unit 5 is, for example, an accelerator pedal positioned under the driver's feet, and the braking operation unit 6 is, for example, a driver's foot It is a brake pedal located under the foot, and the speed change operation unit 7 is, for example, a shift lever protruding from the center console. The steering unit 4, the acceleration operation unit 5, the braking operation unit 6, the speed change operation unit 7 and the like are not limited to these.

  In addition, a display device 8 as a display output unit and a sound output device 9 as a sound output unit are provided in the passenger compartment 2a. The display device 8 is, for example, an LCD (liquid crystal display) or an OELD (organic electroluminescent display). The audio output device 9 is, for example, a speaker. The display device 8 is covered with a transparent operation input unit 10 such as a touch panel. The occupant can visually recognize an image displayed on the display screen of the display device 8 via the operation input unit 10. In addition, the occupant can execute an operation input by touching, pushing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8. . The display device 8, the audio output device 9, the operation input unit 10, and the like are provided, for example, in the monitor device 11 that is located in the vehicle width direction of the dashboard 24, that is, the central portion in the left-right direction. The monitor device 11 can have an operation input unit (not shown) such as a switch, a dial, a joystick, and a push button. Further, a sound output device (not shown) can be provided at another position in the passenger compartment 2a different from the monitor device 11, and sound is output from the sound output device 9 of the monitor device 11 and other sound output devices. be able to. Note that the monitor device 11 can be used also as, for example, a navigation system or an audio system.

  Further, a display device 12 different from the display device 8 is provided in the passenger compartment 2a. As illustrated in FIG. 3, for example, the display device 12 is provided in the instrument panel unit 25 of the dashboard 24, and between the speed display unit 25 a and the rotation speed display unit 25 b at the approximate center of the instrument panel unit 25. Is located. The size of the screen 12a of the display device 12 is smaller than the size of the screen 8a (FIG. 3) of the display device 8. The display device 12 can display an image mainly showing information related to parking assistance of the vehicle 1. The amount of information displayed on the display device 12 may be smaller than the amount of information displayed on the display device 8. The display device 12 is, for example, an LCD or an OELD. Information displayed on the display device 12 may be displayed on the display device 8.

  As illustrated in FIGS. 1 and 2, the vehicle 1 is, for example, a four-wheeled vehicle, and includes two left and right front wheels 3 </ b> F and two right and left rear wheels 3 </ b> R. All of these four wheels 3 can be configured to be steerable. As illustrated in FIG. 4, the vehicle 1 includes a steering system 13 that steers at least two wheels 3. The steering system 13 includes an actuator 13a and a torque sensor 13b. The steering system 13 is electrically controlled by an ECU 14 (electronic control unit) or the like to operate the actuator 13a. The steering system 13 is, for example, an electric power steering system, an SBW (steer by wire) system, or the like. The steering system 13 adds torque, that is, assist torque to the steering unit 4 by the actuator 13a to supplement the steering force, or steers the wheel 3 by the actuator 13a. In this case, the actuator 13a may steer one wheel 3 or may steer a plurality of wheels 3. Moreover, the torque sensor 13b detects the torque which a driver | operator gives to the steering part 4, for example.

  Further, as illustrated in FIG. 2, for example, four imaging units 15 a to 15 d are provided in the vehicle body 2 as the plurality of imaging units 15. The imaging unit 15 is a digital camera including an imaging element such as a charge coupled device (CCD) or a CMOS image sensor (CIS). The imaging unit 15 can output moving image data (captured image data) at a predetermined frame rate. The imaging unit 15 has a wide-angle lens or a fish-eye lens, respectively, and can capture a range of, for example, 140 ° to 220 ° in the horizontal direction. Further, the optical axis of the imaging unit 15 may be set obliquely downward. Therefore, the imaging unit 15 sequentially determines the external environment around the vehicle 1 including the road surface on which the vehicle 1 can move, the area in which the vehicle 1 can be parked, and surrounding objects (obstacles, people, bicycles, vehicles, etc.). A photograph is taken and output as captured image data.

  The imaging unit 15a is located, for example, at the rear end 2e of the vehicle body 2, and is provided on a wall portion below the rear trunk door 2h. The imaging unit 15b is located, for example, at the right end 2f of the vehicle body 2 and provided on the right door mirror 2g. The imaging unit 15c is located, for example, at the end 2c on the front side of the vehicle body 2, that is, the front side in the vehicle front-rear direction, and is provided on a front bumper or the like. The imaging unit 15d is located, for example, at the left end 2d of the vehicle body 2, that is, the left end 2d in the vehicle width direction, and is provided in the door mirror 2g as a left protruding portion. The ECU 14 performs arithmetic processing and image processing based on the image data obtained by the plurality of imaging units 15, generates an image with a wider viewing angle, or creates a virtual overhead view image when the vehicle 1 is viewed from above. Can be generated. Further, the ECU 14 performs arithmetic processing and image processing on the wide-angle image data obtained by the imaging unit 15 to generate an image obtained by cutting out a specific area, or generate image data indicating only the specific area. To do. Further, the ECU 14 can convert the captured image data into virtual image data captured from a virtual viewpoint different from the viewpoint captured by the imaging unit 15. For example, it can be converted into virtual image data indicating an overhead image as if the vehicle 1 is overhead. In addition, the ECU 14 can convert the line-of-sight image data with a line-of-sight direction different from the line-of-sight direction of the captured image data acquired by the imaging unit 15. For example, it is possible to convert to line-of-sight image data when the line of sight is directed to the front of the vehicle 1 or line-of-sight image data when the line of sight is directed to the side. The ECU 14 displays the acquired image data on the display device 8, for example, surrounding monitoring information that allows the vehicle 1 to perform a safety check on the right side or the left side of the vehicle 1 or perform a safety check on its surroundings by overlooking the vehicle 1. I will provide a.

  The ECU 14 identifies the lane markings and the like indicated on the road surface around the vehicle 1 from the captured image data provided from the imaging unit 15, detects (extracts) the parking lane indicated by the lane markings, and the like. Parking assistance can also be implemented.

  As illustrated in FIGS. 1 and 2, the vehicle body 2 includes, for example, four distance measuring sections 16 a to 16 d and eight distance measuring sections 17 a to 17 h as the plurality of distance measuring sections 16 and 17. ing. The distance measuring units 16 and 17 are, for example, sonar that emits ultrasonic waves and captures the reflected waves. The sonar may also be referred to as a sonar sensor, an ultrasonic detector, or an ultrasonic sonar. In the present embodiment, for example, when the vehicle 1 is parked, the distance measuring unit 16 is a first obstacle (adjacent vehicle) aligned with the vehicle 1 or a second obstacle existing on the far side of the space for parking. (For example, curbs, steps, walls, fences, etc.) can be detected and the distance to the obstacle can be measured. For example, when an obstacle (object) approaches the vehicle 1 beyond a predetermined distance, the distance measuring unit 17 detects the approaching obstacle (object) and measures the distance to the obstacle. be able to. In particular, the distance measuring units 17a and 17d disposed on both sides of the rear of the vehicle 1 include the rear corner portion of the vehicle 1 and the first obstacle (adjacent vehicle) when the vehicle 1 enters the parallel parking space while moving backward. And a sensor (clearance sonar) that measures the distance between the rear corner and the second obstacle (such as a wall) after entering. The ECU 14 can measure the presence or absence of an object such as an obstacle located around the vehicle 1 and the distance to the object based on the detection results of the distance measuring units 16 and 17. That is, the distance measuring units 16 and 17 are an example of an object detecting unit that detects an object (a stationary object or a moving object) existing around the vehicle 1. The stationary objects are parked vehicles, walls, curbs, street trees, and the like, and the moving objects are traveling vehicles, bicycles, pedestrians, animals, and the like. The distance measuring unit 17 can be used, for example, for detecting an object at a relatively short distance, and the distance measuring unit 16 can be used for detecting an object at a relatively long distance farther than the distance measuring unit 17, for example. The distance measuring unit 17 can be used, for example, for detecting an object in front of and behind the vehicle 1, and the distance measuring unit 16 can be used for detecting an object on the side of the vehicle 1. The distance measuring unit 17 can function as an proximity sensor that detects that an object (obstacle) has approached a predetermined distance.

  The ECU 14 can also generate an image showing the outer shape of an object existing around the vehicle 1 based on information acquired by the distance measuring unit 16 or the distance measuring unit 17. That is, when the ECU 14 generates (converts) various image data from the captured image data, the information acquired by the distance measurement unit 16 or the distance measurement unit 17 is used to interpolate the converted image data, and the vehicle 1 It is possible to generate data that more clearly (details) indicate the situation around the. For example, an image having a more stereoscopic effect can be formed. Further, shape recognition may be performed on the basis of information acquired by the distance measuring units 16 and 17 (captured image data using ultrasonic waves), and the types of stationary objects and moving objects may be identified and provided to the user. For example, the user can be notified of what is approaching and be more alert. A laser scanner may be used as the distance measuring units 16 and 17. In this case, data can be acquired in more detail, which can contribute to improving the accuracy of the outer shape of the object. For example, when performing viewpoint conversion or line-of-sight conversion, the shape of the original object may be distorted or partially lost. In such a case, by using the detailed data acquired by the laser scanner, it is possible to reduce the distortion or to more accurately reproduce the three-dimensional shape of the object by interpolating the lack.

  Further, as exemplified in FIG. 4, in the periphery monitoring system 100 (perimeter monitoring device), in addition to the ECU 14, the monitor device 11, the steering system 13, the distance measuring units 16 and 17, the brake system 18, the steering angle sensor 19, an accelerator sensor 20, a shift sensor 21, a wheel speed sensor 22, and the like are electrically connected via an in-vehicle network 23 as an electric communication line. The in-vehicle network 23 is configured as a CAN (controller area network), for example. The ECU 14 can control the steering system 13, the brake system 18, and the like by sending a control signal through the in-vehicle network 23. Further, the ECU 14 detects the torque sensor 13b, the brake sensor 18b, the rudder angle sensor 19, the distance measuring unit 16, the distance measuring unit 17, the accelerator sensor 20, the shift sensor 21, the wheel speed sensor 22, and the like via the in-vehicle network 23. Results, operation signals from the operation input unit 10 and the like can be received.

  The ECU 14 includes, for example, a CPU 14a (central processing unit), a ROM 14b (read only memory), a RAM 14c (random access memory), a display control unit 14d, an audio control unit 14e, an SSD 14f (solid state drive, flash memory), and the like. ing. The CPU 14a, for example, performs image processing related to the images displayed on the display devices 8 and 12, determination of the movement target position (parking target position, target position) of the vehicle 1, and guidance route (guidance route, parking route) of the vehicle 1. , Parking guidance route), determination of presence / absence of interference with an object, automatic control of the vehicle 1, and cancellation of automatic control, etc., can be executed. The CPU 14a can read a program installed and stored in a non-volatile storage device such as the ROM 14b, and execute arithmetic processing according to the program. The RAM 14c temporarily stores various types of data used in computations by the CPU 14a. In addition, the display control unit 14 d mainly executes synthesis of image data displayed on the display device 8 among the arithmetic processing in the ECU 14. In addition, the voice control unit 14 e mainly executes processing of voice data output from the voice output device 9 among the calculation processes in the ECU 14. The SSD 14f is a rewritable nonvolatile storage unit that can store data even when the ECU 14 is powered off. The CPU 14a, the ROM 14b, the RAM 14c, and the like can be integrated in the same package. Further, the ECU 14 may have a configuration in which another logical operation processor such as a DSP (digital signal processor) or a logic circuit is used instead of the CPU 14a. Further, an HDD (hard disk drive) may be provided instead of the SSD 14f, and the SSD 14f and the HDD may be provided separately from the ECU 14.

  The brake system 18 includes, for example, an anti-lock brake system (ABS) that suppresses brake locking, an anti-slip device (ESC: electronic stability control) that suppresses side slip of the vehicle 1 during cornering, and enhances braking force ( Electric brake system that executes brake assist, BBW (brake by wire), etc. The brake system 18 applies a braking force to the wheels 3 and thus to the vehicle 1 via the actuator 18a. The brake system 18 can execute various controls by detecting brake lock, idle rotation of the wheels 3, signs of skidding, and the like from the difference in rotation between the left and right wheels 3. The brake sensor 18b is a sensor that detects the position of the movable part of the braking operation unit 6, for example. The brake sensor 18b can detect the position of a brake pedal as a movable part. The brake sensor 18b includes a displacement sensor.

  The steering angle sensor 19 is, for example, a sensor that detects the steering amount of the steering unit 4 such as a steering wheel. The rudder angle sensor 19 is configured using, for example, a hall element. The ECU 14 obtains the steering amount of the steering unit 4 by the driver, the steering amount of each wheel 3 during automatic steering, and the like from the steering angle sensor 19 and executes various controls. The rudder angle sensor 19 detects the rotation angle of the rotating part included in the steering unit 4. The rudder angle sensor 19 is an example of an angle sensor.

  The accelerator sensor 20 is, for example, a sensor that detects the position of the movable part of the acceleration operation unit 5. The accelerator sensor 20 can detect the position of an accelerator pedal as a movable part. The accelerator sensor 20 includes a displacement sensor.

  The shift sensor 21 is, for example, a sensor that detects the position of the movable part of the speed change operation unit 7. The shift sensor 21 can detect the position of a lever, arm, button, or the like as a movable part. The shift sensor 21 may include a displacement sensor or may be configured as a switch.

  The wheel speed sensor 22 is a sensor that detects the amount of rotation of the wheel 3 and the number of rotations per unit time. The wheel speed sensor 22 outputs a wheel speed pulse number indicating the detected rotation speed as a sensor value. The wheel speed sensor 22 may be configured using, for example, a hall element. The ECU 14 calculates the amount of movement of the vehicle 1 based on the sensor value acquired from the wheel speed sensor 22 and executes various controls. Note that the wheel speed sensor 22 may be provided in the brake system 18. In that case, the ECU 14 acquires the detection result of the wheel speed sensor 22 via the brake system 18.

  The configuration, arrangement, electrical connection form, and the like of the various sensors and actuators described above are examples, and can be set (changed) in various ways.

  As an example, the ECU 14 that realizes the periphery monitoring system faces one first side region (for example, a region in the right direction with reference to the vehicle 1) based on the captured image data captured by the imaging unit 15. The first line-of-sight image data is acquired and displayed on the display device 8. Similarly, the ECU 14 generates second line-of-sight image data that faces the other second side region (for example, a region in the left direction with reference to the vehicle 1) based on the captured image data captured by the imaging unit 15. It is acquired and displayed on the display device 8. At this time, the ECU 14 displays an image based on the second line-of-sight image data instead of the image based on the first line-of-sight image data after causing the display device 8 to display an image based on the first line-of-sight image data. ECU14 implement | achieves the operation | movement which performs the safety check of the right side which a user often performs when approaching the vehicle 1 to an intersection, and the subsequent left side safety check, for example by displaying each side image on the display apparatus 8. doing. Note that the ECU 14 may perform a display such as “right”, “left”, and “right” so that the right confirmation is performed again, such as when the user confirms the left and right.

  For example, consider a case where the vehicle 1 enters an intersection with a road intersecting with the current road from the current road or when entering the road from a parked state. In this case, the user (driver) may try to check the right and left direction by taking a forward leaning posture while sitting in the driver's seat while gradually approaching the front portion of the vehicle 1 into the intersection or road. . On the other hand, in the case of the vehicle 1 according to the present embodiment, as described above, the imaging unit 15c is provided at the front end 2c in the vehicle front-rear direction. And this imaging part 15c can image | photograph the range of 140 degrees-220 degrees, for example in a horizontal direction. That is, if at least the end 2c on the front side of the vehicle 1 enters an intersection or the like, the imaging unit 15c can be connected to the first side even when the user is in a position before the intersection (a position where the left and right directions of the intersection cannot be seen). Information on a region (for example, a right side region) and a second side region (for example, a left side region) can be captured. Then, the ECU 14 displays an image of the right side region on the display device 8 based on the acquired captured image data, and subsequently (instead of) displaying an image of the left side region on the display device 8, thereby Without changing the seating posture, it is possible to check the right and left of the road to be moved forward on the screen of the display device 8. By using the captured image data captured by the imaging unit 15a disposed at the rear end 2e in the vehicle front-rear direction, left and right side images are displayed even when the vehicle 1 is traveling backward while entering the intersection. It can be displayed on the device 8. That is, it is possible to check the right and left of the road to be moved backward on the screen of the display device 8 without changing the seating posture of the user.

  The CPU 14a included in the ECU 14 includes an image acquisition unit 30 as shown in FIG. 5 in order to provide image display based on the first and second line-of-sight image data for left-right confirmation as described above. An operation receiving unit 32, a data control unit 34 (control unit), and the like are included. The data control unit 34 includes, for example, an image conversion unit 36. The image acquisition unit 30, the operation reception unit 32, the data control unit 34, and the image conversion unit 36 can be realized by reading a program installed and stored in a storage device such as the ROM 14b and executing it.

  The image acquisition unit 30 acquires captured image data output from the imaging unit 15 provided in the vehicle 1 and imaging the periphery of the vehicle 1 via the display control unit 14d. The display control unit 14d may output the captured image data captured by the imaging unit 15 to the display device 8 or the display device 12 as it is. In this case, the display contents desired by the user may be selected using an input device such as the operation unit 14g. That is, the display control unit 14d selectively displays an image selected by the operation of the operation unit 14g. For example, a rear image of the vehicle 1 captured by the imaging unit 15a can be displayed, or a left side image captured by the imaging unit 15d can be displayed.

  The operation receiving unit 32 acquires a signal generated by an operation input from the operation input unit 10 or the operation unit 14g. The operation unit 14g is, for example, a push button, a toggle switch, a rotary switch, or the like, and a peripheral monitoring image (a left side region, for example) indicating a first side region (for example, a right side region) or a second side region (for example, a left side region) by the user. (Right / left confirmation image) display request, cancellation, and other various settings can be made. Further, the operation receiving unit 32 may receive an ON signal of an ignition switch (not shown), and may be regarded as having acquired a display request for a peripheral monitoring image (left / right confirmation image) triggered by the ON signal. For example, when the user gets into the vehicle 1 and turns on the ignition switch, the left and right confirmation image is automatically displayed to notify the user of the situation around the vehicle 1 before starting the vehicle 1. May be. Moreover, the operation reception part 32 may receive the signal which shows another start. For example, the operation reception unit 32 may receive a signal indicating movement of the shift operation unit 7 to the “D range”, movement to the “R range”, a parking brake OFF operation, or the like. Since these signals can also be signals indicating the intention to start the vehicle 1, before the user starts the vehicle 1, the left and right confirmation images are automatically displayed so as to notify the user of the situation around the vehicle 1. You may make it display.

  The image conversion unit 36 included in the data control unit 34 converts the captured image data acquired by the image acquisition unit 30 into an image having the same content as a sight to be seen when the user performs left / right confirmation. For example, consider a case where the vehicle 1 enters an intersection by traveling forward. In this case, captured image data captured by the imaging unit 15c is used. As described above, since the imaging unit 15c is equipped with a wide-angle lens or a fish-eye lens, it is possible to acquire a wide-angle image data by photographing a range of, for example, 190 ° in the horizontal direction. That is, a wide range of image data including the traveling direction (for example, forward) of the vehicle 1 can be efficiently acquired. The captured image data captured by the imaging unit 15 c includes a wide range of information, but the peripheral portion of the image is greatly distorted, and when displayed on the display device 8, content different from the real world is displayed. In addition, since the line-of-sight direction (optical axis direction) of the imaging unit 15c faces the front direction of the vehicle 1, the first side region (right side region) or the second side in the captured image data captured by the imaging unit 15c. The image in the side area (left side area) is a perspective image. As a result, it may be difficult for an unfamiliar user to understand the image contents of the right side region and the left side region. Therefore, the image conversion unit 36 performs an image viewpoint conversion process and a line-of-sight conversion process based on the captured image data, and performs a distortion correction process. In addition, the trimming process is performed so that an image corresponding to a range that can be visually recognized when the user stands at the front end of the vehicle 1 and looks to the right side is converted into first line-of-sight image data. The image conversion unit 36 performs the same processing and converts it into second line-of-sight image data corresponding to a range that can be visually recognized when the user stands at the front end of the vehicle 1 and looks to the left side. The viewpoint conversion process and the line-of-sight conversion process can be executed by various known processing methods. As an example, the image conversion unit 36 applies the conversion information of the mapping table held in, for example, the ROM 14b to the captured image data including the right side region, the left side region, and the front region of the vehicle 1 captured by the imaging unit 15c. Thus, converted image data is generated. For example, the image conversion unit 36 generates a planar image (sometimes referred to as first planar image data) that faces the first lateral region from the vehicle front end position in the traveling direction of the vehicle 1 as the captured image data and generates the first line of sight. Let it be image data. Similarly, a planar image (sometimes referred to as second planar image data) facing the second side region is generated.

  In the case of the present embodiment, the ROM 14b is configured to display, for example, first line-of-sight image data (first) that can be displayed when the user stands near the front left corner of the vehicle 1 and faces the right side of the vehicle 1. A mapping table that can be generated from captured image data is stored in advance. Similarly, the second line-of-sight image data (second plane image data) that can be displayed when the user stands near the front right corner of the vehicle 1 and faces the left side of the vehicle 1 is captured image data. A mapping table that can be generated from is stored in advance. The ECU 14 executes image display using the first line-of-sight image data (first plane image data), and then executes image display using the second line-of-sight image data (second plane image data) instead. This switching may be performed immediately after the display of the image based on the first line-of-sight image data, to the display of the image based on the second line-of-sight image data. Further, an image based on the third line-of-sight image data facing the front area including the traveling direction of the vehicle 1 is displayed while switching from the image display based on the first line-of-sight image data to the image display based on the second line-of-sight image data. It may be. That is, the ECU 14 may switch display while gradually transitioning from displaying an image based on the first line-of-sight image data to displaying an image based on the second line-of-sight image data. In this case, for example, the ROM 14b can generate third line-of-sight image data that can display contents visible when the user stands near the front center of the vehicle 1 and faces the front of the vehicle 1 from the captured image data. A mapping table is held in advance. Note that the conversion information of the mapping table is preferably prepared for each line-of-sight direction. In this case, when a transition display including an image based on the third line-of-sight image data is performed, it is desirable that the display is a continuous and smooth moving image. Therefore, the conversion information of the mapping table is prepared so that the line-of-sight image data can be generated every time the line-of-sight direction changes by, for example, A °. Note that if the number of pieces of conversion information in the mapping table is increased, the number of line-of-sight image data to be generated (number of frames) can be increased to create a smooth moving image, but the processing load on the CPU 14a tends to increase. Conversely, if the number of pieces of conversion information in the mapping table is reduced, the number of line-of-sight image data generated may be reduced and the quality of the transition image (moving image) may be reduced, but the processing load on the CPU 14a can be reduced. Therefore, the number of conversion information in the mapping table can be determined according to the quality of the required image (substantial moving image). When the number of pieces of conversion information in the mapping table is reduced, when moving images are created, interpolation processing based on previous and next line-of-sight image data may be performed to correct the moving images (frame drop correction). In the following description, the ECU 14 sandwiches the display of the image based on the third visual line image data between the display of the image based on the first visual line image data and the display of the image based on the second visual line image data, and gradually changes the visual line direction. The case where the display is switched while transitioning is shown.

  The data control unit 34 arranges the plurality of line-of-sight image data generated by the image conversion unit 36 such that the line-of-sight direction is moving from right to left or from left to right of the vehicle 1, and display control is performed. It is supplied to the part 14d and displayed on the display device 8. FIG. 6 is a display example in which a moving image for left / right confirmation displayed on the display device 8 is simply shown in a frame advance format. For the sake of explanation, FIG. 6 shows a schematic scene in which, for example, cubic objects 40 and 42 (obstacles) exist diagonally forward from the left and right of the vehicle 1.

  The ECU 14 first displays the entire vehicle 1 and its entire periphery in a bird's-eye view on the display device 8 (screen P1 to screen P3). This image can be obtained by performing known viewpoint conversion processing and distortion correction processing on the captured image data obtained by the imaging units 15a to 15d. By displaying such a bird's-eye view image, a rough outline of the situation around the vehicle 1 can be presented to the user. Further, the display of the bird's-eye view image can be used as a start demonstration (screen P1 to screen P4) for notifying the user that an image display for left / right confirmation will be started. In addition, the information regarding all the external shapes of the vehicle 1 displayed on a bird's-eye view image cannot be acquired in the captured image data of the imaging units 15a to 15d. Therefore, it is desirable to store the external data of the vehicle 1 used in the overhead image in advance in the ROM 14b or the SSD 14f.

  Subsequently, the ECU 14 displays an image of the front area of the vehicle 1 based on the third line-of-sight image data (screen P4), and then displays the first side area, that is, an enlarged display on the right side of the vehicle 1 (rightward direction). (Status confirmation image) is performed (screen P5 to screen P7). This image can be obtained by performing well-known viewpoint conversion processing and line-of-sight conversion processing and distortion correction processing on the captured image data obtained by the imaging unit 15c. That is, it is possible to display the same information on the display device 8 as when the user stands at the front end of the vehicle 1 and turns to the right from the front to check the surrounding situation. In this case, since the display area is mainly limited to the right side of the vehicle 1, information on the right side can be enlarged and displayed as compared with the case where the left and right and front information of the vehicle 1 is displayed on the display device 8 at a time. More detailed information on the right side can be provided to the user, and the display content can be concentrated. Further, by performing a moving image display indicating a transition from the front area to the first side area (right side area), it is possible to make it easier for the user to recognize which direction the image is currently being displayed. . In the present embodiment, the virtual viewpoint position is set near the left front corner of the vehicle 1 when displaying the first side area. By setting the virtual viewpoint position at such a position, the right side area of the vehicle 1 and the front area on the left side of the position where the imaging unit 15c is arranged are also displayed within the display range of the first side area. Thus, it is possible to provide the user with more information, in particular, information on the position close to the front bumper that is difficult for the user to visually confirm without substantially reducing the enlargement display magnification by the right-side limited display. For example, even when a small animal such as a cat is present in the immediate vicinity of the front bumper or when a child or the like is bent, the situation can be easily recognized by the user. In addition, when displaying the image of the first lateral region, the ECU 14 may display a fixed line-of-sight image in which the line-of-sight direction does not change in a state where the line-of-sight is almost right side (for example, the screen P6). The display period of the line-of-sight fixed image can be set as appropriate, but can be set to several seconds (for example, 2 seconds). By displaying the fixed line-of-sight image, it is possible to confirm in detail the surrounding conditions of a part of the right side area and the front area of the vehicle 1. Note that the first line-of-sight image data is updated even during the display of the line-of-sight fixed image. For example, a state in which an object (another vehicle, a pedestrian, or the like) moves is displayed on the display device 8 in real time. In addition, since the line-of-sight fixed image does not move the line of sight compared to the moving image, it is easy to confirm the display content and it can contribute to the reduction of eye strain.

  By the way, the user (driver) can visually check the front area (at least a few meters ahead of the vehicle 1) through the front window. Therefore, the priority order for recognizing the contents of the front area (the display contents of the third line-of-sight image data) in the image displayed on the display device 8 is the contents of the side area when viewed from the front end of the vehicle, which is difficult for the user to see directly ( The display content of the first line-of-sight image data and the display content of the second line-of-sight image data are considered to be low. Therefore, in the present embodiment, the ECU 14 shortens the image display period based on the third line-of-sight image data from the image display period based on the first line-of-sight image data and the image display period based on the second line-of-sight image data. Yes. That is, the transition speed (moving picture display speed) of the image from the front area to the right area is made faster than the image transition speeds of the screens P5 to P7 that display the right area. That is, the screen P4 to the screen P5 are fast-forward display, the period from the screen P5 to the screen P6 and the period from the screen P6 to the screen P7 through the display of the fixed line-of-sight image on the screen P6 are slower than the first speed. It is displayed at a certain second speed (for example, slow display). When the display scene reaches the display range using the third line-of-sight image data (after the screen P7), the moving image display from the screen P8 to the screen P9 is executed at the first speed faster than the second speed. A second speed (for example, slow display) that is a transition speed of an image that is slower than the first speed through the period from the screen P9 to the screen P10 and the display of the fixed line-of-sight image on the screen P10. Is displayed. That is, the ECU 14 (data control unit 34) includes the image of the side area including the line-of-sight fixed image in order to recognize the situation (display contents) of the side area of the vehicle 1 displayed on the display device 8 in sufficient detail. Display a slow video. On the other hand, for the front area corresponding to the contents that the user can see through the front window, the type of the side area (right side area or left side area) is confirmed and the image display of the left and right side areas is continuously performed. Can be used to make it look like By changing the transition speed of the image for each display area, the operation when the user actually confirms left and right, for example, turning the neck from the front to the right and confirming the right side, then the neck to the left A display close to the actual operation of checking the left side by turning can be performed on the display device 8. As a result, it is possible to realize a display that can be easily accepted by a user who is not accustomed to confirming the surrounding situation with the content displayed on the display device 8. Further, by performing fast-forward display for a display period with a low recognition priority between a series of moving image displays, it is possible to reduce a burden (fatigue) on the user who views the display device 8. As a result, conversely, it becomes easier to concentrate on the display of the fixed line-of-sight image and the slowly displayed portion, which can contribute to an improvement in the degree of recognition of the display content.

  In another embodiment, the moving image is displayed from the screen P4 to the screen P6 at the first speed, the fixed line-of-sight image is displayed on the screen P6, and the period from the screen P6 to the screen P7 is set to the second speed (for example, the slow speed). Display). That is, the display control may be performed so as to reproduce the user's action of swinging the neck quickly from the front to the right end and then turning to the left while slowly checking the right side situation. Similarly, the moving images are displayed from the screen P7 to the screen P10 at the first speed, the fixed line-of-sight image is displayed on the screen P10, and then the period from the screen P10 to the screen P11 is displayed at the second speed (for example, slow display). You may do it. In other words, after confirming the right side area, display control is performed so as to reproduce the user's action of swinging the neck quickly through the front to the left end and then turning right while slowly checking the situation on the left side. You may go.

  In the case of the example in FIG. 6, the ECU 14 displays an image based on the first line-of-sight image data (displays the first side area, displays the right side area, and displays the screens P5 to P7), the second line-of-sight image. Following the display of the image based on the data (display of the second side area, display of the left side area, display of the screens P9 to P11), the display of the image based on the first line-of-sight image data (first display) is performed again via the screen P12. Display of the side area, display of the right side area, and display of screens P13 to P15). By performing such a display, for example, a display closer to the confirmation operation in order of “right”, “left”, and “right”, which is generally performed by the user, can be realized, and the user's sense of security is further improved. In addition, it is possible to further improve the accuracy of confirmation of the side region.

  When the display corresponding to the confirmation operation in the order of “right”, “left”, and “right” is completed, the ECU 14 sets the image transition speed to the first speed, and transitions from the front area display to the overhead view display of the vehicle 1. An end demo (screen P16 to screen P18) to be displayed is displayed to notify the user that the display for a series of side monitoring (peripheral monitoring) has ended.

  According to such a periphery monitoring system 100, it is possible to provide an image display that reproduces “right / left confirmation” that is generally performed by a user. As a result, it is possible to provide a peripheral monitoring image using the display device 8 in a manner that is easily received by the user. In addition, since the image of the first side region (for example, the right side of the vehicle 1) and the image of the second side region (for example, the left side of the vehicle 1) are displayed separately, the limited display region of the display device 8 Can display individual images as large as possible. As a result, the possibility of overlooking noticeable information can be reduced.

  In the example illustrated in FIG. 6, the same image as when the user stands on the front side of the vehicle 1 and confirms the left and right sides is displayed using the captured image data captured by the imaging unit 15 c on the front side of the vehicle 1. An example to do. In another embodiment, even when a user stands on the rear side of the vehicle 1 and confirms the left and right sides using the captured image data captured by the rear imaging unit 15a of the vehicle 1, the same image is displayed. Good. For example, it is possible to provide an image of the side area that is effective when the vehicle 1 leaves the parking area during backward traveling or when the vehicle 1 enters the traveling road intersecting with the alley from backward traveling. In this case, since the user can check the display device 8 while keeping the normal sitting posture in the driver's seat and does not need to twist the posture, the user's burden reduction effect is great.

  The operation of the periphery monitoring system 100 that performs the display as described above will be described with reference to the flowchart of FIG. Note that the flowchart of FIG. 7 is an example in which the images of the side areas are mainly displayed in the order of “right”, “left”, and “right”.

  The CPU 14a (ECU 14) first determines whether or not there is a display request for the side region image (S100). For example, the CPU 14a checks whether or not there is a display request for the side area image from the user by an input operation using the operation input unit 10 or the operation unit 14g. In this case, the traveling direction of the vehicle 1 is input by the user, and the captured image data (imaging unit 15) used for displaying the side region is determined (manual display mode). In another embodiment, for example, when a parking assistance system is provided, the ECU 14 can store the attitude when the vehicle is parked. That is, it is possible to acquire in advance information on whether to go out in forward travel or to go out in reverse travel. Therefore, the ECU 14 may determine the captured image data to be used (imaging unit 15) based on information at the time of parking assistance. Further, the ECU 14 may use operation state information of the speed change operation unit 7. For example, when the speed change operation unit 7 is in the “P range”, it can be considered that the vehicle is going to start or enter an intersection in forward travel. In the case of the “R range”, it can be considered that the vehicle is going to start or enter an intersection in reverse travel. Moreover, when it is going to approach an intersection at low speed, you may acquire the information of the position and the advancing direction of the vehicle 1 from a navigation system. Then, the ECU 14 considers that there is a display request when the vehicle 1 travels at a predetermined speed or less (for example, 5 km / h or less) and is about to enter an intersection, and imaging that is used according to the traveling direction of the vehicle 1 Image data (imaging unit 15) may be determined (automatic display mode). In addition, the display request may be input using a voice recognition device. Note that the imaging units 15a to 15d and the distance measuring units 16 and 17 may always acquire information indicating a surrounding situation, or may acquire information on a necessary portion (direction).

  When determining that “no display request” (No in S100), the CPU 14a temporarily ends this flow. On the other hand, when the CPU 14a determines that “display request is present” (Yes in S100), the image acquisition unit 30 captures captured image data around the vehicle 1 captured by the imaging units 15a to 15d via the display control unit 14d ( Current image) is acquired (S102). Then, the image conversion unit 36 generates (converts) the line-of-sight image data (including the overhead image data) in each line-of-sight direction using the conversion information of the mapping table stored in the ROM 14b for the acquired captured image data (S104). ).

  The data control unit 34 confirms whether or not the operation receiving unit 32 has received a request for redisplaying a side area described later (S106). When the re-display request is not accepted (No in S106), the data control unit 34 starts the demonstration that indicates that the display of the front-side or rear-side side area image requested to be displayed is started. Display is started (S108). In the case of FIG. 6, screens P <b> 1 to P <b> 3 are diagrams for explaining the transition of the demo screen. 7 shows an example in which an image corresponding to a side area that can be confirmed when the user stands on the front side of the vehicle 1 is displayed. In the case of FIG. 6, the start demonstration image shows an example in which the virtual viewpoint position moves from the bird's-eye view state of the vehicle 1 to the rear of the vehicle 1 and descends to the user's line-of-sight height. By displaying such a bird's-eye view image, it is possible to make the user grasp the rough situation around the vehicle 1 while displaying the demo image. The demo image is not limited to such a bird's-eye view image. For example, the display of the bird's-eye view images such as the screens P1 to P3 is omitted, and the user is informed that the display of the side region image is started by a message. It may be.

  When the display of the demo image proceeds and the front image (screen P4, front region) based on the third line-of-sight image data has been displayed (Yes in S110), the data control unit 34 converts the display image to the right image (first image at the first speed). A transition is made to an image of the first lateral region based on the first line-of-sight image data (S112). That is, a transition image (moving image) that reproduces a state in which the user turns the neck to the right at the position of the front bumper of the vehicle 1 is displayed (screen P4 to screen P6). Then, the transition image is stopped for a predetermined period (for example, 2 seconds) in the image at the terminal position on the right side (screen P6), and the fixed line-of-sight image is displayed (S114). By displaying the fixed line-of-sight image, the content of the screen P6 can be confirmed in detail by the user. Further, since the screen P6 is limited to the right side area, the right side area can be enlarged and the image content of the right side area can be confirmed in detail by the user with the limited display area of the display device 8. Can be made. If the front area image display has not yet been performed in S110 (No in S110), the data control unit 34 determines that the start demonstration has not been completed yet, returns to S108, and overlooks the start demonstration. Continue to display using image data. The transition speed of the demo image may be the first speed or a speed slower than the first speed.

  When the display of the fixed line-of-sight image (screen P6) for a predetermined period (for example, 2 seconds) is completed, the data control unit 34 gradually moves the line-of-sight direction to the front direction at a second speed that is lower than the first speed. In this manner, a plurality of first line-of-sight image data having different line-of-sight directions are sequentially displayed (S116). That is, the data control unit 34 displays a moving image at the second speed (screen P6 to screen P7). In this case as well, the transition speed of the image is slower than the transition from the front area to the right area, so that the user can easily and in detail confirm the image contents while transitioning. It is possible to reduce the time burden (eye strain associated with movement of the viewpoint, etc.).

  Subsequently, the data control unit 34 checks whether or not the transition image has reached the image of the front area (front image area) (S118). That is, when the transition to the display range of the third visual line image data has not yet been made (No in S118), the process returns to S116, and the transition display of the first visual line image data at the second speed is continued. On the other hand, when the transition image reaches the image of the front area (front image area) (Yes in S118), the data control unit 34 switches the transition speed to the first speed, and changes the front area using the third line-of-sight image data. Transition to the image and the left image (the image of the second lateral region based on the second line-of-sight image data) (S120). That is, a transition image (moving image) that reproduces a state in which the user turns the neck to the left at the position of the front bumper of the vehicle 1 is executed (screen P7 to screen P10). Then, the transition image is stopped for a predetermined period (for example, 2 seconds) in the image at the terminal position on the left side (screen P10), and the fixed line-of-sight image is displayed (S122). By displaying the fixed line-of-sight image also in the left side region, the user can confirm the contents of the screen P10 in detail and easily. Further, since the screen P10 is displayed limited to the left side area, the left side area can be enlarged and the user can check the image contents of the left side area in detail with the limited display area of the display device 8. Can be made.

  When the display of the fixed line-of-sight image (screen P10) for a predetermined period (for example, 2 seconds) is completed, the data control unit 34 determines that the transition speed is lower than the first speed as in the case where the line of sight is shifted to the left side. A plurality of second line-of-sight image data with different line-of-sight directions are sequentially displayed so that the line-of-sight direction gradually moves in the front direction at two speeds (S124). That is, the data control unit 34 performs moving image display at the second speed (screen P10 to screen P11). In this case as well, the transition speed of the image is slower than the transition from the front area to the left area, so that the user can easily confirm the details of the image during the transition and confirm the details. It is possible to reduce the time burden (eye strain associated with movement of the viewpoint, etc.).

  Subsequently, the data control unit 34 executes processing for displaying the image of the right side area again. That is, it is confirmed whether or not the transition image has reached the image of the front area (front image area) (S126). If the transition has not yet been made to the display range of the third visual line image data (No in S126), the process returns to S124, and the transition display of the second visual line image data at the second speed is continued. On the other hand, when the transition image reaches the image of the front area (front image area) (Yes in S126), the data control unit 34 switches the transition speed to the first speed and the front area using the third line-of-sight image data. Transition to the image and the right image (image of the first lateral region based on the first line-of-sight image data) (S128). That is, a transition image (moving image) that reproduces a state in which the user turns the neck to the right at the position of the front bumper of the vehicle 1 is displayed (screen P11 to screen P14). Then, the transition image is stopped for a predetermined period (for example, 2 seconds) in the right end position image (screen P14), and the fixed line-of-sight image is displayed (S130). By displaying the fixed line-of-sight image in the right side area again, it is possible to make the user confirm the details of the screen P14 in detail and also to reconfirm the status of the right side area that was first confirmed. As well as more reliable information about the surroundings of the vehicle 1 can be provided.

  The image of the right side area displayed for the second time may be displayed using the first line-of-sight image data based on the captured image data newly acquired by the imaging unit 15c at that time. In another embodiment, the image of the right-side area displayed for the first time may be displayed again. Since the side images using the first line-of-sight image data and the second line-of-sight image data are enlarged and displayed as described above, it is possible to provide information up to a distance. For example, the user can recognize an object such as a vehicle approaching from a distance at an early stage. That is, it is considered that there is no significant difference between the first right side image and the display content of the second right side image after the left side display. Therefore, it is also possible to display the first display of the right side area as an image for reconfirmation. In this case, there is an advantage that the image processing burden on the CPU 14a can be reduced.

  When the data control unit 34 executes the second display of the right-side image (screen P14) for a predetermined period (for example, 2 seconds) to complete the display of the fixed gaze image, the transition speed is slower than the first speed. A plurality of second line-of-sight image data having different line-of-sight directions are sequentially displayed so that the line-of-sight direction gradually moves in the front direction at the second speed (S132). That is, the data control unit 34 displays a moving image at the second speed (screen P14 to screen P15). Also in this case, the transition speed of the image is slower than the transition from the front area to the right area, so that the user can easily confirm the image contents while transitioning, and at the time of confirmation It is possible to reduce the burden (eye strain associated with the movement of the viewpoint).

  Subsequently, the data control unit 34 checks whether or not there is a request for redisplaying the side area (S134). For example, when the user does not operate the operation input unit 10 or the operation unit 14g and the request for redisplaying the side region image is not input (No in S134), the data control unit 34 displays the third line-of-sight image data. The display of the transition image is stopped on the image (screen P16) of the front area by (S136). Then, an end demonstration indicating that the display of the side area image is ended is displayed (S138). That is, the display content is switched from the screen P16 that is an image of the front area of the user's line-of-sight height to an overhead image that overlooks the vehicle 1 (screen P17, screen P18), and image display processing for confirming a series of side areas. Exit.

  On the other hand, in S134, when the operation accepting unit 32 accepts that a request for redisplaying the side area image by the operation of the operation input unit 10 or the operation unit 14g by the user is input (Yes in S134), the process of S102 Migrate to That is, the peripheral captured image data is acquired again, and line-of-sight image data in each line-of-sight direction is generated (converted) (S104). In this case, the generation of the overhead image data may be omitted. In S134, since a redisplay request has been made, in the process of S106, a redisplay request is in progress (Yes in S106), the process proceeds to S118, and the subsequent processes are executed. That is, an image of the second lateral region (left side region) is displayed again using the second line-of-sight image data, and an image of the first lateral region (right side region) based on the first line-of-sight image data is displayed again. Continue the process to check the left and right situation. For example, when a user goes out on a travel route with a large amount of traffic, the user may repeatedly check the left and right until a safe start can be made. According to the periphery monitoring system 100 of the present embodiment, such a display that reproduces the continuous reconfirmation operation can be performed. As a result, a display closer to the user's confirmation operation can be realized.

  Since it is desirable that the redisplay request can be made in the driving operation posture, for example, a redisplay request switch may be provided on the steering wheel, or a redisplay request may be received by a voice recognition system. Further, depending on the display contents of the side area, it may be desired to change the display period of the fixed line-of-sight image in the left-side area or the fixed line-of-sight image in the right side area. For example, when there is a large amount of traffic, the fixed line of sight image may be displayed over, for example, 2 seconds, which is the standard display period of the fixed line of sight image, so that the change in traffic volume can be confirmed. Conversely, when the traffic volume is low and the situation in the side area can be grasped instantaneously, the display period is shortened from the standard gaze-fixed image display period, for example, 2 seconds, and the lateral direction in the reverse direction The area may be displayed quickly. In this way, by making it possible to change the display period of the line-of-sight fixed image, it is possible to realize a display that is closer to the left / right confirmation operation that the user normally performs. In addition, since it is desirable that the display period of the line-of-sight fixed image can be adjusted in the driving operation posture, for example, the change setting may be made by a re-display request switch provided on the steering wheel or a voice recognition system. . Similarly, the second speed when the image is transitioned from the screen P6 to the screen P7 may be changed according to the display content of the side area. For example, when the user determines that there is no object to be noted in the image of the side area, the display may be made quickly to display the other side area. Conversely, when it is desired to confirm the side area in detail, the transition speed may be slowed down. Further, the line of sight may be fixed during the transition. For example, when a target object is found during the transition, a fixed gaze image may be displayed in the gaze direction. Further, while displaying the fixed line-of-sight image, it may be possible to display the enlarged image around an arbitrary position on the image. In this way, by enabling the selection of the display timing of the line-of-sight fixed image, the adjustment of the length of the display period, and the transition speed of the image, the image in a state closer to the case where the user himself checks the surrounding situation of the vehicle 1 Display becomes possible, and it can contribute to the improvement of the recognition of the side area using the display device 8.

  8 and 9 are display examples of images displayed on the display device 8. FIG. 8 shows a case where only a stationary object (for example, the vehicle 1 that is the own vehicle) exists in the right side region, and FIG. 9 shows a movement in addition to a stationary object (for example, the vehicle 1 that is the own vehicle) in the left side region. This is a case where an object 44 (for example, a pedestrian) exists. In the case of the periphery monitoring system 100 according to the present embodiment, distance measuring units 16 and 17 using ultrasonic waves are provided. Therefore, the distance measuring units 16 and 17 function as an object detecting unit for detecting the moving object 44 and its outer shape as described above. Therefore, the image acquisition unit 30 can acquire object image data indicating the moving object 44 around the vehicle 1 based on the information provided from the distance measurement units 16 and 17. Then, the data control unit 34 superimposes an image based on the object image data on an image based on the captured image data captured by the imaging unit 15c and the like. Further, the moving object 44 may be surrounded by an emphasis frame 46 for highlighting the moving object 44, and the presence of the moving object 44 may be notified to the user. It is desirable that the highlight frame 46 be displayed so as to follow the movement of the moving object 44 so as to always alert the user. In addition, the CPU 14a may output a warning voice message via the voice control unit 14e in conjunction with the display of the moving object 44 and the highlight frame 46. For example, a voice message such as “A moving object is approaching from the left. Please be careful.” May be output. In addition, the object image data acquired from the distance measuring units 16 and 17 may be subjected to pattern recognition processing to specifically provide the type of moving object. For example, a voice message such as “a vehicle is approaching” or “a pedestrian is approaching” may be output. Thus, by providing the moving object 44, the highlight frame 46 display, the voice message, and the like, the user can be alerted more reliably. Note that the display of the moving object 44 may be an icon or a character for indicating its presence, and the same effect can be obtained. The emphasis frame 46 may be applied not only to the moving object 44 but also to a stationary object. For example, you may emphasize a parked vehicle, a roadside tree, a road surface installation thing, etc. In this case, the type of the stationary object may be identified by the pattern recognition process, and whether or not the emphasis frame 46 is superimposed, that is, what particularly calls attention and what is not so may be displayed separately.

  In the above-described embodiment, an example is shown in which the display of the side region images is displayed in the order of “right”, “left”, and “right”, but the order of “left”, “right”, and “left” is shown. May be displayed. These orders may be selected using the operation input unit 10 or the operation unit 14g. By making the display order selectable in this way, it is possible to display an image according to the user's preference (癖), and a display that can be easily accepted by the user.

  In the display example shown in FIG. 6, for example, after displaying the fixed line-of-sight image on the screen P <b> 6, the line-of-sight direction is gradually moved to the front area in the first lateral area at a slow transition speed (second speed). An example is shown. In another embodiment, after displaying the fixed line-of-sight image on the screen P6, immediately in the reverse side area, that is, displaying the fixed line-of-sight image of P10 at a fast transition speed (first speed). Also good. In this case, as shown in FIG. 6, the display range of the side area may be narrower than when the line-of-sight direction is changed at the second speed, but the content of the side area is checked with the line of sight fixed. Therefore, it is possible to further reduce the burden on the user and reduce oversight of display contents. In addition, when there is a concern that the display range of the side area becomes narrower than the case where the line-of-sight direction is changed at the second speed as described above, by adjusting the enlargement ratio when displaying the line-of-sight fixed image, The inconvenience that the display range becomes narrow can be solved.

  Although embodiments and modifications of the present invention have been described, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 2 ... Vehicle body, 8 ... Display apparatus, 14 ... ECU, 14a ... CPU, 14b ... ROM, 14d ... Display control part, 14g ... Operation part, 15, 15a, 15b, 15c, 15d ... Imaging part, 30 DESCRIPTION OF SYMBOLS ... Image acquisition part, 32 ... Operation reception part, 34 ... Data control part, 36 ... Image conversion part, 40, 42 ... Object, 44 ... Moving object, 46 ... Emphasis frame, 100 ... Perimeter monitoring system (perimeter monitoring apparatus).

Claims (6)

An image acquisition unit that is provided in a vehicle and acquires captured image data output from an imaging unit that images the periphery of the vehicle;
A display device provided in a vehicle interior of the vehicle for displaying an image;
Based on the captured image data, obtain at least first line-of-sight image data that faces one first side area with respect to the vehicle and second line-of-sight image data that faces the other second side area with respect to the vehicle, A control unit for displaying an image based on the second visual line image data instead of an image based on the first visual line image data after displaying an image based on the first visual line image data on the display device;
A peripheral monitoring device comprising: The captured image data is wide-angle image data including the traveling direction of the vehicle,
The control unit obtains the first line-of-sight image data by converting the captured image data from a vehicle tip position in a traveling direction of the vehicle to a planar image facing the first side region, and the second side image data. The periphery monitoring device according to claim 1, further comprising an image conversion unit that converts the image into a planar image that faces an area and acquires the second line-of-sight image data.   The control unit converts the third line-of-sight image data facing the front area including the traveling direction of the vehicle during the transition from the image display based on the first line-of-sight image data to the image display based on the second line-of-sight image data. The periphery monitoring apparatus according to claim 1 or 2, wherein an image based on the display is displayed.   The control unit shortens an image display period based on the third line-of-sight image data from an image display period based on the first line-of-sight image data and an image display period based on the second line-of-sight image data. The peripheral monitoring device described. The image acquisition unit further acquires object image data output from an object detection unit that is provided in a vehicle and detects a moving object around the vehicle,
The periphery monitoring device according to any one of claims 1 to 4, wherein the control unit reports the presence of the moving object by superimposing an image based on the object image data on an image based on the captured image data. .   The control unit performs display of an image based on the first line-of-sight image data again, following display of an image based on the first line-of-sight image data and display of an image based on the second line-of-sight image data. Item 6. The periphery monitoring device according to any one of items 5 to 6.

Images