WO2018003188A1

WO2018003188A1 - Periphery monitoring device

Info

Publication number: WO2018003188A1
Application number: PCT/JP2017/009743
Authority: WO
Inventors: 渡邊　一矢
Original assignee: アイシン精機株式会社
Priority date: 2016-06-30
Filing date: 2017-03-10
Publication date: 2018-01-04
Also published as: JP6642307B2; DE112017003278T5; JP2018001899A; CN109314770A; CN109314770B

Abstract

This periphery monitoring device comprises a control unit which displays, on a display unit, an image on the basis of captured image data outputted from an imaging unit that images the body surface of a vehicle and the surrounds of said body surface as an imaging range; and an image processing unit that superimposes waterline boundary lines on the basis of waterline boundary information when the vehicle is fording a body of water, in a corresponding position to the vehicle surface on the image.

Description

Perimeter monitoring device

Embodiments of the present invention relate to a periphery monitoring device.

Conventionally, there are vehicles equipped with the Wataru function. For example, a vehicle referred to as an off-road vehicle may be waterproofed so that it can travel across a swamp, a river, a flooded channel, etc. (underwater driving). In such vehicles, for example, the distance to the water surface is measured using a sensor such as an ultrasonic sonar, and a graphic image of the water surface and the vehicle is displayed on a display device provided in the passenger compartment. Some of them are equipped with a system that displays the information in a pseudo manner to the driver.

Special table 2015-512825 gazette

However, when the vehicle travels on the road, the water surface is often wavy. When the graphic image showing the water surface is displayed by detecting the distance to the water surface with a sensor such as an ultrasonic sonar as in the above-described technique, it is difficult to accurately measure the distance to the water surface due to the water surface moving up and down due to undulations. There is a case. In this case, the waved state and the height to the water surface cannot be accurately represented by a graphic image, and there is a problem in that it is difficult to appropriately provide the user with information on whether or not the water surface state is displayed and whether the traveling is appropriate.

Therefore, one of the problems of the present invention is to provide a periphery monitoring device that can appropriately provide the user with information on whether or not the water surface state is displayed and whether or not the traveling (underwater traveling) is appropriate.

The periphery monitoring device according to the embodiment of the present invention includes, for example, a control unit that causes a display device to display an image based on captured image data output from an image capturing unit that captures an image of the vehicle body surface and the periphery of the vehicle body surface as an imaging range. And an image processing unit that superimposes a water level limit line based on the water level limit information when the vehicle travels on the vehicle on a corresponding position of the vehicle body surface on the image. According to this configuration, for example, the display device displays a real image based on the captured image data captured by the imaging unit, and the actual vehicle body surface of the host vehicle and the surrounding situation during crossing travel (underwater travel), that is, the water surface The status (water level, undulation status, etc.) is displayed. Further, the water level limit line is superimposed on the corresponding position of the vehicle body surface on the image. As a result, it is easy to convey the actual situation, that is, where the water level is in the vehicle body, to the user more accurately, and it is possible to make the user intuitively grasp the situation at the time of travel.

In the periphery monitoring device, for example, the control unit may display an image obtained by correcting the captured image data obtained by capturing the side surface of the vehicle. According to this configuration, for example, the visibility of the screen content (the vehicle side surface and the water level limit line superimposed thereon) is further improved, and the actual situation, that is, where the water level is located in the vehicle body. Furthermore, it becomes easy to make a user understand correctly.

In the periphery monitoring device, for example, the control unit may perform viewpoint conversion processing on the captured image data as the correction. According to this configuration, for example, it is possible to provide an image that looks at the side of the vehicle from the side of the vehicle, and an image that makes it easier to understand the relationship between the water level limit line and the water surface and the relationship between the water surface and the vehicle body is provided. it can.

In the periphery monitoring device, for example, the control unit may perform a distortion correction process on the captured image data as the correction. According to this configuration, for example, the shape of the vehicle can be more easily recognized, and an image that makes it easier to understand the relationship between the water level limit line and the water surface and the relationship between the water surface and the vehicle body can be provided.

In the periphery monitoring device, for example, the control unit may perform a cutting process for cutting out a part of the captured image data as the correction. According to this configuration, for example, in a vehicle, it is possible to enlarge and display a specific portion that should pay attention to the relationship with the water level, and it is easier to understand the relationship between the vehicle body and the water level limit line and the relationship between the water surface and the vehicle body. Can be provided.

In the periphery monitoring device, for example, the water level limit line having a shape along the surface shape of the vehicle may be superimposed on the corresponding position. According to this configuration, for example, the deviation between the surface of the vehicle and the water level limit line is reduced, and an image that makes it easier to understand the relationship between the water level limit line and the water surface and the relationship between the water surface and the vehicle body can be provided.

In the periphery monitoring device, for example, the image processing unit may superimpose the straight water level limit line on the corresponding position. According to this configuration, for example, water level limit line superimposition processing is facilitated, and the processing load can be reduced. Further, the water level limit line is simply displayed for the vehicle, and it is possible to display intuitively the relationship between the vehicle, the water level limit line, and the water surface.

In the periphery monitoring device, for example, when the water level limit line is displayed, the control unit is configured to make the display area of the vehicle body surface in the image larger than when the water level limit line is not displayed. Also good. According to this configuration, for example, the water level limit line superimposed on the vehicle body can be easily recognized, and it can be easily recognized that the screen is switched to the display mode of the water level limit line.

Further, in the periphery monitoring device, for example, the image processing unit may superimpose a water level reference line at a position lower in the vehicle height direction than the water level limit line superimposed on the image. According to this configuration, the water level reference line can indicate the degree of the rise of the water level relative to the vehicle body in stages, so that a step-by-step alert for the water level rise can be performed.

In the periphery monitoring device, for example, the image processing unit may superimpose the water level reference line in a display mode different from the water level limit line. According to this configuration, it is possible to more clearly alert the water level rise by the water level reference line.

FIG. 1 is a perspective view illustrating an example of a state in which a part of a passenger compartment of a vehicle on which the periphery monitoring device according to the embodiment is mounted is seen through. FIG. 2 is a plan view illustrating an example of a vehicle on which the periphery monitoring device according to the embodiment is mounted. FIG. 3 is an example of a dashboard of a vehicle on which the periphery monitoring device according to the embodiment is mounted, and is a diagram in a view from the rear of the vehicle. FIG. 4 is a block diagram illustrating an example of an image control system including the periphery monitoring device according to the embodiment. FIG. 5 is an explanatory diagram illustrating a display example of a water level limit line by the periphery monitoring device according to the embodiment. FIG. 6 is a display example of the periphery monitoring apparatus according to the embodiment, and is an explanatory diagram illustrating a state in which a water level limit line and a water level reference line are superimposed on an image after distortion correction and viewpoint conversion. FIG. 7 is a display example by the periphery monitoring apparatus according to the embodiment, and is an explanatory diagram illustrating a state in which a water level limit line and a water level reference line are superimposed on another image after distortion correction and viewpoint conversion. FIG. 8 is a block diagram illustrating an example of a configuration of a CPU for realizing display of a water level limit line realized in the ECU of the periphery monitoring device according to the embodiment. FIG. 9 is a display example of the display device of the periphery monitoring device according to the embodiment, and is a diagram illustrating a screen layout and a display example in the standard display mode before displaying the water level limit line. FIG. 10 is a display example of the display device of the periphery monitoring device according to the embodiment, and is a diagram illustrating a screen layout and a display example in the special display mode after the display of the water level limit line. FIG. 11 is a flowchart for explaining an example of the display process of the periphery monitoring image including the display of the water level limit line by the periphery monitoring apparatus according to the embodiment.

Hereinafter, exemplary embodiments of the present invention will be disclosed. The configuration of the embodiment shown below and the operations, results, and effects brought about by the configuration are examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and at least one of various effects based on the basic configuration and derivative effects can be obtained. .

In the present embodiment, the vehicle 1 equipped with the periphery monitoring device (perimeter monitoring system) may be, for example, an automobile using an internal combustion engine (not shown) as a drive source, that is, an internal combustion engine automobile, or an electric motor (not shown). It may be a vehicle as a drive source, that is, an electric vehicle, a fuel cell vehicle, or the like. Moreover, the hybrid vehicle which uses both of them as a drive source may be sufficient, and the vehicle provided with the other drive source may be sufficient. Further, the vehicle 1 can be mounted with various transmissions, and various devices necessary for driving the internal combustion engine and the electric motor, such as systems and components, can be mounted. In addition, the vehicle 1 preferably travels on an “off-road” (mainly an unpaved rough road, etc.) in addition to a so-called “on-road” (mainly a paved road or an equivalent road). It is a vehicle that can be used. As a drive system, it is possible to provide a four-wheel drive vehicle that transmits drive force to all four wheels 3 and uses all four wheels as drive wheels. Various methods, numbers, layouts, and the like of devices related to driving of the wheel 3 can be set. For example, a vehicle mainly intended for “on-road” traveling may be used. Further, the driving method is not limited to the four-wheel driving method, and may be a front wheel driving method or a rear wheel driving method, for example.

As illustrated in FIG. 1, the vehicle body 2 constitutes a passenger compartment 2a in which a passenger (not shown) gets. In the passenger compartment 2a, a steering section 4, an acceleration operation section 5, a braking operation section 6, a shift operation section 7 and the like are provided in a state facing the driver's seat 2b as a passenger. The steering unit 4 is, for example, a steering wheel protruding from the dashboard 24, the acceleration operation unit 5 is, for example, an accelerator pedal positioned under the driver's feet, and the braking operation unit 6 is, for example, a driver's foot It is a brake pedal located under the foot, and the speed change operation unit 7 is, for example, a shift lever protruding from the center console. The steering unit 4, the acceleration operation unit 5, the braking operation unit 6, the speed change operation unit 7 and the like are not limited to these.

Further, a display device 8 as a display output unit and a sound output device 9 as a sound output unit are provided in the passenger compartment 2a. The display device 8 is, for example, an LCD (liquid crystal display) or an OELD (organic electroluminescent display). The audio output device 9 is, for example, a speaker. The display device 8 is covered with a transparent operation input unit 10 such as a touch panel. The occupant can visually recognize an image displayed on the display screen of the display device 8 via the operation input unit 10. In addition, the occupant can execute an operation input by touching, pushing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8. . The display device 8, the audio output device 9, the operation input unit 10, and the like are provided, for example, in the monitor device 11 that is located in the vehicle width direction of the dashboard 24, that is, the central portion in the left-right direction. The monitor device 11 can have an operation input unit (not shown) such as a switch, a dial, a joystick, and a push button. Further, a sound output device (not shown) can be provided at another position in the passenger compartment 2a different from the monitor device 11, and sound is output from the sound output device 9 of the monitor device 11 and other sound output devices. be able to. Note that the monitor device 11 can be used also as, for example, a navigation system or an audio system.

Further, a display device 12 different from the display device 8 is provided in the passenger compartment 2a. As illustrated in FIG. 3, for example, the display device 12 is provided in the instrument panel unit 25 of the dashboard 24, and between the speed display unit 25 a and the rotation speed display unit 25 b at the approximate center of the instrument panel unit 25. Is located. The size of the screen 12a of the display device 12 is smaller than the size of the screen 8a (FIG. 3) of the display device 8. The display device 12 can display an image indicating an indicator, a mark, or character information as auxiliary information when, for example, the periphery monitoring of the vehicle 1 or other functions are operating. The amount of information displayed on the display device 12 may be smaller than the amount of information displayed on the display device 8. The display device 12 is, for example, an LCD or an OELD. Information displayed on the display device 12 may be displayed on the display device 8.

Further, as illustrated in FIGS. 1 and 2, the vehicle 1 is, for example, a four-wheeled vehicle, and has two left and right front wheels 3F and two right and left rear wheels 3R. All of these four wheels 3 can be configured to be steerable. As illustrated in FIG. 4, the vehicle 1 includes a steering system 13 that steers at least two wheels 3. The steering system 13 includes an actuator 13a and a torque sensor 13b. The steering system 13 is electrically controlled by an ECU 14 (electronic control unit) or the like to operate the actuator 13a. The steering system 13 is, for example, an electric power steering system, an SBW (steer by wire) system, or the like. Moreover, the torque sensor 13b detects the torque which a driver | operator gives to the steering part 4, for example.

Further, as illustrated in FIG. 2, the vehicle body 2 is provided with, for example, four imaging units 15a to 15d as the plurality of imaging units 15. The imaging unit 15 is a digital camera that incorporates an imaging element such as a CCD (charge coupled device) or a CIS (CMOS image sensor). The imaging unit 15 can output moving image data (captured image data) at a predetermined frame rate. Each of the imaging units 15 includes a wide-angle lens or a fish-eye lens, and can capture a range of, for example, 140 ° to 220 ° in the horizontal direction. Further, the optical axis of the imaging unit 15 may be set obliquely downward. Therefore, the imaging unit 15 is configured such that the road surface on which the vehicle 1 is movable, the water surface at the time of traveling, the surrounding conditions (the presence or absence of water, the state of the water surface, the height to the water surface, etc.) The external environment around the vehicle 1 including a rock, a tree, a person, a bicycle, a vehicle, etc.) is sequentially photographed and output as captured image data.

The imaging unit 15a is located, for example, at the rear end 2e of the vehicle body 2 and is provided on a wall portion below the rear window of the rear hatch door 2h. The imaging unit 15b is located, for example, at the right end 2f of the vehicle body 2 and provided on the right door mirror 2g. The imaging unit 15c is located, for example, on the front side of the vehicle body 2, that is, the front end 2c in the vehicle front-rear direction, and is provided on a front bumper, a front grill, or the like. The imaging unit 15d is located, for example, on the left side of the vehicle body 2, that is, on the left end 2d in the vehicle width direction, and is provided on the left door mirror 2g. The ECU 14 performs arithmetic processing and image processing based on the captured image data obtained by the plurality of imaging units 15 to generate an image with a wider viewing angle, or a virtual overhead view image of the vehicle 1 viewed from above. Can be generated. In addition, the ECU 14 performs a distortion correction process for correcting distortion by performing arithmetic processing or image processing on the data of the wide-angle image (curved image data) obtained by the imaging unit 15, or an image obtained by cutting out a specific region. Or a cutting process for generating image data showing only a specific area. Further, the ECU 14 can execute viewpoint conversion processing for converting captured image data into virtual image data captured from a virtual viewpoint different from the viewpoint captured by the imaging unit 15. For example, it is converted into virtual image data indicating a bird's-eye view as if the vehicle 1 is viewed from above, or converted into virtual image data indicating a side-view image such that the side surface of the vehicle 1 is viewed from a position away from the vehicle 1. Can be. The ECU 14 displays the acquired image data on the display device 8 so that, for example, safety confirmation on the right side or left side of the vehicle 1, confirmation of a water level at the time of travel traveling, which will be described later, Provide peripheral monitoring information that can be used to confirm safety.

Further, the ECU 14 displays a part of the vehicle body 2 together with the state of the water surface and displays the relationship between the vehicle 1 and the water surface at the time of the crossing travel, and executes the crossing travel support (underwater travel support). You can also. In this case, as will be described later, the ECU 14 can display an image in which a water level limit line, a water level reference line, or the like is superimposed on a part of the vehicle 1, for example, the side surface of the vehicle body 2. The ECU 14 identifies the lane markings and the like shown on the road surface around the vehicle 1 from the captured image data provided from the imaging unit 15 and executes driving support, or detects (extracts) the parking lane and parks the vehicle. You can also provide assistance.

As illustrated in FIGS. 1 and 2, the vehicle body 2 is provided with, for example, four distance measuring sections 16a to 16d and eight distance measuring sections 17a to 17h as a plurality of

distance measuring sections

16 and 17. ing. The

distance measuring units

16 and 17 are, for example, sonar that emits ultrasonic waves and captures the reflected waves. The sonar may also be referred to as a sonar sensor, an ultrasonic detector, or an ultrasonic sonar. In the present embodiment, the

distance measuring units

16 and 17 are provided at low positions in the vehicle height direction of the vehicle 1, for example, front and rear bumpers, detect, for example, obstacles around the vehicle 1, and determine the distance to the obstacles. Can be measured. The

distance measuring units

16 and 17 can be used as sensors for determining whether or not the vehicle 1 is in an underwater approach (running) state. As described above, the

distance measuring units

16 and 17 are provided on the front and rear bumpers that are low in the vehicle height direction of the vehicle 1. Submerged in a relatively early stage before reaching the water level limit line). For example, when the

distance measuring units

16 and 17 are submerged, the reception state of the reflected wave becomes unstable and an operation error occurs. Therefore, when the vehicle 1 enters a river or a swamp and is submerged, an error signal can be output almost simultaneously by the plurality of

distance measuring units

16 and 17. For example, when the vehicle 1 travels forward and enters a river or swamp, the

distance measuring units

17e, 17f, 17g, and 17h submerge almost simultaneously and output an error signal, and then the

distance measuring units

16c and 16d An error signal is output. Similarly, when the vehicle 1 travels backward and enters a river or swamp, the

distance measuring units

17a, 17b, 17c, and 17d are submerged almost simultaneously and output an error signal, and then the

distance measuring units

16a and 16b. Outputs an error signal. That is, it is possible to obtain information for determining whether or not the vehicle 1 is in the traveling state based on the output mode of the error signals of the

distance measuring units

16 and 17. When the vehicle 1 completes the crossing travel (when landing), if the vehicle 1 escapes from the river or swamp while traveling forward, the

distance measuring units

17e, 17f, 17g, and 17h return almost simultaneously and continue. Thus, the

distance measuring sections

16c and 16d are restored. When the landing further proceeds, the

distance measuring sections

16a and 16b are restored, and finally the

distance measuring sections

17a, 17b, 17c and 17d are restored almost simultaneously. As described above, when all the

distance measuring units

16 and 17 are restored, it is possible to obtain information for determining that the vehicle 1 has escaped (complete landing) from the traveling travel. The

distance measuring units

16 and 17 have a waterproof structure and are configured so as not to be damaged by submergence.

Further, as exemplified in FIG. 4, in the periphery monitoring system 100 (perimeter monitoring device), in addition to the ECU 14, the monitor device 11, the steering system 13, the

distance measuring units

16 and 17, the brake system 18, the steering angle sensor 19, an accelerator sensor 20, a shift sensor 21, a wheel speed sensor 22, an acceleration sensor 26, and the like are electrically connected via an in-vehicle network 23 as an electric communication line. The in-vehicle network 23 is configured as a CAN (controller area network), for example. The ECU 14 can control the steering system 13, the brake system 18, and the like by sending a control signal through the in-vehicle network 23. The ECU 14 also has a torque sensor 13b, a brake sensor 18b, a rudder angle sensor 19, a distance measuring unit 16, a distance measuring unit 17, an accelerator sensor 20, a shift sensor 21, a wheel speed sensor 22, and an acceleration sensor via the in-vehicle network 23. 26, etc., the operation signal of the operation input unit 10, etc. can be received.

The ECU 14 includes, for example, a CPU 14a (central processing unit), a ROM 14b (read only memory), a RAM 14c (random access memory), a display control unit 14d, an audio control unit 14e, an SSD 14f (solid state drive, flash memory), and the like. ing. For example, the CPU 14a can execute arithmetic processing and control of image processing related to images displayed on the

display devices

8 and 12. For example, a process or calculation for displaying an image in a standard display mode displayed during land travel, an image in a special display mode displayed during crossing travel, or the like is executed. In addition, the CPU 14a determines a movement target position (parking target position, target position) of the vehicle 1, calculates a guidance route (guidance route, parking route, parking guidance route) of the vehicle 1, and determines whether there is interference with an object. Various arithmetic processes and controls such as automatic control of the vehicle 1 and cancellation of the automatic control can be executed.

The CPU 14a can read a program installed and stored in a non-volatile storage device such as the ROM 14b and execute arithmetic processing according to the program. The RAM 14c temporarily stores various types of data used in computations by the CPU 14a. In addition, the display control unit 14 d mainly executes synthesis of image data displayed on the display device 8 among the arithmetic processing in the ECU 14. In addition, the voice control unit 14 e mainly executes processing of voice data output from the voice output device 9 among the calculation processes in the ECU 14. The SSD 14f is a rewritable nonvolatile storage unit, and can store data even when the power of the ECU 14 is turned off. The CPU 14a, the ROM 14b, the RAM 14c, and the like can be integrated in the same package. Further, the ECU 14 may have a configuration in which another logic operation processor, a logic circuit, or the like such as a DSP (digital signal processor) is used instead of the CPU 14a. Further, an HDD (hard disk drive) may be provided instead of the SSD 14f, and the SSD 14f and the HDD may be provided separately from the ECU 14.

The brake system 18 is, for example, an ABS (anti-lock brake system) that suppresses the locking of the brake, a skid prevention device (ESC: electronic stability control) that suppresses the skidding of the vehicle 1 during cornering, and enhances the braking force ( Electric brake system that executes brake assist), BBW (brake by wire), etc. The brake system 18 applies a braking force to the wheels 3 and thus to the vehicle 1 via the actuator 18a. The brake system 18 can execute various controls by detecting brake lock, idle rotation of the wheels 3, signs of skidding, and the like from the difference in rotation between the left and right wheels 3. The brake sensor 18b is a sensor that detects the position of the movable part of the braking operation unit 6, for example.

The steering angle sensor 19 is a sensor that detects the steering amount of the steering unit 4 such as a steering wheel. The ECU 14 obtains the steering amount of the steering unit 4 by the driver, the steering amount of each wheel 3 during automatic steering, and the like from the steering angle sensor 19 and executes various controls. The accelerator sensor 20 is, for example, a sensor that detects the position of the movable part of the acceleration operation unit 5. The shift sensor 21 is, for example, a sensor that detects the position of the movable part of the speed change operation unit 7. The wheel speed sensor 22 is a sensor that detects the amount of rotation of the wheel 3 and the number of rotations per unit time. The wheel speed sensor 22 outputs a wheel speed pulse number indicating the detected rotation speed as a sensor value. The ECU 14 calculates the amount of movement of the vehicle 1 based on the sensor value acquired from the wheel speed sensor 22 and executes various controls. In the present embodiment, it is assumed that the vehicle 1 is provided with two acceleration sensors 26 (26a, 26b). In addition, when the vehicle 1 mounts ESC, the acceleration sensor 26 (26a, 26b) conventionally mounted in ESC is used. In addition, this embodiment does not restrict | limit an acceleration sensor, What is necessary is just a sensor which can detect the acceleration of the left-right direction of the vehicle 1. FIG. In the present embodiment, the longitudinal acceleration and the lateral acceleration are derived.

The configuration, arrangement, electrical connection form, and the like of the various sensors and actuators described above are merely examples, and can be set (changed) in various ways.

As an example, the ECU 14 that implements the periphery monitoring system 100 uses the actual image based on the captured image data captured by the imaging unit 15 during the traveling of the vehicle 1, the water surface around the vehicle 1, and the “water level limit line”. The display device 8 displays the relationship with the superimposed vehicle body 2. In this case, the captured image data captured by the imaging unit 15 is data obtained by capturing the surface of the vehicle body 2 of the vehicle 1 and the periphery of the surface as an imaging range. Here, the “water level limit line” means that even if the vehicle 1 is submerged, it is possible to prevent the water level that allows the functions provided in the vehicle 1 such as the traveling function to operate normally and water intrusion into the passenger compartment 2a. It is an index that indicates the limit of the water level.

5 to 7 are explanatory diagrams showing a state in which the “water level limit line L” is superimposed on the image based on the captured image data. The “water level limit line L” is a line that is set at a position of, for example, 600 mm in the vehicle height direction from the ground contact position of the wheel 3, and is a height that is subjected to waterproofing treatment or water stoppage treatment at the design stage of the vehicle 1. The height is set in advance depending on the driving performance of the vehicle 1 and the like. The “water level limit line L” can be superimposed and displayed, for example, along the body side surface 2 m (along the curved surface of the vehicle body 2) of the vehicle body 2 included in the image based on the captured image data captured by the imaging unit 15. . For example, when the body side surface 2m is a curved surface, the water level limit line L is also preferably a shape (curve) along the surface shape (horizontal cross-sectional shape) of the vehicle 1. Further, the water level limit line L is superimposed within the range of the vehicle body 2. In this case, the water level limit line L is superimposed including the wheel 3 portion. Thus, by superimposing the water level limit line L along the body side surface 2m and within the range of the vehicle body 2, it is possible to make the user easily recognize where the water level has risen to the vehicle body 2 without a sense of incongruity.

As described above, the imaging unit 15b and the imaging unit 15d having a wide-angle lens or a fisheye lens are fixed to the door mirror 2g. Therefore, the imaging unit 15b and the imaging unit 15d have a part of the body side surface 2m (including the front wheel 3F and the rear wheel 3R) of the vehicle 1 and a landscape on its side as an imaging range. That is, when the vehicle 1 enters the water such as a river, a marsh, or a submerged channel, an image can be obtained in which the body side surface 2m of the vehicle 1 and the water surface that has risen due to entering the water are contained in the same screen. . Then, the ECU 14 displays an image in which the “water level limit line L” is superimposed and displayed on the body side surface 2m of the vehicle 1 displayed together with the water surface on the display device 8, so that the vehicle 1 is submerged and submerged during the traveling travel. Can be visually confirmed by the user (driver or the like). Moreover, since the water surface is displayed as a real image on the screen, it is possible to make the user intuitively grasp the change in the water level. In this case, since the image to be displayed is a real image, even when the water surface is undulating, it becomes possible for the user to recognize the change in the water surface in real time, making it easier to make a situation determination at the time of travel traveling more appropriately. be able to.

The ECU 14 may superimpose the “water level reference line K” at a position lower in the vehicle height direction than the “water level limit line L” as an index similar to the “water level limit line L” as shown in FIGS. it can. The “water level reference line K” is an index line that is superimposed substantially in parallel with the water level limit line L and that notifies the user of a rise in the water level without reaching the water level limit line L in advance. For example, when two “water level reference lines K” are overlapped, one is set as “first water level reference line K1” below “water level limit line L”, for example, 400 mm, that is, 200 mm from the ground contact surface of wheel 3. Superimposed. The other line is superimposed as a “second water level reference line K2” below the “water level limit line L”, for example, 200 mm, that is, 400 mm from the ground contact surface of the wheel 3. Thus, by providing the “water level reference line K”, it is possible to indicate the level of rise of the water level with respect to the vehicle body 2 in a stepwise manner, so that a step-by-step alert to the water level rise can be performed. Note that whether or not the “water level reference line K” is displayed may be selected by the user. Further, the number of “water level reference lines K” and the superimposed position (the height of the wheel 3 from the ground contact position) can be set as appropriate.

As shown in FIG. 8, the CPU 14a included in the ECU 14 realizes the provision of an image including the “water level limit line L” at the time of crossing travel, as shown in FIG. A control unit), an image processing unit 34, and an output unit 36. The display processing unit 32 includes a distortion correction unit 38, a viewpoint conversion unit 40, and a layout adjustment unit 42. The image processing unit 34 includes a water level limit line superimposing unit 44 and a water level reference line superimposing unit 46. The image acquisition unit 30, the display processing unit 32, the image processing unit 34, the output unit 36, the distortion correction unit 38, the viewpoint conversion unit 40, the layout adjustment unit 42, the water level limit line superimposing unit 44, and the water level reference line superimposing unit 46 are the ROM 14b. It can be realized by reading a program installed and stored in a storage device such as the above and executing it.

The image acquisition unit 30 acquires captured image data output from the imaging unit 15 provided in the vehicle 1 and imaging the periphery of the vehicle 1 via the display control unit 14d. The display control unit 14d may output the captured image data captured by the imaging unit 15 to the display device 8 or the display device 12 as it is without going through the processing by the CPU 14a. In addition, the CPU 14a may cause the user to select desired display contents using an input device such as the operation input unit 10 or the operation unit 14g. That is, the display control unit 14d can selectively display the image selected by the operation of the operation input unit 10 or the operation unit 14g on the display device 8. For example, the rear image of the vehicle 1 captured by the imaging unit 15a can be displayed on the display device 8, and the left side image captured by the imaging unit 15d can be displayed on the display device 8.

The display processing unit 32 performs various image processing on the captured image data output from the imaging unit 15 that captures the surface of the vehicle body 2 of the vehicle 1 and the periphery of the surface as an imaging range, and displays an image based on the captured image data as “ The “water level limit line L” or “water level reference line K” is converted into an image that can be easily recognized and displayed on the display device 8.

For example, the distortion correction unit 38 performs a known distortion correction process on the captured image data or the image based on the captured image data. As described above, when the imaging unit 15b and the imaging unit 15d include a wide-angle lens or a fisheye lens, the original image based on the captured image data is a curved image as illustrated in FIG. In this case, the curved image is an image that includes a wide range of situations of the vehicle 1 and the front, rear, and side of the vehicle 1 in the imaging range. That is, it is possible to provide an image that allows the user to easily grasp the situation of the vehicle 1 and the entire periphery of the vehicle 1. On the other hand, a user who is unfamiliar with the curved image may take time to understand the display content. In addition, since the “water level limit line L”, which should be close to a straight line, is curved, it may give an uncomfortable feeling. Therefore, for example, the distortion correction unit 38 (CPU 14a) corrects the captured image data obtained by capturing the side surface of the vehicle 1 by applying a displacement amount corresponding to the pixel position based on the correction information held in the ROM 14b. To reduce or remove the curvature. As a result, the shape of the vehicle 1 displayed on the display device 8 can be brought close to the actual shape, and the user can easily understand the display contents.

The viewpoint conversion unit 40 performs a known viewpoint conversion process on the captured image data or the image based on the captured image data. For example, in order to make it easier for the user to understand the relationship between the “water level limit line L” or “water level reference line K” superimposed on the body side surface 2 m of the vehicle body 2 and the water surface W (water surface line), the water level limit line L It is desirable to provide an image in which the line of sight is directed from a direction in which the surface of the vehicle body 2 (for example, the body side surface 2m) on which the water level reference line K is superimposed is easily visible. For example, as shown in FIGS. 6 and 7, the image after conversion faces the body side surface 2m from a position separated from the body side surface 2m, and shows the vicinity of the contact portion (water surface line) between the body side surface 2m and the water surface W. The image is obtained with a line of sight looking down. As a result, the converted image makes it easier to understand the relationship between the body side surface 2m and the water surface W. The viewpoint conversion unit 40 (CPU 14a) applies the conversion information of the mapping table held in, for example, the ROM 14b to the captured image data obtained by capturing the side surface of the vehicle 1, and separates the vehicle 1 from the vehicle 1 by separating it. Virtual image data (viewpoint conversion image) when viewed from the facing viewpoint position is generated. 6 and 7 illustrate an example in which distortion correction and viewpoint conversion processing are performed on an image based on captured image data, but the distortion correction processing may be omitted and the viewpoint conversion processing may be executed. In this case, an image as shown in FIG. 5 is viewed from a position away from the body side surface 2m. Further, the viewpoint conversion unit 40 may perform distortion correction during the viewpoint conversion process. Further, the distortion correction unit 38 and the viewpoint conversion unit 40 may be configured as an image conversion unit.

The viewpoint conversion unit 40 can appropriately set the viewpoint position. For example, as shown in FIG. 6, a line-of-sight conversion image may be obtained in which the viewpoint is placed at a position separated from the body side surface 2 m at a substantially central portion of the body side surface 2 m and facing the front side of the vehicle 1. Further, as shown in FIG. 7, the viewpoint conversion unit 40 places a viewpoint at a position separated from the body side surface 2 m at a substantially central portion of the body side surface 2 m, and generates a line-of-sight conversion image that faces the body side surface 2 m of the vehicle 1. Also good. Similarly, it may be a line-of-sight converted image that faces the rear side of the vehicle 1 from a substantially central portion of the body side surface 2m. In this case, a clipping process for appropriately cutting out a part of the image based on the captured image data may be performed. In this way, the viewpoint position can be changed as appropriate, or the enlarged display can be appropriately performed by cutting out processing, so that the water level limit line L (water level reference line K) and the water surface W of the part that the user wants to pay attention to are displayed. An image showing the relationship can be provided, and the grasp of the crossing traveling state can be easily recognized. For example, when an image facing the body side surface 2m as shown in FIG. 7 is displayed, an image facing the body side surface 2m opposite to the driver seat side may be displayed on the display device 8. In this case, the body side surface 2m on the driver's seat side can be visually observed by the driver, and the opposite body side surface 2m can be confirmed on the display device 8. These viewpoint switching can be selected by, for example, input from the operation input unit 10 or the operation unit 14g.

The layout adjustment unit 42 adjusts (changes) the layout of the screen displayed on the display device 8. As shown in FIG. 9, the ECU 14 divides the display area of the display device 8 into a plurality of images and displays clinometers 50 indicating the orientation of the vehicle 1 as a travel support screen that is normally displayed on the display device 8. Is displayed. FIG. 9 is an example of an image in the standard display mode that is displayed when the vehicle 1 is traveling on land. In this case, as the standard display mode, for example, the layout adjustment unit 42 arranges the front display area FV at the upper center of the display area of the display device 8, and arranges the left display area SVL and the right display area SVR on the left and right. Further, an orientation display area PV for displaying the inclinometer 50 is arranged below the front display area FV. When traveling on land, the forward display area FV includes a forward reference line Qa indicating a route index R indicating the estimated travel direction of the vehicle 1 and a distance from the front end 2c of the vehicle body 2 as necessary. The side reference line Pa or the like indicating the distance from the side ends 2d, 2f of the vehicle body 2 may be displayed to provide driving support. In the left display area SVL and the right display area SVR, the vehicle body 2 and the road surface are displayed as images in which distortion is corrected and distortion is reduced. In the standard display mode that is mainly displayed during land driving, the layout adjustment unit 42 increases the display ratio of the vehicle body 2 and the road surface on the road surface in the left display area SVL and the right display area SVR. That is, in the standard display mode, images that make it easy to grasp the road surface condition around the front wheel 3F are displayed in the left display area SVL and the right display area SVR. The inclinometer 50 displays the inclination of the vehicle 1 in the left-right direction (roll angle) and the inclination in the front-rear direction (pitch angle) in the posture of the symbol 52 based on the signal from the acceleration sensor 26 (26a, 26b).

Also, the layout adjustment unit 42 can change the display contents (layout change) so that the relationship between the water level limit line L or the water level reference line K and the water surface W can be more easily understood during the traveling. FIG. 10 is an example of an image in the special display mode that is displayed when traveling on a road. In the case of FIG. 10, the layout of the front display area FV, the left display area SVL, the right display area SVR, and the posture display area PV is the same as that during land travel in FIG. 9, but the left display area SVL and the right display area. The internal layout of the SVR is different from that in FIG. As shown in FIG. 10, in the special display mode in which the water level limit line L (water level reference line K) is displayed, the display area of the surface of the vehicle body 2 in the image displayed in the left display area SVL and the right display area SVR. Is made larger than when the water level limit line L (water level reference line K) is not displayed. In the case of the special display mode, how much the water surface W has risen with respect to the water level limit line L (water level reference line K) is mainly displayed. Therefore, in the special display mode, the display area of the body side surface 2m in the left display area SVL and the right display area SVR is increased. Thus, the visibility of the relationship between the water level limit line L (water level reference line K) and the water surface W can be improved by changing the layout in the image. Moreover, it can be made easy to recognize that the screen of the display device 8 is switched to the display mode of the water level limit line L. Note that the captured image data captured by the imaging unit 15 has an area larger than the area displayed in the left display area SVL and the right display area SVR as an imaging range, for example, by performing a cutting process from the left display area SVL. Or in the right display area SVR. Therefore. In order to enlarge the display area on the surface of the vehicle body 2, for example, it can be performed by adjusting the cutout range from the original image. In another embodiment, the water level limit line L (water level reference line K) may be displayed while the display area on the surface of the vehicle body 2 remains as shown in FIG.

Further, the layout adjustment unit 42 uses the entire display area of the display device 8 by changing the layout, and shows an image showing the relationship between the water level limit line L (water level reference line K) and the water surface W as shown in FIGS. May be displayed. By performing such display, the visibility of the relationship between the water level limit line L (water level reference line K) and the water surface W is further improved, and the water level limit line L (water level reference line K) and the water surface W are It can be emphasized to pay particular attention to the relationship.

The image processing unit 34 performs processing for superimposing the water level limit line L and the water level reference line K based on the water level limit information when the vehicle 1 travels on the corresponding position on the surface of the vehicle body 2 on the image displayed on the display device 8. . The water level limit information is, for example, information for superimposing the water level limit line L and the water level reference line K on the image, and may be information for designating an image on the image, or may be information from a reference line, for example, the ground contact position of the wheel 3. It may be height information. When the conditions for displaying the water level limit line L are aligned, the CPU 14a superimposes the water level limit line L at a height position determined in the vehicle height direction at the vehicle 1 design stage. As described above, each imaging unit 15 is fixed to the vehicle body 2 and the imaging range is determined. Accordingly, where the vehicle body 2 is displayed in the display range in the captured image data or the image based on the captured image data can be calculated, the water level limit line L is superimposed on the position. Similarly, the water level reference line superimposing unit 46 superimposes the water level reference line K at a height position determined in the vehicle height direction at the design stage of the vehicle 1. The water level limit line superimposing unit 44 and the water level reference line superimposing unit 46 are displayed differently so that the water level limit line L and the water level reference line K (first water level reference line K1, second water level reference line K2) can be easily identified. You may superimpose in the aspect. For example, the line type and line color can be changed. In addition, when selecting a line color, it is desirable to use a color that can be identified with respect to the color of the vehicle body 2. For example, the water level limit line superimposing unit 44 and the water level reference line superimposing unit 46 may acquire color information of the vehicle body 2 and automatically select an identifiable line color, or may be selectable by the user. Good. In addition, when the water level limit line L is superimposed and displayed, the water level limit line superimposing unit 44, as shown in FIGS. 6 and 7, shows information indicating the water level limit line L, for example, a “mark” such as an arrow, “Character information” such as Limit may be added. In addition, when “mark”, “character information” or the like is added, it is superimposed on the upper side of the water level limit line L so that the “mark” or “character information” can be easily seen even when the water surface W rises. Is desirable. Similarly, when the water level reference line K is superimposed by the water level reference line superimposing unit 46, “mark” or “character information” may be added. In FIGS. 6 and 7, the water level reference line K is shown as being superimposed in parallel with the water level limit line L. However, the water level reference line K is superimposed in a direction orthogonal to the water level limit line L, and the water depth is increased. You may display with the scale shown.

The CPU 14a performs manual operation by the user, for example, an operation input unit, for displaying whether or not the water level limit line L (water level reference line K) is displayed, for example, switching from the display state of FIG. 9 to the display state of FIG. 10 or an input operation by the operation unit 14g. Further, as described above, the switching may be performed according to the operation state (error signal output state) of the distance measurement unit 16 and the distance measurement unit 17. In addition, the switching operation may be performed using an input unit such as voice recognition or gesture recognition.

A control example of the periphery monitoring system 100 configured as described above will be described with reference to the flowchart of FIG. Note that the flow shown in FIG. 11 is repeatedly executed at a predetermined processing cycle.

First, the CPU 14a determines whether the periphery is currently being monitored (S100). If the periphery is not being monitored (No in S100), for example, when the vehicle 1 is turned off or the display device 8 is used for other purposes (navigation, If it is used for audio, etc., this flow is once terminated. On the other hand, when determining that the periphery is currently being monitored (Yes in S100), the CPU 14a determines whether the travel travel mode switching condition is satisfied (S102). As described above, the CPU 14a considers that the traveling travel mode switching condition is satisfied when an input operation is performed by the operation input unit 10 or the operation unit 14g. Further, the CPU 14a, when the distance measurement unit 16 or the distance measurement unit 17 outputs an error signal in a predetermined pattern, for example, when the distance measurement units 17e to 17h output error signals almost simultaneously (for example, within 2 seconds). It is considered that the conditions for switching to the travel mode have been met.

In S102, the CPU 14a acquires captured image data currently captured by the imaging unit 15, for example, a side image of the vehicle 1, via the image acquisition unit 30 when the travel travel mode switching condition is satisfied (Yes in S102). (S104). Subsequently, the CPU 14a performs image conversion processing on the acquired side image (S106). In the image conversion process, at least one of the distortion correction process by the distortion correction unit 38 and the viewpoint conversion process by the viewpoint conversion unit 40 is executed based on the display image mode setting by the user. Note that when the water level limit line L or the like is superimposed and displayed on the original image shown in FIG. 5 by the selection or setting by the user, the process of S106 may be skipped.

Subsequently, the CPU 14 a superimposes the “water level limit line L” on the body side surface 2 m of the vehicle body 2 by the water level limit line superimposing unit 44, and the “water level reference line K” by the water level reference line superimposing unit 46. 2m is superimposed (S108). In this case, the “water level reference line K” can be hidden by selection or setting by the user. By hiding the “water level reference line K”, the image on which the water level limit line L is superimposed can be simplified. That is, a display corresponding to a user who prefers a simple display can be performed.

Subsequently, when the layout adjustment unit 42 switches from the standard display mode screen of FIG. 9 displayed as the peripheral monitoring screen to the interference travel mode screen, whether or not the interference travel mode screen is designated as the main screen. (S110). For example, when “main display” is selected by an input operation using the operation input unit 10 or the operation unit 14g (Yes in S110), the layout adjustment unit 42 displays the screen in the standard display mode in FIG. Switch to a screen that emphasizes the relationship between the water level limit line L (water level reference line K) and the water surface W as shown in FIG. (S112).

On the other hand, when the main display selection is not performed (No in S110), the layout adjustment unit 42, as shown in FIG. 10, displays the surface of the vehicle body 2 in the images displayed in the left display area SVL and the right display area SVR. A process of displaying the display area on the display device 8 by switching to the special display mode in which the water level limit line L (water level reference line K) is not displayed is executed (S114).

Whether the CPU 14a satisfies the mode return condition for returning from the crossing travel mode to the land travel mode while the display in the crossing travel mode indicating the relationship between the water level limit line L (water level reference line K) and the water surface W is executed. It is confirmed whether or not (S116). In the case of manual return, the CPU 14a considers that the mode return condition is satisfied when a screen return request (screen display request for land travel mode) is received from the user through an input operation via the operation input unit 10 or the operation unit 14g. . In the case of automatic return, the CPU 14a is in the case where all of the distance measurement unit 16 and the distance measurement unit 17 have been returned (when all error signals have been canceled due to landing), and after a predetermined period, for example, 5 seconds have elapsed. It is considered that the mode return condition is satisfied. When the mode return condition is satisfied (Yes in S116), the layout adjustment unit 42 returns the image displayed on the display device 8 to the image in the standard display mode (screen in FIG. 9) (S118), and a series of crossing travels. The display processing of the water level limit line L (water level reference line K) according to the mode ends.

In S116, when the mode return condition is not satisfied (No in S116), the CPU 14a proceeds to S104 and sequentially updates the display images during the traveling by executing the processes after S104. The transition of the relationship between the change in the water surface W (change in the water level) and the water level limit line L (water level reference line K) is displayed via the display device 8 in real time. In S102, when the conditions for switching to the traveling mode are not satisfied (No in S102), that is, when display of the water level limit line L (water level reference line K) is unnecessary, the process proceeds to S118 and the standard display mode is changed. The process of displaying the screen on the display device 8 is continued.

As described above, according to the periphery monitoring system 100 of the present embodiment, the display device 8 displays the actual image based on the captured image data captured by the imaging unit 15, and the actual surface of the vehicle body 2 and the interference of the vehicle 1. The periphery at the time of traveling, that is, the water surface W is displayed. Further, the water level limit line L is superimposed on the corresponding position of the surface of the vehicle body 2 on the image. As a result, the actual situation, that is, where the water surface W is located on the vehicle body 2 and the state of the water surface, etc. are more accurately reported, and the information is intuitively presented to the user. Can be easily grasped.

In the case of the above-described embodiment, an example in which the water level limit line L and the water level reference line K are superimposed on the body side surface 2m based on the captured image data captured by the imaging unit 15b and the imaging unit 15d provided in the door mirror 2g is shown. . In another embodiment, captured image data captured by another imaging unit 15 may be used. For example, when the front part (front side of the body) of the vehicle body 2 is included in the imaging range of the imaging unit 15c provided at the front part of the vehicle body 2, the water level limit line L and the water level reference line K are superimposed and displayed on the image of the front surface of the body. You may display on the front display area FV of the apparatus 8. FIG. In this case, when the vehicle 1 is moved forward, it becomes easy to grasp the change in the water level with respect to the vehicle body 2. Similarly, when the rear part (body rear surface) of the vehicle body 2 is included in the imaging range of the imaging unit 15a provided at the rear part of the vehicle body 2, the water level limit line L and the water level reference line K are superimposed on the image on the rear surface of the body. 8 may be displayed. In this case, the front and rear images may be switched based on the shift position of the speed change operation unit 7 (shift lever). Alternatively, the display area of the display device 8 may be divided into four, and the water level limit line L (water level reference line K) may be superimposed and displayed on each of the front, rear, left and right images. In this case, since the relationship between the water surface W and the water level limit line L (water level reference line K) can be shown with respect to the entire circumference of the vehicle 1, for example, when the bottom of the water 1 is large and the vehicle 1 is greatly inclined. However, it is possible to make the user easily understand the relationship between the water level limit line L (water level reference line K) and the water surface W.

Moreover, in this embodiment, the example which superimposes the water level limit line L and the water level reference line K along the body side surface 2m was shown. That is, when the body side surface 2m is a curved surface, the water level limit line L and the water level reference line K are also superimposed so as to correspond to the curved surface, but this is not restrictive. For example, when distortion correction is performed on captured image data or an image based on the captured image data so as to approximate the actual shape of the vehicle 1, the water level limit line L and the water level reference line K are superimposed in the front-rear direction of the vehicle body 2. Since the shape is a linear shape, the water level limit line L and the water level reference line K may be simply expressed as a straight line. In this case, the image processing load by the ECU 14 can be reduced. In addition, since the water level limit line L and the water level reference line K are displayed as simple straight lines, the relationship with the water surface can be easily understood intuitively.

In the above-described embodiment, a so-called off-road vehicle has been described. However, the periphery monitoring device of the present embodiment can also be applied to a so-called on-road vehicle (passenger car or the like), and the same effect can be obtained. Further, by processing the image on which the water level reference line K and the water level limit line L are superimposed, the position of the water surface W with respect to the water level reference line K and the water level limit line L is detected, and a warning message by voice or the like is output. It may be.

Further, in the above-described embodiment, an example in which signals from the distance measurement unit 16 and the distance measurement unit 17 are used when automatically switching to the travel mode (underwater travel mode) is shown, but the present invention is not limited to this. You may switch using the signal from a sensor. Further, the water surface W may be detected and switched based on the image captured by the imaging unit 15. The mode switching may be manual only.

Although embodiments and modifications of the present invention have been described, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 ... Vehicle, 2 ... Vehicle body, 2m ... Body side surface, 8 ... Display apparatus, 14 ... ECU, 14a ... CPU, 14b ... ROM, 14d ... Display control part, 14g ... Operation part, 15, 15a, 15b, 15c, 15d ... Imaging unit, 30 ... Image acquisition unit, 32 ... Display processing unit, 34 ... Image processing unit, 36 ... Output unit, 38 ... Distortion correction unit, 40 ... Viewpoint conversion unit, 42 ... Layout adjustment unit, 44 ... Water level limit line Superimposition unit, 46 ... water level reference line superimposition unit, L ... water level limit line, K ... water level reference line, W ... water surface, 100 ... perimeter monitoring system (perimeter monitoring device).

Claims

A control unit that causes a display device to display an image based on captured image data output from an imaging unit that captures an image of the vehicle body surface and the periphery of the vehicle body surface as an imaging range;
An image processing unit that superimposes a water level limit line based on the water level limit information at the time of traveling of the vehicle on a corresponding position of the vehicle body surface on the image;
A peripheral monitoring device comprising:
The periphery monitoring device according to claim 1, wherein the control unit displays an image obtained by correcting the captured image data obtained by capturing the side surface of the vehicle.
The periphery monitoring device according to claim 2, wherein the control unit performs a viewpoint conversion process on the captured image data as the correction.
4. The periphery monitoring apparatus according to claim 2, wherein the control unit performs a distortion correction process on the captured image data as the correction.
The periphery monitoring device according to any one of claims 2 to 4, wherein the control unit performs a cutting process for cutting out a part of the captured image data as the correction.
The periphery monitoring device according to any one of claims 1 to 5, wherein the image processing unit superimposes the water level limit line having a shape along a surface shape of the vehicle on the corresponding position.
The periphery monitoring device according to any one of claims 1 to 5, wherein the image processing unit superimposes the straight water level limit line on the corresponding position.
The said control part makes the display area of the said vehicle body surface in the said image larger than the case where the said water level limit line is non-displayed when the said water level limit line is displayed in any one of Claims 1-7. The peripheral monitoring device described.
The periphery monitoring device according to any one of claims 1 to 8, wherein the image processing unit superimposes a water level reference line at a position lower in a vehicle height direction than the water level limit line superimposed on the image.
The periphery monitoring device according to claim 9, wherein the image processing unit superimposes the water level reference line in a display mode different from the water level limit line.