JP6976298B2 - Vehicle surroundings monitoring device and vehicle surroundings monitoring method - Google Patents

Description

The present application relates to a vehicle surroundings monitoring device and a vehicle surroundings monitoring method.

In recent years, a surveillance camera device has been disclosed in which a surveillance camera is mounted on a vehicle to provide an image of the surroundings of the vehicle, which tends to be a blind spot for a driver. Recently, a peripheral surveillance camera device has also been disclosed in which a plurality of cameras that capture the surroundings of a vehicle are mounted on the vehicle to provide a bird's-eye view of the surroundings 360 ° from above the vehicle. This peripheral surveillance camera device generates a bird's-eye view image as if a 360 ° peripheral view is seen from above the vehicle by converting a plurality of images captured by a plurality of cameras by coordinate conversion and synthesizing them (for example, Patent Document 1). reference).

In addition, the vehicle is equipped with a camera that captures the rear side, recognizes the driving lane on the rear side of the own vehicle from the captured image, and detects the following vehicle traveling in the monitoring area of the driving lane. Surveillance camera devices are disclosed. The rear side surveillance camera device has a function of notifying an alarm when the driver operates the winker in the direction of the detected following vehicle (see, for example, Patent Document 2).

Japanese Patent No. 3286306 Japanese Unexamined Patent Publication No. 2-287799

The peripheral surveillance camera device is mainly used when parking, and the rear side surveillance camera device is mainly used when driving. When these two devices are mounted on one vehicle, it becomes a vehicle surrounding monitoring device that improves the safety of the vehicle both when parked and when traveling. However, when these two devices are simply mounted on one vehicle, there is a problem that the peripheral monitoring device for the vehicle becomes large because each device space is required.

This application has been made to solve the above-mentioned problems. Focusing on the fact that the vehicle conditions are different when a plurality of surveillance camera devices are used, the common parts of those devices are shared into one device. The purpose of this integration is to reduce the size of the vehicle surroundings monitoring device.

The vehicle surrounding monitoring device according to the present application includes a camera unit composed of a plurality of cameras, a vehicle condition determination unit that determines the position of the own vehicle based on the latitude and longitude information and map information of the own vehicle, and this vehicle condition determination. Based on the position of the vehicle determined by the unit, one or more monitoring functions are selected as the selection monitoring function, the input image from the camera unit required for the selection monitoring function is determined, and the selection monitoring function is supported. The monitoring function selection unit that determines the video processing method to be performed, the input video acquisition unit that switches and acquires the input video from the camera unit determined by this monitoring function selection unit, and the input video acquired by this input video acquisition unit. It is equipped with an input video processing unit that performs video processing by the video processing method determined by the monitoring function selection unit, the vehicle status determination unit determines the current location of the own vehicle, and the vehicle status determination unit determines the current location of the own vehicle. When it is determined that the vehicle is a parking lot, the monitoring function selection unit selects the peripheral monitoring function as the selection monitoring function, and when the vehicle status determination unit determines that the current location of the own vehicle is the roadway, the monitoring function selection unit determines. Select the rear side monitoring function as the selection monitoring function.

In the vehicle surroundings monitoring device of the present application, one or more monitoring functions are selected as the selective monitoring function based on the position of the own vehicle determined by the vehicle condition determination unit based on the longitude / latitude information and the map information of the own vehicle. , Input from the camera unit determined by the monitoring function selection unit that determines the video processing method corresponding to the selection monitoring function while determining the input video from the camera unit required for the selection monitoring function, and the input from the camera unit determined by this monitoring function selection unit. It is equipped with an input video acquisition unit that switches and acquires video, and an input video processing unit that processes the input video acquired by this input video acquisition unit using the video processing method determined by the monitoring function selection unit. When the vehicle condition determination unit determines the current location of the own vehicle and the vehicle condition determination unit determines that the current location of the own vehicle is a parking lot, the monitoring function selection unit selects the peripheral monitoring function as the selection monitoring function and the vehicle. When the situation judgment unit determines that the current location of the own vehicle is the roadway, the monitoring function selection unit selects the rear side monitoring function as the selection monitoring function, so that the increase in size of the vehicle surroundings monitoring device is suppressed. At the same time, the rear side monitoring function can be automatically activated when the vehicle is about to start traveling from a state of being stopped on the road.

It is a block diagram of the surroundings monitoring device for a vehicle which concerns on Embodiment 1. FIG. It is a schematic diagram which shows the imaging range of the surveillance camera in Embodiment 1. FIG. It is a figure which shows an example of the peripheral monitoring image in Embodiment 1. FIG. It is a figure which shows an example of the rear side monitoring image in Embodiment 1. FIG. It is a flowchart of the surroundings monitoring method for a vehicle in Embodiment 1. It is a flowchart of the surroundings monitoring method for a vehicle in Embodiment 1. It is a schematic diagram which shows an example of the hardware of the vehicle surroundings monitoring apparatus in Embodiment 1. FIG.

Hereinafter, the vehicle surroundings monitoring device according to the embodiment for carrying out the present application will be described in detail with reference to the drawings. In each figure, the same reference numerals indicate the same or corresponding parts.

Embodiment 1.
FIG. 1 is a configuration diagram of a vehicle surroundings monitoring device according to the first embodiment. The vehicle surroundings monitoring device of the present embodiment is mounted on the vehicle and has a peripheral monitoring function and a rear side monitoring function. The peripheral monitoring function is a function to display a bird's-eye view image for the driver that looks over the surrounding 360 ° from the top of the vehicle, detects the parking border from the image around the vehicle, and automatically follows the detected parking border. It has an automatic parking function for parking, a collision avoidance function by detecting pedestrians, vehicles, obstacles, etc. around and behind the vehicle when the vehicle is reversing, and warning and brake control. In addition, the rear side monitoring function is a following vehicle detection function that detects the following vehicle traveling behind the own vehicle, and a lane change that notifies an alarm when the driver operates the winker in the direction of the detected following vehicle. It has an alarm function and so on.

The vehicle peripheral monitoring device 1 of the present embodiment shown in FIG. 1 includes a peripheral monitoring camera 2 that captures a peripheral monitoring image of the own vehicle, and a rear side monitoring camera 3 that captures a rear side monitoring image of the own vehicle. , One or more peripheral monitoring functions and one or more rear sides based on the vehicle condition determination unit 4 for determining the vehicle condition of the own vehicle and the vehicle condition of the own vehicle determined by the vehicle condition determination unit 4. One or more of the direction monitoring functions are selected as the selection monitoring function, and either the peripheral monitoring image or the rear side monitoring image is determined as the input image based on the selection monitoring function, and the selection monitoring function is supported. The monitoring function selection unit 5 that determines the video processing method to be performed, and the input video acquisition unit 6 that acquires the input video determined by the monitoring function selection unit 5 from either the peripheral surveillance camera 2 or the rear side surveillance camera 3. And an input video processing unit 7 that performs video processing on the input video acquired by the input video acquisition unit 6 by the video processing method determined by the monitoring function selection unit 5.

The peripheral surveillance camera 2 is composed of one or more optical cameras such as a front camera, a rear camera, a left side camera, and a right side camera whose imaging region is the periphery of the vehicle of the own vehicle. In the present embodiment, the peripheral surveillance camera 2 is composed of four cameras, but any number of cameras may be used as long as it is a camera for photographing the periphery of the own vehicle. The peripheral surveillance camera 2 outputs the images captured by these cameras as peripheral surveillance images.

The rear side surveillance camera 3 is composed of one or more optical cameras such as a rear right side camera and a rear left side camera whose imaging region is the rear side of the own vehicle. In the present embodiment, the rear side surveillance camera 3 is composed of two cameras, but any number of cameras may be used as long as it is a camera for photographing the rear side of the own vehicle. .. The posterior surveillance camera 3 outputs the images captured by these cameras as posterior surveillance images.

FIG. 2 is a schematic diagram showing an example of the imaging range of the peripheral surveillance camera 2 and the imaging range of the rear side surveillance camera 3 in the present embodiment. The peripheral surveillance cameras 2 include a front camera 21 attached to the front grill of the vehicle 18, a right side camera 22 and a left side camera 23 attached under the left and right front doors, respectively, and a rear camera 24 attached to the rear bumper. It is composed of and. The rear side surveillance camera 3 includes a rear right side camera 31 attached to the right end of the rear bumper and a rear left side camera 32 attached to the left end of the rear bumper. The image pickup range 19 of the peripheral surveillance camera 2 is a range in which the distance from the vehicle 18 is about 10 m and the circumference is about 360 °. The image pickup range 20 of the rear side monitoring camera 3 is a range of the rear side on the left side and the rear side on the right side of the vehicle 18 at a distance of about 100 m from the vehicle 18. In the vehicle shown in FIG. 2, each of the cameras constituting the peripheral surveillance camera 2 and the rear side surveillance camera 3 is attached to the outside of the vehicle, but may be attached to the inside of the vehicle. For example, the front camera 21 may be attached to the upper part inside the windshield.

The vehicle condition determination unit 4 receives a vehicle speed signal, a steering wheel steering angle signal, a winker signal, a gear signal, and the like of the own vehicle from the vehicle condition notification unit 11. The vehicle status notification unit 11 is, for example, an electronic control unit. Hereinafter, the vehicle speed signal, steering wheel angle signal, winker signal, gear signal, etc. are collectively referred to as CAN information (Control Area Area Network information: CAN is a registered trademark). Further, the vehicle condition determination unit 4 receives latitude and longitude information, map information, and the like from the vehicle position information notification unit 12. The vehicle position information notification unit 12 is, for example, a car navigation system or the like, and can acquire latitude and longitude information from GPS (Global Positioning System) and map information from a map database. The vehicle condition determination unit 4 determines the vehicle condition of the own vehicle based on the CAN information received from the vehicle condition notification unit 11, the latitude and longitude information received from the vehicle position information notification unit 12, the map information, and the like. Then, the vehicle condition determination unit 4 notifies the monitoring function selection unit 5 of the vehicle condition of the determined own vehicle. Three examples of specific determination of vehicle conditions are shown. As a first example, the vehicle condition determination unit 4 acquires whether or not the gear is set to the reverse gear from the CAN information, and if it is set to the reverse gear, determines that the own vehicle is retracting. .. Then, when the gear is set to a gear other than the reverse gear, the vehicle condition determination unit 4 determines that the own vehicle is moving forward or stopped. As a second example, the vehicle condition determination unit 4 determines whether or not the vehicle is traveling in the parking lot from the current location of the vehicle obtained from the latitude and longitude information and the map information of the vehicle. As a third example, the vehicle condition determination unit 4 determines whether or not the vehicle is traveling on the road from the current location of the vehicle. The vehicle status determined by the vehicle status determination unit 4 may be any other vehicle status as long as it is useful for selecting at least one of the peripheral monitoring function and the rear side monitoring function as the selective monitoring function. The vehicle status determination unit 4 may directly acquire the current location of the own vehicle from the vehicle position information notification unit 12.

The monitoring function selection unit 5 has a plurality of peripheral monitoring functions and a plurality of rear side monitoring functions based on the driver instruction information notified from the driver instruction information notification unit 13 and the vehicle status notified from the vehicle condition determination unit 4. Select one or more of them as the selection monitoring function. Then, the monitoring function selection unit 5 determines either the peripheral monitoring image or the rear side monitoring image as an input image based on the selection monitoring function. Further, when the peripheral monitoring function is selected as the selective monitoring function, the monitoring function selection unit 5 determines the corresponding peripheral monitoring video processing method as the video processing method, and selects the rear side monitoring function as the selective monitoring function. Determines the corresponding rear side surveillance video processing method as the video processing method. Further, the monitoring function selection unit 5 notifies the input video acquisition unit 6 of the determined input video, and also notifies the input video processing unit 7 of the determined video processing method. The driver instruction information notification unit 13 is, for example, an execution function setting button installed on an instrument panel, a center cluster, a steering wheel, or a button on the screen of a car navigation system. The driver instruction information notification unit 13 notifies the monitoring function selection unit 5 of the driver instruction information when the driver presses these buttons.

Next, the operations of the vehicle condition determination unit 4 and the monitoring function selection unit 5 will be described. When the driver instruction information is notified from the driver instruction information notification unit 13, the monitoring function selection unit 5 selects one or more of the peripheral monitoring function and the rear side monitoring function as the selection monitoring function according to the driver instruction information. Then, the input video and the video processing method corresponding to the input video are determined. When the driver instruction information is not notified from the driver instruction information notification unit 13, the monitoring function selection unit 5 of the peripheral monitoring function and the rear side monitoring function based on the vehicle condition notified from the vehicle condition determination unit 4. Select one or more of them as the selection monitoring function, and determine the input video and video processing method corresponding to it. For example, the vehicle condition determination unit 4 determines whether the own vehicle is moving backward, moving forward, or stopped based on the CAN information. When the vehicle condition determination unit 4 determines that the vehicle is moving backward, the monitoring function selection unit 5 selects one or more of the peripheral monitoring functions as the selection monitoring function, and the corresponding input video and video. Determine the processing method. When the vehicle condition determination unit 4 determines that the vehicle is moving forward or stopped, the vehicle condition determination unit 4 determines that the vehicle is based on the current location of the vehicle obtained from the vehicle position information notification unit 12. Determine if the location is a parking lot, a roadway, or neither. When the vehicle condition determination unit 4 determines that the position of the own vehicle is a parking lot, the monitoring function selection unit 5 selects one or more of the peripheral monitoring functions as the selection monitoring function, and the corresponding input video and video. Determine the processing method. When the vehicle condition determination unit 4 determines that the position of the own vehicle is the roadway, the monitoring function selection unit 5 selects one or more of the rear side monitoring functions as the selection monitoring function, and the corresponding input video and Determine the video processing method. When the vehicle condition determination unit 4 determines that the position of the own vehicle is neither the parking lot nor the roadway, the monitoring function selection unit 5 selects one or more of the rear side monitoring functions as the selection monitoring function. , Determine the corresponding input video and video processing method. Then, the monitoring function selection unit 5 outputs video processing information that specifies the determined video processing method.

The input video acquisition unit 6 acquires the peripheral surveillance video or the rear side surveillance video determined as the input video by the monitoring function selection unit 5 from either the peripheral surveillance camera 2 or the rear side surveillance camera 3. In the input video acquisition unit 6, the method of switching the input video between the peripheral surveillance video and the rear side surveillance video may be a method of switching by hardware or a method of switching by software. The input video acquisition unit 6 sends the acquired input video to the input video processing unit 7.

The input video processing unit 7 performs video processing on the input video sent from the input video acquisition unit 6 by using the video processing method determined by the monitoring function selection unit 5. When the peripheral monitoring video is sent as the input video, the input video processing unit 7 performs video processing using the peripheral monitoring video processing method determined by the monitoring function selection unit 5 as the video processing method. When the rear side surveillance video is sent as the input video, the input video processing unit 7 performs video processing using the rear side surveillance video processing method determined by the monitoring function selection unit 5 as the video processing method. For example, when a peripheral monitoring video is sent as an input video, the input video processing unit 7 generates a bird's-eye view image that looks over the surrounding 360 ° from the top of the vehicle by converting the coordinates of this peripheral monitoring video and synthesizing it. do. Then, the input image processing unit 7 detects the parking frame line for automatic parking by using machine learning such as deep learning or edge detection for the peripheral monitoring image. Alternatively, the input image processing unit 7 uses machine learning or optical flow using HoG (Histograms of Oriented Radiants) features, deep learning, etc. for the peripheral monitoring image to pedestrians, vehicles, and vehicles around or behind the vehicle. Detects obstacles, etc. As described above, there are a plurality of peripheral monitoring functions such as a bird's-eye view image display function for drivers, an automatic parking function that automatically parks along the parking frame line, and a collision avoidance function by warning and brake control. In addition, there are a plurality of peripheral surveillance video processing methods corresponding to each of these plurality of functions. The monitoring function selection unit 5 selects one or a plurality of peripheral monitoring functions as the selection monitoring function, and determines a video processing method corresponding to the selection monitoring function.

FIG. 3 is an example of a peripheral monitoring image in the present embodiment. The peripheral monitoring image shown in FIG. 3 is a bird's-eye view image as if the peripheral 360 ° was seen from above the vehicle, and is a peripheral monitoring image when the automatic parking function is operating. This peripheral monitoring image shows a state in which the input video processing unit 7 has detected the parking frame line 25 and another vehicle 26 within the imaging range 19 of the peripheral monitoring camera.

When the rear side surveillance video is sent as the input video, the input video processing unit 7 uses machine learning or optical flow using HoG features, deep learning, etc. for the rear side surveillance video to rearward. It detects vehicles on the side and two-wheeled vehicles. There are also multiple rear side monitoring functions, such as a lane change warning function that detects the following vehicle and notifies the driver when the driver operates the winker in the direction of the detected following vehicle, and a function that detects and warns of a two-wheeled vehicle passing through. .. In addition, there are a plurality of rear side surveillance video processing methods corresponding to each of these multiple functions. The monitoring function selection unit 5 selects one or a plurality of rear side monitoring functions as the selection monitoring function, and determines a video processing method corresponding to the selection monitoring function.

FIG. 4 is an example of a rear side monitoring image in the present embodiment. The rear side monitoring image shown in FIG. 4 is a rear side monitoring image when the following vehicle detection function is operating. This posterior surveillance image shows a state in which the input video processing unit 7 detects the white line 27 in the imaging range 20 of the posterior surveillance camera and recognizes the adjacent traveling lane on the posterior side. Further, the input video processing unit 7 detects the following vehicle 28 traveling in the traveling lane.

The video processing method performed by the input video processing unit 7 may be a video processing method other than the above-mentioned method. The input video processing unit 7 outputs the result of performing video processing on the input video as video processing result information. The video processing result information is information obtained as a result of video processing, and is, for example, information on whether or not the own vehicle may collide with a pedestrian, a vehicle, a two-wheeled vehicle, an obstacle, or the like. Further, in the vehicle surroundings monitoring device of the present embodiment, an example in which there are a plurality of peripheral monitoring functions and a plurality of rear side monitoring functions are shown, but the peripheral monitoring function and the rear side monitoring function may be one or more each. Just do it.

The vehicle surroundings monitoring device 1 of the present embodiment may include at least one of a video output determination unit 8, an alarm output determination unit 9, and a vehicle control determination unit 10. The video output determination unit 8, the alarm output determination unit 9, and the vehicle control determination unit 10 are notified of the video processing information output from the monitoring function selection unit 5 and the video processing result information output from the input video processing unit 7. To. Further, in addition to the video processing result information, the video output determination unit 8 also sends the display video for the driver, which has been video-processed for the input video by the input video processing unit 7. Further, the warning output determination unit 9 and the vehicle control determination unit 10 are also notified of CAN information from the vehicle status notification unit 11.

The video output determination unit 8 determines the necessity of video display based on the video processing information notified from the monitoring function selection unit 5 and the video processing result information notified from the input video processing unit 7. Then, when the video output determination unit 8 determines that the video display is necessary, the video output unit 14 displays the display video for the driver sent from the input video processing unit 7. For example, when the video processing information is a peripheral monitoring video processing method, the video output determination unit 8 displays a bird's-eye view video generated by the input video processing unit 7 overlooking 360 ° around the vehicle on the video output unit 14. On the other hand, when the video processing information is the rear side monitoring video processing method, the video output determination unit 8 determines that it is not necessary to display the video on the video output unit 14.

The alarm output determination unit 9 is based on the CAN information notified from the vehicle status notification unit 11, the video processing information notified from the monitoring function selection unit 5, and the video processing result information notified from the input video processing unit 7. To determine the necessity of alarm output. Then, when the alarm output determination unit 9 determines that the alarm output is necessary, it determines the alarm output method and notifies the alarm output unit 15. For example, when the video processing information is the peripheral monitoring video processing method, the alarm output determination unit 9 determines that there is an obstacle around or behind the vehicle and may come into contact with the own vehicle based on the video processing result information. do. At this time, the alarm output determination unit 9 notifies the alarm output unit 15 of the output of an alarm sound by a buzzer or the like, or the output of an alarm vibration that vibrates the seat or the steering wheel.

On the other hand, when the video processing information is the rear side monitoring video processing method, it is assumed that the alarm output determination unit 9 determines that a nearby vehicle exists on the rear side based on the video processing result information. At this time, the alarm output determination unit 9 notifies the alarm output unit 15 so as to light an indicator such as an LED installed near the left and right side mirrors, for example. By doing so, the alarm output unit 15 can transmit the presence of an obstacle on the rear side to the driver by visual information and alert the driver when changing lanes. Further, when the driver operates the steering wheel or the winker in order to change the lane to the adjacent traveling lane side where the nearby vehicle exists in the state where the attention is given by the visual information, the warning output determination unit 9 sets the alarm output determination unit 9. , The output of the alarm sound by a buzzer or the like, and the output of the alarm vibration that vibrates the seat or the handle are notified to the alarm output unit 15. By doing so, the alarm output determination unit 9 can call attention by auditory information and tactile information in addition to visual information. The alarm method may be another method.

The vehicle control determination unit 10 is based on the CAN information notified from the vehicle status notification unit 11, the video processing information notified from the monitoring function selection unit 5, and the video processing result information notified from the input video processing unit 7. To determine the necessity of vehicle control. Then, when the vehicle control determination unit 10 determines that vehicle control is necessary, the vehicle control method and the vehicle control amount are determined and notified to the vehicle control unit 16. For example, when the video processing information is the peripheral monitoring video processing method, it is assumed that the vehicle control determination unit 10 determines from the video processing result information that an obstacle exists around or behind the vehicle and may come into contact with the own vehicle. At this time, the vehicle control determination unit 10 determines the brake control amount so as to avoid contact and notifies the vehicle control unit 16. Alternatively, it is assumed that the vehicle control determination unit 10 determines from the video processing result information that the vehicle is automatically parked along the parking frame line. At this time, the vehicle control determination unit 10 determines the drive amount and the steering wheel steering amount and notifies the vehicle control unit 16. On the other hand, when the video processing information is the rear side monitoring video processing method, the vehicle control determination unit 10 determines from the video processing result information that there is a nearby vehicle on the rear side and there is a possibility of contact if the course is changed. And. At this time, the vehicle control determination unit 10 determines the steering wheel steering amount and the brake control amount so as to avoid contact, and notifies the vehicle control unit 16.

The video output unit 14 is, for example, a monitor of a car navigation system, an instrument panel, a rear-view mirror monitor, or the like, and displays the input video when the video-processed input video is sent from the video output determination unit 8. The alarm output unit 15 outputs a necessary alarm using a buzzer, LED, seat, steering wheel, etc., based on the alarm output information notified from the alarm output determination unit 9. The vehicle control unit 16 is, for example, an electronic control unit or the like, and performs drive control, steering wheel control, brake control, and the like based on the vehicle control method and the vehicle control amount notified from the vehicle control determination unit 10.

As shown in FIG. 1, the vehicle status determination unit 4, the monitoring function selection unit 5, the input video processing unit 7, the video output determination unit 8, the alarm output determination unit 9, and the vehicle control determination unit 10 are one CPU ( It is preferable that it is integrated in Central Processing Unit) 17. By integrating these components in one CPU 17, the data transmission time between the components is shortened.

Next, a vehicle surroundings monitoring method in the vehicle surroundings monitoring device of the present embodiment will be described. 5 and 6 are flowcharts of the vehicle surroundings monitoring method according to the present embodiment. The flowchart of FIG. 5 and the flowchart of FIG. 6 are one flowchart connected at the points A and B shown in FIGS. 5 and 6, respectively.

In step S1 of FIG. 5, the vehicle key is turned on. Next, in step S2, the vehicle status determination unit 4 acquires CAN information from the vehicle status notification unit 11. Next, in step S3, the vehicle condition determination unit 4 acquires the latitude and longitude information obtained from the GPS of the vehicle position information notification unit 12 and the map information obtained from the map database. Next, in step S4, the monitoring function selection unit 5 acquires the driver instruction information from the driver instruction information notification unit 13. Next, in step S5, the vehicle condition determination unit 4 determines the vehicle condition of the own vehicle including the current location of the own vehicle, the traveling direction of the own vehicle, and the traveling state such as the vehicle speed from the acquired CAN information, latitude and longitude information, and map information. do. The vehicle condition determination unit 4 notifies the monitoring function selection unit 5 of the determined vehicle condition.

In step S6, the monitoring function selection unit 5 has one or a plurality of peripherals based on the vehicle status of the own vehicle notified from the vehicle condition determination unit 4 and the driver instruction information notified from the driver instruction information notification unit 13. Select one or more of the monitoring function and one or more rear side monitoring functions as the selective monitoring function. Further, the monitoring function selection unit 5 determines the input video and the video processing method corresponding to the selection monitoring function. Further, the monitoring function selection unit 5 notifies the input video acquisition unit 6 of the determined input video, and also notifies the input video processing unit 7 of the determined video processing method. Further, the monitoring function selection unit 5 notifies the video output determination unit 8, the alarm output determination unit 9, and the vehicle control determination unit 10 of the video processing information regarding the determined video processing method.

When the monitoring function selection unit 5 selects the peripheral monitoring function as the selection monitoring function in step S6, the input video acquisition unit 6 switches the input video and acquires the peripheral monitoring video from the peripheral monitoring camera 2 in step S7. Then, the input video acquisition unit 6 sends the acquired peripheral monitoring video as an input video to the input video processing unit 7. In step S8, the input video processing unit 7 performs video processing on the input video sent from the input video acquisition unit 6 by using the peripheral monitoring video processing method determined as the video processing method by the monitoring function selection unit 5. conduct. Then, the input video processing unit 7 notifies the video output determination unit 8, the alarm output determination unit 9, and the vehicle control determination unit 10 of the video processing result information regarding the input video that has undergone video processing.

When the monitoring function selection unit 5 selects the rear side monitoring function as the selection monitoring function in step S6, in step S9, the input video acquisition unit 6 switches the input video and monitors the rear side from the rear side monitoring camera 3. Get the video. Then, the input video acquisition unit 6 sends the acquired rear side monitoring video as an input video to the input video processing unit 7. In step S10, the input video processing unit 7 processes the input video sent from the input video acquisition unit 6 by using the rear side video processing method determined by the monitoring function selection unit 5 as the video processing method. I do. Then, the input video processing unit 7 notifies the video output determination unit 8, the alarm output determination unit 9, and the vehicle control determination unit 10 of the video processing result information regarding the input video that has undergone video processing.

In step S11, the video output determination unit 8 determines whether or not it is necessary to output the input video processed by the input video processing unit 7 based on the video processing information and the video processing result information. If it is determined in step S11 that video display is necessary (YES), the video output determination unit 8 notifies the video output unit 14 of the video output information. In step S12, the video output unit 14 displays a necessary video. If it is determined in step S11 that the video display is unnecessary (NO), the process proceeds to step S13.

In step S13 of FIG. 6, the alarm output determination unit 9 determines whether or not it is necessary to output an alarm based on the CAN information, the video processing information, and the video processing result information. If it is determined in step S13 that the alarm output is necessary (YES), the alarm output determination unit 9 determines the alarm output method in step S14 and notifies the alarm output unit 15. In step S15, the alarm output unit 15 outputs an alarm based on the alarm output method notified from the alarm output determination unit 9. If it is determined in step S13 that the alarm output is unnecessary (NO), the process proceeds to step S16.

In step S16, the vehicle control determination unit 10 determines whether or not vehicle control is necessary based on the CAN information, the video processing information, and the video processing result information. If it is determined in step S16 that vehicle control is necessary (YES), the vehicle control determination unit 10 determines the vehicle control method in step S17, determines the vehicle control amount in step S18, and uses them as the vehicle control unit. Notify 16. In step S19, the vehicle control unit 16 controls the vehicle based on the vehicle control method and the vehicle control amount notified from the vehicle control determination unit 10. If it is determined in step S16 that vehicle control is unnecessary (NO), the process proceeds to step S20.

In step S20, it is determined whether or not the key has been turned off. If the key is turned off (YES), the series of vehicle surrounding monitoring methods is terminated. If the key is not turned off (NO), the process returns to step S2.

The vehicle status determination unit 4 acquires latitude and longitude information and map information from the vehicle position information notification unit 12 in step S3, and determines the current location of the own vehicle from the latitude and longitude information and map information in step S5. , It is also possible to directly acquire the current location of the own vehicle from the vehicle position information notification unit 12 in step S3.

Further, the operation related to the video output determination unit 8 shown in steps S11 to S12, the operation related to the alarm output determination unit 9 shown in steps S13 to S15, and the vehicle control determination unit 10 shown in steps S16 to S19. The order of the operations related to the above may be changed.

In the vehicle surroundings monitoring device configured in this way, even if the peripheral monitoring camera device and the rear side monitoring camera device are mounted on one vehicle, the monitoring function selection unit has one or more peripheral monitoring functions and a plurality of peripheral monitoring functions. Since one or more of the one or more rear side monitoring functions are selected as the selection monitoring function, and the input video and the video processing method corresponding to the selection monitoring function are determined, the input video processing unit. Can be standardized and integrated into one device. As a result, it is possible to prevent the vehicle surroundings monitoring device from becoming large. In addition, by standardizing the input video processing unit and appropriately switching the video processing according to the vehicle conditions of the own vehicle, the processing load of the arithmetic processing is reduced, so that the chip unit price of the input video processing unit can be suppressed. can.

Further, the monitoring function selection unit 5 selects one or more of the plurality of peripheral monitoring functions and the plurality of rear side monitoring functions as the selection monitoring function based on the vehicle condition of the own vehicle, and according to the selection monitoring function. Since the input video and the video processing are switched, this vehicle surroundings monitoring device can perform more appropriate surrounding monitoring. For example, the monitoring function selection unit 5 may select the rear side monitoring function when traveling at high speed and the peripheral monitoring function when traveling at low speed. However, if the driver tries to start driving from a state of being stopped on the roadway, the driver may want to use the rear side monitoring function to check the traveling vehicle behind. In the vehicle surroundings monitoring device of the present embodiment, whether the current location of the own vehicle is a roadway, a parking lot, or a location other than that based on the information notified from the vehicle position information notification unit. Can be determined. Therefore, the monitoring function selection unit 5 can automatically select the rear side monitoring function when the position of the own vehicle is on the roadway even when traveling at low speed. As a result, the rear side monitoring function is automatically activated when the vehicle is about to start traveling from a state of being stopped on the roadway.

Further, for example, the monitoring function selection unit 5 may select the peripheral monitoring function when the gear is set to the reverse gear, and may select the rear side monitoring function when the gear is set to a gear other than the reverse gear. However, the driver does not always park in the retreat. When the driver parks along the parking frame while moving forward, the driver may want to use the peripheral monitoring function to check for obstacles in the vicinity. In the vehicle surroundings monitoring device of the present embodiment, whether the current location of the own vehicle is a roadway, a parking lot, or a location other than that based on the information notified from the vehicle position information notification unit. Can be determined. Therefore, the monitoring function selection unit 5 can automatically select the peripheral monitoring function when the position of the own vehicle is a parking lot even if the gear is set to a gear other than the reverse gear. As a result, the peripheral monitoring function is automatically activated when parking along the parking frame line while moving forward.

In the present embodiment, the peripheral surveillance camera 2 and the rear side surveillance camera 3 are configured by different cameras, but some cameras may be the same camera as long as the camera has a wide imaging area. .. Even if the video is captured by the same camera, the input video processing unit processes the video using the video processing method determined by the monitoring function selection unit, and the video is processed into peripheral surveillance video or rear side surveillance video. Can be processed appropriately.

In the present embodiment, the monitoring function selection unit selects the selection monitoring function from the peripheral monitoring function and the rear side monitoring function, but other monitoring functions may be selected. For example, other monitoring functions include a front monitoring function for monitoring a vehicle traveling in front and preventing a rear-end collision.

The vehicle status determination unit 4, the monitoring function selection unit 5, the input video processing unit 7, the video output determination unit 8, the alarm output determination unit 9, and the vehicle control determination unit 10 are preferably integrated in the CPU 17. An example of this CPU 17 is shown in FIG. The CPU 17 includes a processor 100 and a storage device 101. Although the storage device is not shown, it includes a volatile storage device such as a random access memory and a non-volatile auxiliary storage device such as a flash memory. Further, the auxiliary storage device of the hard disk may be provided instead of the flash memory. The processor 100 executes the program input from the storage device 101. In this case, a program is input from the auxiliary storage device to the processor 100 via the volatile storage device. Further, the processor 100 may output data such as a calculation result to the volatile storage device of the storage device 101, or may store the data in the auxiliary storage device via the volatile storage device.

Although the present application describes various exemplary embodiments, the various features, embodiments, and functions described in one or more embodiments are limited to the application of the particular embodiment. Rather, it can be applied to embodiments alone or in various combinations.
Therefore, innumerable variations not exemplified are envisioned within the scope of the techniques disclosed in the present application. For example, it is assumed that at least one component is modified, added or omitted, and further, at least one component is extracted and combined with the components of other embodiments.

1 Vehicle surroundings monitoring device, 2 Peripheral monitoring camera, 3 Rear side monitoring camera, 4 Vehicle status judgment unit, 5 Monitoring function selection unit, 6 Input video acquisition unit, 7 Input video processing unit, 8 Video output judgment unit, 9 Alarm output judgment unit, 10 vehicle control judgment unit, 11 vehicle status notification unit, 12 vehicle position information notification unit, 13 driver instruction information notification unit, 14 video output unit, 15 alarm output unit, 16 vehicle control unit, 17 CPU, 18 Vehicle, 19, 20 Imaging range, 21 Front camera, 22 Right side camera, 23 Left side camera, 24 Rear camera, 25 Parking border, 26 Other vehicles, 27 White line, 28 Following vehicle, 31 Rear right side camera, 32 Rear left side camera, 100 processors, 101 storage devices.

Claims (6)

Peripheral surveillance cameras that capture peripheral surveillance images and peripheral surveillance cameras
A posterior surveillance camera that captures posterior surveillance video,
A vehicle condition determination unit that determines the position of the own vehicle based on the latitude and longitude information and map information of the own vehicle, and
One or more of one or more peripheral monitoring functions and one or more rear side monitoring functions are selected as the selection monitoring function based on the position of the own vehicle determined by the vehicle condition determination unit. If the selective monitoring function is the peripheral monitoring function, the peripheral monitoring image is determined as an input image, and if the selective monitoring function is the rear side monitoring function, the rear side monitoring image is used as the input image. A monitoring function selection unit that determines the video processing method corresponding to the selection monitoring function, and a monitoring function selection unit.
An input video acquisition unit that acquires the input video determined by the monitoring function selection unit from either the peripheral surveillance camera or the rear side surveillance camera, and the input video acquisition unit.
A vehicle surroundings monitoring device including an input video processing unit that performs video processing on the input video acquired by the input video acquisition unit by the video processing method determined by the monitoring function selection unit.
When the vehicle condition determination unit determines the current location of the own vehicle and the vehicle condition determination unit determines that the current location of the own vehicle is a parking lot, the monitoring function selection unit selects the peripheral monitoring function. When selected as a monitoring function and the vehicle condition determination unit determines that the current location of the own vehicle is a roadway, the monitoring function selection unit is characterized in that the rear side monitoring function is selected as the selection monitoring function. Peripheral monitoring device for vehicles. Whether or not to output the input video processed by the input video processing unit based on the video processing information notified from the monitoring function selection unit and the video processing result information notified from the input video processing unit. The vehicle surroundings monitoring device according to claim 1, further comprising a video output determination unit for determination. Further provided with an alarm output determination unit for determining whether or not to output an alarm based on the video processing information notified from the monitoring function selection unit and the video processing result information notified from the input video processing unit. The vehicle surrounding monitoring device according to claim 1 or 2. Further provided with a vehicle control determination unit for determining whether or not to perform vehicle control based on the video processing information notified from the monitoring function selection unit and the video processing result information notified from the input video processing unit. The vehicle peripheral monitoring device according to any one of claims 1 to 3. When the driver instruction information is notified, the monitoring function selection unit selects the selection monitoring function by giving priority to the driver instruction information over the vehicle condition of the own vehicle determined by the vehicle condition determination unit. The vehicle surrounding monitoring device according to any one of claims 1 to 4, which is characterized. A vehicle condition determination step for determining the position of the own vehicle based on the latitude and longitude information and map information of the own vehicle, and
One or more of the one or more peripheral monitoring functions and the one or more rear side monitoring functions based on the position of the own vehicle determined in this vehicle condition determination step is selected as the selective monitoring function. A monitoring function selection step of selecting and determining either a peripheral monitoring image or a rear side monitoring image as an input image based on the selection monitoring function and determining a video processing method corresponding to the selection monitoring function.
An input video acquisition step of acquiring the input video determined in this monitoring function selection step from either a peripheral surveillance camera or a rear side surveillance camera, and
A vehicle surroundings monitoring method including an input video processing step in which video processing is performed on the input video acquired in the input video acquisition step by the video processing method determined in the monitoring function selection step.
When the current location of the own vehicle is determined in the vehicle condition determination step and it is determined in the vehicle condition determination step that the current location of the own vehicle is a parking lot, the peripheral monitoring function is selectively monitored in the monitoring function selection step. It is characterized in that when it is selected as a function and it is determined in the vehicle condition determination step that the current location of the own vehicle is a roadway, the rear side monitoring function is selected as the selection monitoring function in the monitoring function selection step. Vehicle surrounding monitoring method.

Images