KR101270602B1 - Method for providing around view of vehicle - Google Patents

Abstract

The present invention provides a diagnostic image of each device and the exterior of the vehicle and the highlight image of the occurrence of the abnormality before driving the vehicle, and pre-warning the risk of collision with obstacles approaching the blind spot, the distance information between the obstacle and each door of the vehicle The present invention relates to a method of providing a surrounding image of a vehicle to improve a safe driving environment and convenience by calculating a door opening angle based on the provided angle to a driver.
A method of providing a peripheral image of a vehicle in which a plurality of cameras are installed at an outer end of each side mirror and front, rear, and bottom of each vehicle, the method comprising: detecting entry of a vehicle diagnostic mode, and a plurality of cameras when the side mirror is folded Receiving vehicle diagnostic image data from the vehicle; generating vehicle diagnostic image by combining the plurality of received vehicle diagnostic image data; receiving state detection information of the vehicle from a sensor installed at each part of the vehicle; Comparing the diagnostic image with a preset reference image and analyzing the received vehicle state information to diagnose a vehicle state; if an abnormality occurs as a result of the vehicle state diagnosis, highlighting an error occurrence part in the vehicle diagnostic image, and highlighting the vehicle And displaying the diagnostic image and the abnormal diagnostic information.

Description

How to provide video around the vehicle {METHOD FOR PROVIDING AROUND VIEW OF VEHICLE}

The present invention relates to a method for providing a surrounding image of a vehicle, and in particular, provides diagnostic information of each device and exterior of a vehicle and highlight images of an error occurrence unit before driving the vehicle, and pre-warns the risk of collision with obstacles approaching from a blind spot. In addition, the present invention relates to a method of providing a surrounding image of a vehicle, which improves a safe driving environment and convenience by calculating and providing a door opening angle to a driver based on distance information between an obstacle and each vehicle door.

 In general, the driver uses the left and right outer side mirrors and the inner room mirrors to assist the driver while driving or parking to visually check the surroundings of the vehicle.

However, the area in which the driver can grasp the surroundings of the vehicle through the exterior side mirror and the interior room mirror is limited, and thus, the watch is not sufficiently secured, so that the situation around the vehicle cannot be accurately recognized.

In order to solve this problem, recently, a plurality of cameras have been installed on each part of the vehicle, and the images captured by the camera can be monitored inside the vehicle, and when the collision with an obstacle is expected, an around view image outputting a warning sound is provided. A system has been provided.

Since the conventional around view system simply displays the surrounding image of the vehicle, when the driver has an abnormality in the vehicle, the driver does not easily grasp the abnormal state of the vehicle, thereby causing a safety accident.

In addition, the conventional around view system detects an obstacle through an ultrasonic sensor and simply outputs a warning sound through a speaker when a collision risk with the obstacle is expected. There is a drawback that cannot be grasped. In addition, since the obstacle is detected by the ultrasonic sensor, there is a disadvantage that the obstacle is not recognized when there is an obstacle having a low height, such as a road threshold.

In addition, since the driver subjectively determines whether the door can be opened by referring to the around view image when parking, even if the parking is avoided by obstacles, the door may be difficult to get off due to the small opening angle of the door, but the driver may get off. There is a disadvantage that can not know objectively. In particular, when there is an obstacle having a low height, such as a road boundary jaw around the vehicle, because the ultrasonic sensor does not recognize the obstacle, it may be impossible to get off the door due to the obstacle, but there is a disadvantage that it is not known.

An object of the present invention is to compare the detection information received from the sensors installed in each device of the vehicle and the external appearance diagnostic image information of the vehicle with the preset information, if an abnormality occurs, the highlight image is displayed on the vehicle exterior diagnostic image and the abnormality occurrence information The present invention provides a method of providing a surrounding image of a vehicle so that a driver can grasp a state of the vehicle before driving.

In addition, when the obstacle approaching the collision risk is approaching in a blind spot that the driver cannot visually recognize, the present invention enlarges and displays the image in the corresponding direction and provides the distance and the approach speed to the obstacle so that the driver can interact with the obstacle. The present invention provides a method of providing a surrounding image of a vehicle so as to quickly deal with a collision risk.

In addition, the present invention calculates the distance and height information between the low-height obstacle and the vehicle door is detected from the image around the vehicle and not detected by the ultrasonic sensor from the vehicle surrounding image, the calculated distance and obstacle between the vehicle door and the obstacle The present invention provides a method of providing a surrounding image of a vehicle capable of determining whether or not it is possible to get off after calculating an opening angle of a door based on height information.

A method of providing a peripheral image of a vehicle in which a plurality of cameras are installed at an outer end of each side mirror and front, rear, and bottom surfaces of a vehicle for detecting a problem, the method comprising: detecting entering a vehicle diagnostic mode, and in a state in which a side mirror is folded Receiving vehicle diagnostic image data from a plurality of cameras, generating a vehicle diagnostic image by combining the plurality of received vehicle diagnostic image data, and receiving state detection information of the vehicle from a sensor installed at each part of the vehicle. And diagnosing the vehicle state by comparing the vehicle diagnosis image with a preset reference image and analyzing the received vehicle state information, and highlighting an error occurrence part in the vehicle diagnosis image when an abnormality occurs as a result of the vehicle state diagnosis. And displaying the processed vehicle diagnosis image and the abnormal diagnosis information. .

Here, when the vehicle diagnostic mode is completed, receiving surrounding image data of the vehicle from a plurality of cameras with the side mirrors unfolded; generating and displaying the surrounding image of the vehicle by combining the plurality of received vehicle surrounding image data; When the approach detection signal of the obstacle is received from the ultrasonic sensor installed in the vehicle, calculating the distance and the approach speed of the obstacle, and if the risk of collision is expected as a result of the distance and the approach speed of the obstacle, The method may further include displaying distance and approach speed information from the obstacle.

The method may further include generating parking space information when the vehicle enters the parking mode, calculating a parking progress rate of the vehicle based on the generated parking space information; At least one or more of calculating the distance to the obstacle, the step of calculating all the door opening angle of the vehicle on the basis of the calculated distance information of the obstacle and comparing with the predetermined opening angle of the door and all calculated door opening angle If less than the predetermined door opening angle may further comprise the step of warning to not get off.

In addition, the parking space information may be generated based on distance information of the vehicle and the parking line or surrounding obstacles.

In addition, calculating the distance to the obstacle around the vehicle when entering the parking completion mode may further calculate the height information of the obstacle.

The present invention provides each device and vehicle exterior state information before driving the vehicle, so that the driver can grasp the state of the vehicle, and warn in advance of the risk of collision that the collision risk is expected in a blind spot that the driver cannot visually recognize. By allowing the driver to cope quickly with the risk of collision, there is an advantage that can provide a safe driving environment for the driver.

In addition, the present invention has the advantage of improving the convenience of parking by informing the availability of the door after calculating the door opening angle in consideration of the height of the low obstacle that is not recognized by the ultrasonic sensor or the naked eye after parking.

1 is a block diagram schematically showing a system for providing a vehicle surrounding image of the present invention.
2 is a plan view of a vehicle to which the vehicle surrounding image providing system of the present invention is applied.
3 to 5 are flowcharts sequentially showing a method of providing a surrounding image of a vehicle according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing a vehicle surrounding image providing system of the present invention, Figure 2 is a plan view of a vehicle to which the vehicle surrounding image providing system of the present invention is applied, the sensor unit 100, ECU 200, display Device 300 and speaker 400.

The sensor unit 100 includes a vehicle state sensor 110, a start sensor 120, a gear position sensor 130, a vehicle speed sensor 140, an ultrasonic sensor 150, a touch sensor 160, and a plurality of sensors. Including the camera 170, the information detected by the sensor unit 100 is transmitted to the ECU 200 connected through a controller area network (CAN).

In this case, the vehicle state sensor 110 may include an engine sensor for detecting an engine state such as a throttle angle, an ignition timing, a load, a fuel trip, a coolant sensor for detecting a temperature of coolant, an intake sensor for detecting an intake pressure, and a temperature. It includes a tire pneumatic sensor for detecting the air pressure of the door, a door opening sensor for detecting whether the door is open, a trunk opening sensor for detecting whether the trunk is open or the like.

The start detection sensor 120 detects the operation signal of the generator when the vehicle starts, the gear position detection sensor 130 detects the state of the gear from the gear switch, and the vehicle speed detection sensor 140 detects the driving speed of the vehicle. do.

The ultrasonic sensor 150 is installed in a plurality of front and rear and rear and rear of the vehicle to detect obstacles around the vehicle, so as to detect the distance to the obstacle and the approaching speed of the obstacle.

The touch sensor 160 is integrated with the display apparatus 300 to sense a user's touch signal and receives various menu information such as speaker volume and mode selection.

The plurality of cameras 170 are installed at the outer end of each side mirror M and at the front and rear sides and the bottom of the vehicle to capture the surrounding image of the vehicle. At this time, the side mirror M is foldable, and the state is turned off and remains folded for a set time after the start of the start, and is extended when the diagnosis mode cancellation command is input after the start time exceeds the set time or the user inputs the diagnosis. Accordingly, as shown in (a) of FIG. 2, when the side mirror is folded, the camera C1 mounted on the side mirror M photographs the side surface of the vehicle, and when the side mirror M is open. The camera C1 captures the surrounding image of the vehicle. Here, the vehicle side photographed image is used as a vehicle diagnostic image to be described later. In addition, among the plurality of cameras installed in the vehicle, the cameras C2 and C3 installed at the front and rear of the vehicle operate only when the vehicle surrounding image is acquired, and the camera C4 installed on the bottom of the vehicle operates only in the vehicle state diagnosis mode.

Meanwhile, the ECU 200 controls the display apparatus 300 or the speaker 400 and the overall driving of the vehicle based on the detection information transmitted from the sensors installed in the respective parts of the vehicle.

Here, the ECU 200 includes an image processor 210, a vehicle condition diagnosis unit 220, a touch coordinate calculator 230, a parking area scanning unit 240, a parking progress rate calculator 250, and a door opening angle calculator. 250, a data storage unit 270, and a control unit 280.

The image processor 210 generates a vehicle surrounding image and a vehicle diagnostic image by combining image information transmitted from the plurality of cameras 170. In this case, the image processor 210 generates a vehicle surrounding image by performing distortion correction, viewpoint transformation, and synthesis on the plurality of captured images. In addition, when the vehicle state diagnosis unit 220 determines that the vehicle has an abnormality, the image processor 210 performs a highlight process on a portion where the abnormality has occurred.

The vehicle state diagnosis unit 220 compares the vehicle state detection result input from the vehicle state sensor 110 and the vehicle diagnosis image generated by the image processor 210 with a preset reference image to determine whether an abnormality has occurred.

The touch coordinate calculator 230 calculates coordinates of the corresponding touch position by analyzing touch information transmitted from the touch sensor 160.

The parking area scanning unit 240 generates parking space information based on the distance information with respect to the parking line or obstacles around the vehicle. In this case, the parking space information is given priority to the parking line information, and if there is no parking line information or the distance to the obstacle is closer than the distance to the parking line, the parking space information is given priority. In addition, if it is determined that the obstacle is detected by the ultrasonic sensor, the parking area scanning unit 240 may park the space information based on the distance information of the obstacle detected by the ultrasonic sensor and the parking line information generated by the image processor 210. Create In addition, when the obstacle is not detected from the ultrasonic sensor 150 and it is determined that the obstacle exists in the vehicle surrounding image, the parking space information is generated based on the distance information with the obstacle calculated by the image processor 210.

The parking progress calculating unit 250 calculates an area ratio where the vehicles overlap in the parking space generated by the parking area scanning unit 240, and determines that parking is completed when the vehicle area overlaps 100% in the parking space. Here, the parking progress rate calculation is performed when the shift gear is in the "P" state or the shift gear is in the "R" state and the vehicle speed is "0" as a result of the detection by the gear position sensor 130.

The door opening angle calculator 260 calculates the distance between the handle and the obstacle of each door and each door based on the parking space information generated by the parking area scanning unit 240 and the height information of the obstacle calculated by the image processor 210. Calculate the opening angle of the door using the bottom height and the height information of the obstacle. That is, conventionally, since the height of the obstacle is not considered low, the door is difficult to open due to the obstacle after parking, which is inconvenient to get off. Accordingly, the present invention calculates the opening angle of the door in consideration of the height information as well as the distance to the obstacle to provide a calculation result, so that the driver can park the obstacle to avoid the low height.

The data storage unit 270 stores the image information processed by the image processor 210 and the vehicle information detected by the sensor unit 100 in real time. In addition, the reference image of the vehicle is stored in the data storage unit 270, and the opening angle of the door is set corresponding to the distance between the obstacle and each door handle of the vehicle and the height of the obstacle and the bottom height of each door. The door opening angle that can be unloaded is set.

The controller 280 causes the image around the vehicle processed by the image processor 210 to be displayed on the display apparatus 300. The display apparatus 300 controls the display apparatus 300 to display the vehicle diagnostic image in which the generator is highlighted.

In addition, when it is determined that there is an obstacle in the surrounding image of the vehicle generated by the image processor 210, the controller 280 calculates the size and height information of the obstacle. Here, the height information of the obstacle is calculated using a three-dimensional information extraction method using a single camera. For example, an obstacle is extracted from successive frames taken by one camera, the distance traveled by the obstacle between two successive frames is calculated, and then the first image is taken from the actual camera. After assuming that the image is captured and the second image is captured by the virtual camera, a method of extracting three-dimensional information of the moving object may be used by using the image captured by the actual camera and the image captured by the virtual camera. Here, since a method of extracting 3D information using a single camera is already known, a detailed description thereof will be omitted.

In addition, the controller 280 determines whether the collision risk with the surrounding obstacles is based on the distance information with the surrounding obstacles detected by the ultrasonic sensor 150 and the vehicle speed information detected by the vehicle speed sensor 140, and the collision risk. If this is expected, the magnified image in the anticipated collision direction and the distance to the obstacle and the approaching speed are displayed through the display apparatus 300, and a warning sound indicating a collision risk is output through the speaker 400.

In addition, the controller 280 performs various signal processing such as, for example, adjusting speaker volume and canceling a diagnostic mode so that signal processing corresponding to the coordinate information calculated by the touch coordinate calculating unit 230 is performed.

In addition, when the control unit 280 determines that the parking is completed by the parking progress calculation unit 250, the door opening angle calculated by the door opening angle calculation unit 260 is compared with a preset door opening angle that is possible to get off, If it is determined that it is impossible to get a warning to get off through the display device 300 and the speaker 400, and when possible to get off the vehicle surrounding image and the door opening angle is displayed.

2 to 4 are flowcharts sequentially showing a method of providing a surrounding image of a vehicle according to the present invention.

2 is a flowchart illustrating a process of providing a vehicle diagnostic image, and first detects entry of a vehicle diagnostic mode (S100). Here, when the start is detected by the start detection sensor 120 or the coordinate information input by the touch sensor 160 corresponds to the vehicle diagnosis mode 170 selection menu, the start detection sensor 120 determines that the vehicle diagnosis mode is the vehicle diagnosis mode.

When the vehicle diagnostic mode entry is detected, the vehicle diagnostic image data is received (S110), and the vehicle diagnostic image is generated by performing distortion correction, viewpoint transformation, and synthesis processing on the received vehicle diagnostic image (S120). In this case, the vehicle diagnostic image corresponds to a vehicle side image photographed by a plurality of cameras installed outside the end of the side mirror while the side mirror is folded.

Subsequently, each state detection information of the vehicle is received from the vehicle state detection sensor 110 (S130).

Then, the vehicle state is diagnosed based on the received vehicle state detection information and the vehicle diagnostic image to determine whether an abnormality has occurred in the vehicle (S140). In this case, the information diagnosis of the vehicle compares the value detected by each sensor of the vehicle with the setting value, and determines that an abnormality has occurred when it is out of the setting range. In addition, it is determined that an abnormality occurs when the vehicle diagnosis image does not match with the preset vehicle reference image. For example, when oil leakage occurs, the image acquired through the camera installed on the bottom of the vehicle is inconsistent with the preset image, and when the gas, the door or the trunk of the vehicle is opened, or an object is caught in the door, Since the set image is inconsistent, it is determined that an abnormality has occurred in this case.

On the other hand, if it is determined that the abnormality in the vehicle as a result of the vehicle condition diagnosis, highlight the portion where the abnormality in the vehicle diagnostic image (S150). In this case, the highlighting process may include a method of generating a point at an abnormal occurrence point or a method of making the color of the abnormal occurrence part different from the surrounding area.

In operation S160, the highlighted vehicle diagnostic image and abnormal diagnosis information are displayed.

Subsequently, when the vehicle diagnosis mode is completed, the vehicle surrounding image data is received from the plurality of cameras 170 (S170), and the vehicle surrounding image is generated by performing distortion correction, viewpoint conversion, and synthesis processing on the received vehicle surrounding image. (S180). At this time, the vehicle surrounding image data corresponds to the vehicle outer peripheral image photographed by a camera installed on the outside of the side mirror end in the state in which the side mirror is opened. In addition, the diagnosis mode of the vehicle is determined to be completed when the vehicle state diagnosis determines that the vehicle is in a normal state and when the diagnosis mode cancellation signal by the touch sensor operation of the user is input.

Thereafter, the generated vehicle surrounding image is displayed on the display apparatus 300 (S190).

As described above, the present invention provides a safe driving environment to the driver by displaying the condition diagnosis information of the vehicle and the highlight image of the abnormality generating unit before driving of the vehicle so that the driver can grasp the state of the vehicle before driving.

3 is a flowchart illustrating a process of providing collision risk information with an obstacle.

First, when an obstacle approach detection signal is received from the ultrasonic sensor 150 while the surrounding image of the vehicle is displayed (S200), the distance between the obstacle and the vehicle and the approach speed of the obstacle are calculated (S210).

Then, it is determined whether the collision with the obstacle is possible based on the current driving direction, the traveling speed, the approaching direction of the obstacle, the distance between the obstacle and the vehicle, and the access speed information of the obstacle (S220).

Subsequently, when the risk of collision with the obstacle is anticipated, the distance information with the obstacle and the approach speed information of the obstacle are displayed together with the enlarged image of the anticipated collision direction (S230).

As described above, when the obstacle approaching a collision risk is approached in a blind spot that the driver cannot visually recognize, the driver zooms in and displays the image in the corresponding direction and provides the distance and approach speed to the obstacle so that the driver can interact with the obstacle. Respond quickly to collision risks to ensure safe driving.

4 is a flowchart illustrating a process of providing the opening angle of the door and the availability of getting off the vehicle after parking is completed.

First, the entrance of the parking mode is detected (S300). Here, the entry of the parking mode corresponds to a case in which the shift gear is in an “R” state as a result of the detection by the gear position sensor 130 or when a parking mode selection signal is input by a user's touch sensor manipulation.

Subsequently, parking space information is generated based on the distance information between the parking line and the surrounding obstacle (S310), and the parking progress rate of the vehicle in the parking space is calculated (S320). At this time, the parking progress calculation unit 250 is calculated through the area where the vehicle overlaps in the parking space. Here, the parking progress rate calculation is performed when the shift gear is in the "P" state or the shift gear is in the "R" state and the vehicle speed is "0" as a result of the detection by the gear position sensor 130.

If the vehicle area is 100% overlapped in the parking space as a result of calculating the parking progress rate, it is determined that the parking completion mode has been entered (S330).

Subsequently, when there is an obstacle in the vicinity of the vehicle, the vehicle surrounding image generated in two dimensions is converted into a three-dimensional image to calculate the height information of the obstacle and the distance between the obstacle and each door handle (S340).

The door angle is calculated by matching the calculated distance information with the height of the lower end of the door and the height of the obstacle with a preset value in the database (S350), and it is determined whether it is possible to get off (S350). In this case, whether or not it is possible to get off, it is determined that getting off is impossible when at least one or more of the calculated door opening angles are smaller than the set door opening angles by comparing all calculated door opening angles with a predetermined door opening angle. For example, when the obstacle is located close to the vehicle but the height of the obstacle is lower than the height of the bottom of the vehicle door, the door opening angle is increased, so it is determined that the vehicle can be disembarked.

Subsequently, if it is determined that getting off is impossible, the vehicle is warned that the getting off is not possible through the display apparatus 300 and the speaker 400, and if it is determined that getting off is possible, the image around the vehicle is displayed together with the door opening angle information (S380).

As described above, when the parking of the vehicle is completed, the present invention calculates the door opening angle based on the distance information between the surrounding obstacle and the vehicle door, determines whether the driver can get off according to the calculated door opening angle, and then informs the driver. You can objectively determine whether you can get off. In particular, the present invention further improves the parking convenience by calculating the door opening angle in consideration of the height of the low obstacle that is not recognized by the ultrasonic sensor or the naked eye.

Although the present invention has been described in connection with the above-mentioned preferred embodiments, it is possible to make various modifications and variations without departing from the spirit and scope of the invention. Accordingly, the appended claims are intended to cover such modifications or changes as fall within the scope of the invention.

100: sensor unit 110: vehicle state detection sensor
120: start detection sensor 130: gear position detection sensor
140: vehicle speed detection sensor 150: ultrasonic sensor
160: touch sensor 170: camera
200: ECU 210: image processing unit
220: vehicle condition diagnosis unit 230: touch coordinate calculation unit
240: parking area scanning unit 250: parking progress calculation unit
260: door opening angle calculation unit 260: control unit
300: display device 400: speaker

Claims (5)

In the peripheral image providing method of a vehicle in which a plurality of cameras are installed on the outer end of each side mirror and the front, rear and bottom of the vehicle,
Detecting entry into a vehicle diagnostic mode;
Receiving vehicle diagnostic image data from a plurality of cameras while the side mirror is folded;
Generating a vehicle diagnostic image by combining the received plurality of vehicle diagnostic image data;
Receiving state detection information of a vehicle from sensors installed in respective parts of the vehicle;
Comparing the vehicle diagnosis image with a preset reference image and analyzing the received vehicle state information to diagnose a vehicle state;
Highlighting an error occurrence part in the vehicle diagnosis image when an error occurs as a result of the vehicle condition diagnosis;
Displaying the highlighted vehicle diagnostic image and fault diagnosis information;
Generating parking space information when a parking mode entry is detected;
Calculating a parking progress rate of the vehicle based on the generated parking space information;
Calculating a distance to an obstacle around the vehicle when it is detected that the parking completion mode is entered as a result of the parking progress rate calculation;
Calculating all door opening angles of the vehicle based on the calculated distance information with the obstacle, and comparing the door opening angles with preset doors; And
And disabling a warning that at least one or more of the calculated door opening angles are smaller than a predetermined door opening angle.
The method of claim 1,
Receiving peripheral image data of a vehicle from a plurality of cameras when the side mirror is opened when the vehicle diagnosis mode is completed;
Generating and displaying a surrounding image of the vehicle by combining the received plurality of surrounding image data of the vehicle;
Calculating a distance and an approach speed of the approach obstacle when the approach detection signal of the obstacle is received from the ultrasonic sensor installed in the vehicle;
And displaying the collision prediction direction enlarged image and the distance and approach speed information of the obstacle when the collision risk is predicted as a result of the calculation of the distance and the approach speed with the obstacle.
delete The method of claim 1,
The parking space information is generated based on the distance information of the parking line or the surrounding obstacles and the vehicle.
The method of claim 1,
Computing the distance to the obstacle around the vehicle when entering the parking completion mode, the vehicle surrounding image providing method characterized in that further calculating the height information of the obstacle.

Images