KR101923899B1 - Apparatus and Method for AVM Calibration using Tire Press Monitoring Information - Google Patents

Abstract

The vehicle surround view calibration apparatus using tire pressure monitoring information includes a plurality of ambient view cameras mounted on a vehicle, a memory for storing reference position information for each of the plurality of ambient view cameras, A camera real time position calculation unit for calculating real time position information by adjusting the reference position information for each of the plurality of view cameras in accordance with the tire real time pressure information, And an ambient view generation unit for three-dimensionally converting each of the images photographed by the plurality of surrounding view cameras based on real-time location information of the corresponding camera, and synthesizing the three-dimensional converted images to generate an ambient view image.

Description

Technical Field [0001] The present invention relates to an apparatus and a method for calibrating an averaged view of a vehicle using tire pressure monitoring information,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a vehicle surround view technology, and more particularly, to an apparatus and method for a vehicle surround view calibration capable of providing an accurate surround view image based on a calibrated image.

Recently, the automobile technology has been developed to improve driver convenience and safety. Various facilities and apparatus and systems for operating and controlling the same have been developed. In particular, a video camera system is used in a wide range of automobiles. For example, by installing a camera on the outside of a car, a variety of systems have been developed through a display on the driver's instrument panel to allow the driver to recognize external images around the vehicle, recognize objects and lanes around the vehicle, and warn the driver .

Among these, a vehicle AVM (Around View Monitoring) system is a system that displays an image of a vehicle in all directions, and typically a camera is mounted on the front / rear and left / right sides of the vehicle, Synthesizes the surrounding images, displays them on a single screen, and provides them to the driver. Accordingly, the driver can accurately recognize the surroundings of the vehicle through the displayed surrounding environment, and can conveniently park the vehicle without viewing the side mirrors or the rearview mirror.

However, the mounting error of the cameras causes a flaw in the surround view such as a misfit between images taken by individual cameras. For example, the vehicle AVM (Around View Monitor) system is tuned assuming that the cameras are correctly positioned at the initial design position and attitude at vehicle development time. If the cameras can not maintain their initial design position and posture due to various external environments May occur.

On the other hand, when the tire pressure is higher than the specified pressure during running of the vehicle in general, the ride feeling during driving is poor and tire breakage may occur. If the pressure is lower than the specified pressure, steering operation force becomes large and wear of the tire side occurs. . Such a tire pressure change can change the initial design position and attitude of the AVM camera for a vehicle, which may result in an incorrect shot screen and a distorted shot. In this way, when the driver looks at a distorted screen that is largely different from the actual situation, unnecessary collision of the vehicle may occur.

Also, if a lot of people get in the car, or oil is fed, when the luggage is put on the trunk, the car will fall down due to its weight, resulting in distorted images.

Accordingly, the present invention provides an apparatus and method for estimating a vehicle surround view using tire pressure monitoring information capable of correcting a photographing screen distortion caused by a change in position or angle of an arousal view camera caused by a tire pressure change.

In the present invention, the tire pressure is recorded while the vehicle is parked with a car, and a change in the camera position is measured and calibrated by measuring the tire pressure change when the doors, the oil port doors and the trunks are opened and all the doors are locked. The present invention provides an apparatus and method for calibrating an over-the-vehicle vehicle using tire pressure monitoring information.

According to one embodiment, an apparatus for calibrating an overview vehicle for a vehicle using tire pressure monitoring information includes: a plurality of ambient view cameras mounted on a vehicle; a memory for storing reference position information for each of the plurality of ambient view cameras; A camera-real-time position calculation unit for calculating real-time position information by adjusting the reference position information for each of the plurality of the surrounding cameras according to the real-time pressure information for each tire, And an ambient view generation unit for three-dimensionally converting each of the images photographed by the plurality of surrounding cameras based on real-time location information of the corresponding camera, and synthesizing the three-dimensional converted images to generate an ambient view image do.

According to another embodiment of the present invention, there is provided a method of calibrating a vehicle surround view using tire pressure monitoring information, comprising the steps of: storing reference position information for each of the plurality of view cameras; monitoring real-time pressure information for each tire; Calculating real-time position information by adjusting the reference position information for each of the plurality of surround-view cameras according to tire-by-tire real-time pressure information, calculating each of the images photographed by the plurality of surround- And converting the three-dimensional converted images into a three-dimensional converted image to generate an ambient view image.

According to the present invention, it is possible to correct the distortion of the photographing screen according to the position or angle change of the AVM camera caused by the tire pressure change. Also, there is an advantage in that a precise AVM image can be obtained by expressing the coordinates of each AVM camera according to the tire pressure change as six elements represented by (x, y, z, pan, tilt, and rotate).

1 is an external perspective view of a vehicle including an apparatus for calibrating an overview of a vehicle using tire pressure monitoring information according to an embodiment of the present invention.
FIG. 2A is a view schematically showing the position of an around-view camera attached to the vehicle of FIG. 1. FIG.
FIG. 2B is a diagram illustrating a three-dimensional surround view image based on an image captured by the surround view camera of FIG. 2A. FIG.
Fig. 3A is a schematic view showing an object located on the vehicle rear right side of the vehicle.
FIG. 3B is an illustration of a three-dimensional surround view image in the absence of the occupant of the vehicle shown in FIG. 3A. FIG.
FIG. 3C is an illustration of a three-dimensional surround view image when a person rides on the right rear seat of the vehicle shown in FIG. 3A.
FIG. 4A is an internal block diagram of an apparatus for calibrating an overview of a vehicle using tire pressure monitoring information according to an exemplary embodiment of the present invention. Referring to FIG.
4B is an internal block diagram of a camera real-time position calculation unit according to an embodiment of the present invention.
4C is an internal block diagram of the surround view calculation unit according to an embodiment of the present invention.
5A and 5B are diagrams for explaining coordinate information of a camera.
FIG. 6A is a flowchart illustrating a method for calibrating an overview of a vehicle using tire pressure monitoring information according to an exemplary embodiment of the present invention. Referring to FIG.
6B is a flowchart illustrating a camera real-time position calculation step according to an embodiment of the present invention.
6C is a flowchart for explaining an overview view generation step according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. , Which may vary depending on the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification.

Each block of the accompanying block diagrams and combinations of steps of the flowcharts may be performed by computer program instructions (execution engines), which may be stored in a general-purpose computer, special purpose computer, or other processor of a programmable data processing apparatus The instructions that are executed through the processor of the computer or other programmable data processing equipment will generate means for performing the functions described in each block or flowchart of the block diagram.

These computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner so that the computer usable or computer readable memory It is also possible for the instructions stored in the block diagram to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of the flowchart.

And computer program instructions may be loaded onto a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible that the instructions that perform the data processing equipment provide the steps for executing the functions described in each block of the block diagram and at each step of the flowchart.

Also, each block or step may represent a portion of a module, segment, or code that includes one or more executable instructions for executing the specified logical functions, and in some alternative embodiments, It should be noted that functions may occur out of order. For example, two successive blocks or steps may actually be performed substantially concurrently, and it is also possible that the blocks or steps are performed in the reverse order of the function as needed.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the following embodiments of the present invention may be modified into various other forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to enable those skilled in the art to more fully understand the present invention.

The vehicle described herein may be a concept including a car, a motorcycle. Hereinafter, the vehicle will be described mainly with respect to the vehicle. On the other hand, the vehicle described in the present specification may be a concept including both a vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source.

Meanwhile, the vehicle averaged vision calibrating apparatus using the tire pressure monitoring information described in the present specification is provided with a plurality of cameras and a tire pressure monitoring system, and is configured to synthesize / output a plurality of images photographed by a plurality of cameras, And may be an apparatus for calibrating and providing an overview image.

1 is an external perspective view of a vehicle including an apparatus for calibrating an overview of a vehicle using tire pressure monitoring information according to an embodiment of the present invention.

1, the vehicle 1 includes wheels 2FR, 2FL, 2RL, ... rotated by a power source and a plurality of surrounding view cameras 10L, 10B, 10R, 10F mounted on the vehicle 1 . On the other hand, only the left camera 10L and the front camera 10F are shown in the drawings for convenience. The plurality of surrounding view cameras 10L, 10B, 10R, and 10F can be activated when the vehicle speed is less than or equal to the predetermined speed, or when the vehicle is propelled, to acquire the shot image, respectively. However, the present invention is not limited to this, and may be activated depending on the driver's request or preset in the course of running without any speed change.

FIG. 2A is a view schematically showing the position of the surround view camera attached to the vehicle of FIG. 1, and FIG. 2B is an illustration of a three-dimensional surround view image based on an image shot by the surround view camera of FIG.

2A, a plurality of the surrounding view cameras 10L, 10B, 10R, and 10F may be disposed on the left, rear, right, and front of the vehicle, respectively. In particular, the left camera 10L and the right camera 10R may be disposed in a case surrounding the left side mirror and a case surrounding the right side mirror, respectively. On the other hand, the rear camera 10B and the front camera 10F can be disposed in the vicinity of the trunk switch and in the vicinity of the ambulance or the ambulance, respectively. However, the present invention is not limited thereto.

Each of the plurality of images photographed in the plurality of the surrounding view cameras 10L, 10B, 10R and 10F is transmitted to a processor (corresponding to 30 and 40 in Fig. 3A) mounted in the vehicle 1, And combines the images to generate the surround view image. Although the processor according to the embodiment of the present invention has been described as being installed inside the vehicle 1, it is only an example, and it can be implemented in software form and mounted on an ECU of a vehicle or as a portable device.

2B, the three-dimensional surround view image includes a first image area 3Li from the left camera 10L, a second image area 3Bi from the rear camera 10B, a second image area 3Bi from the right camera 10R, 3 image region 3Ri and a fourth image region 3Fi from the front camera 10F are synthesized into a three-dimensional image.

FIG. 3A is a schematic view showing an object located on the vehicle rear right side of the vehicle, FIG. 3B illustrates a three-dimensional surround view image in the absence of a vehicle occupant shown in FIG. 3A, Dimensional surround view image when a person rides on the right and left seats of the vehicle.

When an object 4 of a cube is placed on the right rear side of the vehicle as shown in Fig. 3A, an approach view image as shown in Fig. 3B is generated. That is, the 3-dimensional surround view image 4i is expressed in a well-matched state. 3C, when a person rides on the right rear wheel and the vehicle body on the right rear tire side sits down, the stitching of the well-matched surround view image is shifted as shown in FIG. 3B. That is, as shown in FIG. 3C, the cubes 4i-1 and 4i-2 appear to have poor matching with the two images. That is, when the air pressure of one or more tires is different from the air pressure of other tires, the vehicle tilts in one direction. As a result, the screen received through the cameras mounted on the vehicle also tilts and a screen do. In this way, when the driver views the screen with a large difference from the actual one, a problem such as unnecessary collision of the vehicle may occur.

Therefore, in the present invention, the vehicle surround view calibration apparatus using the tire pressure monitoring information includes a plurality of cameras and a tire pressure monitoring system, and according to the tire pressure monitoring information, a plurality of images photographed by the plurality of cameras are combined / A three-dimensional surround view image can be provided.

FIG. 4A is an internal block diagram of an apparatus for calibrating an overview of a vehicle using tire pressure monitoring information according to an exemplary embodiment of the present invention, FIG. 4B is an internal block diagram of a camera real time position calculation unit according to an exemplary embodiment of the present invention, FIG. 4C is an internal block diagram of the surround view generating unit according to an embodiment of the present invention, and FIGS. 5A and 5B are views for explaining a three-dimensional rectangular coordinate system of the camera.

Referring to FIG. 4A, a vehicle surround view calibration apparatus (hereinafter referred to as "apparatus") using tire pressure monitoring information includes a plurality of surrounding view cameras 10L, 10B, 10R, and 10F, a tire pressure monitoring unit 20, A camera real time position calculation unit 30, an ambient view generation unit 40, and a memory 50. [

The plurality of the surrounding view cameras 10L, 10B, 10R, and 10F are installed to the left, right, front, and rear to provide the surround view image and acquire images of the surroundings of the vehicle.

The memory 50 stores reference position information and real-time position information for each of the plurality of view cameras 10L, 10B, 10R, and 10F. 5A and 5B, reference position information and real-time position information of each of the plurality of surrounding view cameras 10L, 10B, 10R, and 10F are displayed on a three-dimensional The coordinate values x, y, z in the Cartesian coordinate system and the values of pan, tilt, and rotate of the camera as elements, x, y, z, pan, tilt, rotate). Here, the pan means the rotation angle of the camera around the z axis, the tilt means the rotation angle of the camera about the x axis, and the rotation means the camera rotation angle about the y axis do.

The tire pressure monitoring unit 20 includes a front left tire pressure change measuring unit 21, a front right tire pressure measuring unit 22, a rear left tire pressure change measuring unit 23 and a rear right tire pressure change amount measuring unit 24 2FL, 2RL, ...) of each of the vehicle's wheels (2FR, 2FL, 2RL, ...) so that at least one of the vehicle's running, stopped, parked state after parked, And transmits the measured pressure to the camera real-time position calculating unit 30. The camera real- The tire pressure monitoring unit 20 measures the pressure of a tire using a radio frequency identification system (RFID) sensor mounted on each tire of the vehicle's wheels 2FR, 2FL, 2RL, Pressure information can be provided in real time.

The camera real time position calculation unit 30 calculates reference position information for each of the plurality of surrounding view cameras 10L, 10B, 10R, and 10F according to the tire real time pressure information transmitted from the tire pressure monitoring unit 20 And calculates real-time position information. Also, although not shown in the figure, the camera real-time position calculation unit 30 may receive tire pressure monitoring information from the tire pressure monitoring unit 20 through the wireless interface unit.

Referring to FIG. 4B, the camera real-time position calculation unit 30 includes a height change detection unit 31 and a camera coordinate change unit 32 in detail. The height change recognition unit 31 estimates the height change of each of the tires and the height change of the shock observer that supports the vehicle body from the real time pressure information of each tire, And recognizes the height change value of the vehicle body connected to each of them.

The camera position coordinate changing unit 32 changes the reference coordinate of each of the plurality of the surrounding cameras 10L, 10B, 10R, and 10F stored in the memory 50 using the trigonometric function according to the detected height change value of the vehicle body The coordinates are calculated. Specifically, the position change amount calculating unit 32a calculates the position change amount of each of the plurality of the surrounding view cameras 10L, 10B, 10R, and 10F obtained by using the trigonometric function from the changed vehicle height recognized by the height change recognizing unit 310 And calculates the position change amounts (DELTA x, DELTA y, DELTA z, DELTA pan, DELTA tilt, DELTA rotate). The change coordinate calculator 32b stores the calculated positional variation amounts DELTA x, DELTA y, DELTA zan, DELTA pan, DELTA tilt and DELTA rotate in a manner that the plurality of the surrounding view cameras 10L, Y ', z', pan ', tilt', rotate ') in addition to the respective elements of the reference position coordinates (x, y, z, pan, tilt, rotate) . That is, the changed real time position coordinates (x ', y', z ', pan', tilt ', rotate') are (x + Δx, y + Δy, z + Δz, pan + Δpan, tilt + tilt, rotate +? rotate).

The surround view generation unit 40 converts each of the images photographed by the plurality of the surrounding view cameras 10L, 10B, 10R, and 10F into three-dimensional information based on the current location information of the corresponding camera, And generates the surround view image.

Referring to FIG. 4C, the surround view image generator 40 includes an image pixel coordinate converter 41 and an image calibrator 42.

The image pixel coordinate conversion unit 41 converts each pixel constituting each of the images into three-dimensional coordinates using real-time position information of the corresponding camera. For example, a pixel of each image can be converted into a coordinate value having a shape of (x, y, z, pan, tilt, rotate) based on the real-time position information roll of the camera.

The image calibrating unit 42 calibrates the three-dimensional coordinates of the three-dimensional coordinate-converted images of each pixel information so that one image frame is formed. That is, the image calibrating unit 42 may synthesize a front image, a rear image, a left image, and a right image to generate a three-dimensional image expressing 360 degrees based on the front surface of the vehicle about the vehicle periphery. Then, even if the position of the surround camera is changed, a 3-dimensional surround view image in which the cube in an unmatched state is calibrated as shown in FIG. 3B can be generated as shown in FIG. 3C.

The generated three-dimensional surround view image may be displayed on a display means mounted on a vehicle, stored in a memory, or transmitted to the outside through a communication unit. The display means (not shown) may display the surround view image generated by the surround view generating unit 40. On the other hand, it is also possible to provide various user user interfaces when displaying the surround view image, or to provide a touch sensor capable of touch input to the provided user interface. On the other hand, the display means may include a cluster or HUD (Head Up Display) on the inside of the vehicle interior. On the other hand, when the display 60 is the HUD, it may include a projection module for projecting an image on the windshield of the vehicle 1. [

FIG. 6A is a flowchart illustrating a method of calibrating an overview map for a vehicle using tire pressure monitoring information according to an exemplary embodiment of the present invention. FIG. 6B is a flow chart for explaining a camera position adjusting step according to an embodiment of the present invention And FIG. 6C is a flowchart for explaining an overview view generation step according to an embodiment of the present invention.

Referring to FIG. 6A, a vehicle averaging calibration apparatus (hereinafter referred to as "apparatus") using tire pressure monitoring information stores reference position information of each of a plurality of surrounding view cameras 10L, 10B, 10R, and 10F (S110). 5A and 5B, the reference position information of each of the plurality of the surrounding view cameras 10L, 10B, 10R, and 10F is obtained from three-dimensional coordinates in a three- (X, y, z) composed of six coordinate elements by including the values of the coordinate elements x, y, z and the pan, tilt, z, pan, tilt, rotate).

The apparatus monitors the change of each of the wheels 2FR, 2FL, 2RL, .. of the vehicle, and obtains the pressure and temperature of the tire during operation of the vehicle (S120). That is, the tire pressure monitoring unit 20 measures the temperature and pressure of the tire using an RFID (radio frequency identification system) sensor mounted on each tire of the vehicle's wheels 2FR, 2FL, 2RL, Provides pressure and temperature information of the tire of the vehicle in real time.

The device adjusts the positional information for each of the plurality of surrounding view cameras 10L, 10B, 10R, and 10F according to tire-by-tire real-time pressure information (S130).

Referring to FIG. 6B, the apparatus estimates the height change of each of the tires and the height change of the shock observer that supports the vehicle body from the real-time pressure information of each tire to obtain the vehicle's wheels 2FR, 2FL, 2RL, And recognizes the height change value of the vehicle body connected to each of them (S131). Then, the apparatus calculates the changed coordinates from the reference coordinates of each of the plurality of surrounding view cameras 10L, 10B, 10R, and 10F stored in the memory 50 using the trigonometric function according to the recognized height change value of the vehicle body (S132 to S133). In detail, the apparatus calculates the positional variation amounts DELTA x, DELTA y, DELTA z, DELTA pan, DELTA z, DELTA z, DELTA z, DELTA z, DELTA z, DELTA tilt, DELTA rotate) (S132). The apparatus then calculates the positional variation amounts (x, y, z, pan, tilt, and rotate) calculated at the reference position coordinates (x (x ', y', z ', pan', tilt ', rotate') is added to each of the elements of x, y, z, pan, tilt and rotate at step S133. That is, the changed position coordinates (x ', y', z ', pan', tilt ', rotate') are (x + Δx, y + Δy, z + Δz, pan + Δpan, tilt + Δtilt , rotate +? rotate).

Referring again to FIG. 6A, the apparatus three-dimensionally converts each of the images photographed by the plurality of the surrounding view cameras 10L, 10B, 10R, and 10F based on the current position information of the corresponding camera, To generate an overview image (S140). That is, referring to FIG. 6C, the apparatus converts each of pixels constituting each of the images into three-dimensional coordinates using the position information of the corresponding camera (S141). Then, the apparatus calibrates the three-dimensional coordinates of each pixel information by three-dimensionally synthesizing (Stitching) the images so that one image frame is formed (S142). That is, the front image, the rear image, the left image, and the right image may be synthesized to generate a three-dimensional image representing 360 degrees based on the front surface of the vehicle about the surroundings of the vehicle.

Claims (5)

A plurality of surrounding view cameras mounted on the vehicle,
A memory for storing reference position information for each of the plurality of view cameras;
A tire pressure monitoring unit for providing real time pressure information per tire,
A camera real-time position calculator for calculating real-time position information by adjusting the reference position information for each of the plurality of surround-view cameras using the real-time pressure information for each tire;
Dimensional image into three-dimensionally transformed images based on real-time position information of the camera, and combines the three-dimensional converted images to generate an averaged view image And a surrounding view generating unit,
The camera real-
Estimating a height change of a tire and a height change of a shock observer for supporting a vehicle body based on the real time pressure information of each tire, recognizing a height change value of the vehicle body connected to each of the wheels, And calculates the changed real time position coordinates from the reference position coordinates of each of the plurality of view cameras stored in the memory by using the trigonometric function. The method of claim 1, wherein the position information of each of the plurality of view cameras
(X, y, z, pan, y, z) including the coordinate elements x, y, z of the three-dimensional Cartesian coordinate system and the values of the pan, tilt, tilt, and rotate) of the vehicle. delete The apparatus according to claim 1, wherein the camera coordinate changing unit
Δx, Δy, Δz, Δpan, Δtilt, and Δ rotate of each of the plurality of view cameras obtained by using the trigonometric function from the changed body height recognized by the height change recognition unit A coordinate change amount calculating unit for calculating,
(X, y, z, pan, tilt, and rotate) of the plurality of averaging cameras stored in the memory are calculated based on the positional variation amounts (x, y, z, (X ', y', z ', pan', tilt ', rotate') in addition to each of the elements of the tire pressure monitoring information Around view calibration device. 2. The apparatus of claim 1, wherein the surround view generator
An image pixel coordinate converter for converting each of pixels constituting each of the images into three-dimensional coordinates using real-time position information of the corresponding camera;
And an image calibrating unit for calibrating the image so that one image frame is formed by three-dimensionally synthesizing images using the three-dimensional coordinate-converted pixel information. Around view calibration device for vehicles.

Images