KR20200070135A - Around view monitor system for rear open trucks - Google Patents

Abstract

According to an embodiment of the present invention, an around-view monitor (AVM) system for a rear-open truck has a loading compartment rear cover to be opened, and comprises: a plurality of main body cameras dispersed and arranged on a main body of a truck to acquire and provide a front image, a left image, and a right image; at least one cover inside camera arranged inside the cover of the truck to acquire and provide a first rear image; at least one cover outside camera arranged on the outside of the cover of the truck to acquire and provide a second rear image; a camera calibration unit to repeatedly perform camera calibration in a cover-opened state and a cover-closed state to acquire and store a first image conversion model and a second image conversion model; and an AVM image generation unit to use the first image conversion model in a cover-opened state to convert camera images of the main body cameras and the cover inside camera into a first AVM image, and use the second image conversion model in a cover-closed state to convert camera images of the main body cameras and the cover outside camera into a second AVM image.

Description

Around view monitor system for rear open trucks

The present application relates to an around view monitor system for a rear open truck in which the rear cover of the stacker is implemented in an openable form.

An Around View Monitor (AVM) system is a state in which a driver attached to a vehicle displays an image of a vehicle surrounding a vehicle on a monitor in the vehicle, and the driver recognizes the situation around the vehicle through the monitor image. A system that allows you to drive.

In general, the Around View Monitor System displays the front view of the vehicle along with the front view of the vehicle through the monitor, so that the driver can recognize not only the front of the vehicle but also the left, right, and rear situations. To make.

However, when the around view monitor system is applied to a rear open type truck in which the rear cover of a loading box is implemented in an openable form, such as an environmentally friendly vehicle, the operation of the rear open type truck, for example, the opening and closing of the rear cover of the rear open type truck It is difficult to properly acquire the rear image.

Accordingly, in the related art, a method for acquiring and providing an AVM image normally regardless of whether a rear open truck is opened or not is required in an around view monitor system for a rear open truck.

In order to solve the above problems, an embodiment of the present invention provides an around view monitor system for a rear open truck having a stackable rear cover openable.

The around view monitor system for the rear open truck includes: a plurality of body cameras distributed and disposed on the body of the truck to obtain and provide a front image, a left image, and a right image; At least one cover inner camera that is disposed inside the cover of the truck to acquire and provide a first rear image; At least one outer cover camera disposed outside the cover of the truck to acquire and provide a second rear image; A camera calibration unit repeatedly acquiring and storing a first image conversion model and a second image conversion model by repeatedly performing camera calibration in a lid open state and a lid closed state; And in the lid open state, converting the camera images of the main body cameras and the lid inner camera to a first AVM (Around View Monitor) image using the first image conversion model, and in the lid closed state, the second image conversion model. And an AVM image generating unit that converts camera images of the main body cameras and the outer camera of the cover into a second AVM image.

The camera calibration unit repeatedly photographs the calibration plate disposed around the truck while changing the cover state, and then compares and analyzes the calibration plate coordinates extracted from the camera image for each camera image with the calibration plate coordinates calculated based on the mathematical model. After determining the parameters of and collecting and collecting the mathematical model according to the lid state, a first image transform model corresponding to the lid open state and a second image transform model corresponding to the lid closed state are generated.

The mathematical model is a process of converting feature point coordinates defined based on a calibration plate coordinate system to a reference coordinate system, a process of converting feature point coordinates defined as a reference coordinate system reference to a camera coordinate system, and sensor point coordinates defined as a camera coordinate system reference It is characterized in that it is an image conversion model that sequentially performs the process of converting to a reference point and transforming the feature point coordinates defined by the sensor coordinate system reference into radiation coordinates, and then converting the radiation point transformed feature point coordinates into image coordinates.

The around view monitor system further includes the image error correction unit for additionally correcting any one of the first image conversion model and the second image conversion model so that image errors between the first AVM image and the second AVM image are minimized. It is characterized by.

In addition, the solution means of the above-mentioned subject does not list all the characteristics of this invention. Various features of the present invention and advantages and effects thereof may be understood in more detail with reference to specific embodiments below.

According to an embodiment of the present invention, the camera is additionally installed not only on the outside but also on the inside of the cover located at the rear of the rear open truck, and can selectively display a rear image obtained by the outside or inside camera according to the opening and closing of the cover. have. Accordingly, even when the cover of the rear open type truck is opened, the rear image can be normally obtained and provided.

In addition, regardless of whether or not the lid of the stacker is opened, it is possible to acquire and provide an AVM image that can maintain a constant position and angle at all times.

1 is a block diagram of an around view monitor system for a rear open truck according to an embodiment of the present invention.
2 is a view showing a camera layout according to an embodiment of the present invention.
3 to 6 are diagrams for explaining a camera calibration method according to an embodiment of the present invention.
7 is a view for explaining a method of generating an AVM image according to an embodiment of the present invention.

Hereinafter, preferred embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily implement the present invention. However, in the detailed description of a preferred embodiment of the present invention, when it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and functions.

In addition, throughout the specification, when a part is said to be'connected' to another part, it is not only'directly connected', but also'indirectly connected' with another element in between. Includes. In addition, "including" a component means that other components may be further included instead of excluding other components, unless otherwise stated.

1 is a block diagram of an around view monitor system for a rear open truck according to an embodiment of the present invention.

Referring to FIG. 1, an around view monitor system for a rear open truck according to an embodiment of the present invention includes a camera unit 100, a camera calibration unit 200, an AVM image generation unit 300, and an image error correction unit ( 400), a monitor 500, and the like.

In addition to the plurality of body cameras 110 distributed and installed in the body of the vehicle, the camera unit 100 includes at least one cover inside camera 120 disposed inside the truck cover and at least one cover outside disposed on the truck cover. The camera 130 may be further included.

The plurality of main body cameras 110 acquire and provide a plurality of surrounding images by photographing the vehicle surroundings, that is, the front, rear, left, and right sides, respectively, and the cover inner camera 120 displays a rear image when the cover is open Acquisition and provision, and the outer cover camera 130 acquires and provides a rear image when the lid is closed.

For reference, in the case of a rear open-type truck such as a garbage vehicle, the rear image is important, and in particular, the edge region is more important than the center region at the rear of the garbage vehicle when considering the operation of the cover located at the rear of the garbage vehicle and the position of the operator. . In addition, when the cover located at the rear of the garbage vehicle is opened, the position of the camera installed at the rear is changed, which may cause a problem that the rear image cannot be obtained properly.

Therefore, in the present invention, in order to obtain an image of the rear of the truck regardless of whether the cover is opened or not, cameras are installed on both the outside and the inside of the cover located at the rear of the garbage vehicle.

Whether the camera calibration unit 200 has the center of gravity of the vehicle may be changed depending on whether the lid is opened or not, and even if the camera installed at the same position has an external parameter of the camera variable depending on whether the lid is opened, whether the lid is opened or not. Try to do 2 camera calibrations with different.

That is, a camera calibration is performed for the main body cameras 110 and the cover inner camera 120 in the cover open state to obtain and store the first image conversion model, and the body cameras 110 and the cover outside in the cover closed state Camera calibration is performed on the camera 130 to obtain and store a second image conversion model. At this time, whether the cover is opened may be detected using a vehicle ECU or a vehicle sensor.

The AVM image generating unit 300 generates an AVM image while different the type of the camera used and the type of the image conversion model depending on whether the cover is opened or not.

That is, in the cover open state, the camera images obtained by the main body cameras 110 and the cover inner camera 120 are received, and each of these coordinate systems is converted according to the first image conversion model to unify the coordinate system, and then one image is obtained. Is synthesized to generate a first AVM image.

In the closed state, the camera images obtained through the main body cameras 110 and the outer cover camera 130 are received, and each of these coordinate systems is converted according to the second image conversion model to unify the coordinate systems, and then as a single image. Synthesis to generate a second AVM image.

In addition, the present invention is further provided with an image error correction unit 400, so that the image error of the first AVM image and the second AVM image is minimized, any one of the parameters of the first image conversion model and the second image conversion model Make corrections possible.

That is, even if the center of gravity of the vehicle changes depending on whether the cover is opened or not, the position and angle of the first AVM image and the second AVM image can be maintained at a constant value even if the position of the camera changes little by little.

2 is a view showing a camera layout according to an embodiment of the present invention.

As shown in FIG. 2, the main body cameras 110 are disposed on the front, left, and right sides of the vehicle to acquire and provide a front image, a left image, and a right image.

The cover inner camera 120 is installed on the inside of the vehicle rear cover or is disposed on the rear surface of the vehicle that is selectively exposed only when the vehicle rear cover is opened, thereby selectively obtaining and providing a rear image only when the vehicle cover is opened. do.

The outer cover camera 130 is disposed outside the rear cover of the vehicle to selectively acquire and provide a rear image only when the vehicle cover is closed.

3 is a view for explaining a camera calibration method according to an embodiment of the present invention.

As shown in Figure 3, the camera calibration method of the present invention is largely a calibration plate arrangement and coordinate system definition step (S1), camera shooting step (S2), feature point calculation step through a mathematical model (S3), feature points through image processing It consists of an extraction step (S4), a parameter estimation step (S5) of the mathematical model through the optimal junction, and an image conversion model determination step (S6).

In step S1, as shown in FIG. 4, after distributing a plurality of calibration plates around the vehicle of the rear open truck (eg, front left, front right, rear left, rear right), in a mathematical model for calculating feature points Define the coordinate systems to be used.

That is, a camera coordinate system corresponding to the camera body, a calibration plate coordinate system corresponding to the calibration plate, and a reference coordinate system corresponding to the vehicle are defined. Also, although not shown in FIG. 4, a camera sensor coordinate system corresponding to a camera sensor that is built in the camera body and performs an actual shooting operation is additionally defined.

In step S2, after opening or closing the vehicle cover of the rear open vehicle, images of the vehicle front, rear, left, and right are respectively obtained.

If the cover is open, the vehicle front, rear, left, and right images are respectively obtained using the body cameras 110 and the cover inner camera 120, and if the cover is closed, the body cameras 110 and the cover Each of the front, rear, left, and right images of the vehicle may be acquired using the outside camera 130.

In step S3, a process of converting each feature point of the calibration plate into an image coordinate is repeatedly performed using a mathematical model.

In the mathematical model of the present invention, as shown in FIG. 5, the process of firstly converting feature point coordinates defined on the basis of the calibration plate coordinate system to the reference coordinate system (STEP1), and the feature point coordinates defined on the basis of the reference coordinate system are based on the camera coordinate system. Conversion process (STEP2), the third step of converting the feature point coordinates defined based on the camera coordinate system based on the sensor coordinate system (STEP3), and finally converting the feature point coordinates defined based on the sensor coordinate system to radiation distortion, and then transforming the radiation distortion This is an image conversion model that sequentially performs the process of converting the feature point coordinates into image coordinates (STEP4).

The feature point coordinates and the reference coordinate system are defined by Equations 1 and 2, and the feature point coordinates defined by the reference coordinate system are converted into coordinates based on the camera coordinate system according to Equation 3.

[Equation 1]

At this time, c is cos, s is sin, c or s 1 of the subscripts is the angle rotated by the z axis, 2 of the c or s subscripts is the angle rotated by the x axis, 3 of the c or s subscripts is the z axis Rotation angle, x is the x-axis coordinate of the calibration plate coordinate system, y is the y-axis coordinate, and z is the z-axis coordinate.

[Equation 2]

[Equation 3]

The feature point coordinates defined as the camera coordinate system reference are converted to the sensor coordinate system reference according to Equation (4).

[Equation 4]

At this time, f is the focal length of the camera sensor.

The feature point coordinates defined based on the sensor coordinate system are radiation distortion transformed according to Equation 5, and the feature point coordinates transformed by radiation distortion are converted into image coordinates according to Equation 6 again.

[Equation 5]

At this time, Ru is the distance from the center of the radiation distortion to the feature point in the image without radiation distortion, and Rd is the distance from the center of the radiation distortion to the feature point in the image with radiation distortion.

[Equation 6]

At this time, u0 is the x-axis radiation distortion center, v0 is the y-axis radiation distortion center, kx is the coefficient to convert from the x-axis sensor coordinates to image coordinates, and ky is the coefficient to convert from the y-axis sensor coordinates to image coordinates.

In step S4, the calibration plate detection and feature point extraction operations are repeatedly performed for each of the images obtained through step S2 to extract coordinates of the feature plate corresponding to each image.

In step S5, for each image, as shown in FIG. 6, the parameters of the mathematical model for minimizing the error between coordinates are estimated by comparing and analyzing the transformed coordinates of step S3 and the extracted coordinates of step S4 using a nonlinear least squares method. That is, four mathematical models corresponding to each of the front, rear, left, and right images when the lid is open and four mathematical models corresponding to each of the front, rear, left, and right images when the lid is closed are finalized. To create.

In step S6, a first image transformation model including four mathematical models corresponding to each of the front, rear, left, and right images when the lid is open is generated, and the front, rear, and left when the lid is closed, A second image transformation model including four mathematical models corresponding to each of the right images is generated.

However, as described above, depending on whether the cover is opened or not, the vehicle tilts according to the change in the center of gravity. In this case, the internal parameters of the camera and the relative posture of the installed camera are maintained, but the external parameters of the camera are changed. Accordingly, an image error in which the position and angle are different also occurs between the first AVM image and the second AVM image.

Accordingly, in order to minimize image errors between the first AVM image and the second AVM image, the posture of one of the reference coordinate systems corresponding to each of the first image transformation model and the second image transformation model is adjusted and reflected to reflect the image. Allows additional adjustment of the parameters of the transformation model.

Therefore, the present invention, as shown in Figure 7, in the open state, the body using the images obtained by the main body cameras 110 and the cover inner camera 120 is converted according to the first image conversion model and then synthesized to the first AVM Acquire and provide an image, and in the closed state, obtain and provide a second AVM image by converting and synthesizing images obtained through the main body cameras 110 and the outer cover camera 130 according to the second image conversion model. do.

In this case, the first AVM image and the second AVM image may be images having the same location and angle. That is, through the image error correction process, it is possible to prevent the occurrence of an image mismatch according to whether the cover is opened or not.

The present invention is not limited by the above-described embodiments and accompanying drawings. For those skilled in the art to which the present invention pertains, it will be apparent that components according to the present invention can be substituted, modified and changed without departing from the technical spirit of the present invention.

100: camera
200: camera calibration unit
300: AVM image generating unit
400: image error correction unit
500: monitor

Claims (4)

Around view monitor system for a rear open truck in which the rear cover of the loading box is implemented in an openable form,
A plurality of body cameras that are distributedly disposed on the body of the truck to obtain and provide a front image, a left image, and a right image;
At least one cover inner camera disposed inside the cover of the truck to acquire and provide a first rear image;
At least one outer cover camera disposed outside the cover of the truck to acquire and provide a second rear image;
A camera calibration unit repeatedly acquiring and storing a first image conversion model and a second image conversion model by repeatedly performing camera calibration in a lid open state and a lid closed state; And
In the lid open state, the camera images of the main body cameras and the lid inner camera are converted into a first AVM (Around View Monitor) image using the first image conversion model, and in the lid closed state, the second image conversion model is converted. Around view monitor system for a rear open truck including an AVM image generating unit for converting camera images of the main body cameras and the camera outside the cover into a second AVM image. The method of claim 1, wherein the camera calibration unit
After repeatedly photographing the calibration plate disposed around the truck while changing the cover state, the parameters of the mathematical model are determined by comparing and analyzing the calibration plate coordinates extracted from the camera image for each camera image with the calibration plate coordinates calculated based on the mathematical model. Then, by collecting and collecting the mathematical model according to the cover state, for generating a first image conversion model corresponding to the cover open state and the second image conversion model corresponding to the cover closed state for a rear open truck Around view monitor system. The method of claim 2, wherein the mathematical model
The process of converting the feature point coordinates defined by the calibration board coordinate system reference to the reference coordinate system, the process of converting the feature point coordinates defined by the reference coordinate system to the camera coordinate system basis, and converting the feature point coordinates defined by the camera coordinate system reference to the sensor coordinate system reference Around view for a rear open truck, characterized in that it is an image conversion model that sequentially performs the process of converting radiation distortion transformed feature point coordinates into image coordinates after radiation distortion transformation of feature point coordinates defined based on sensor coordinate system. Monitor system. According to claim 1,
And further including the image error correcting unit for additionally correcting any one of the first image conversion model and the second image conversion model so that image errors between the first AVM image and the second AVM image are minimized. Around view monitor system for trucks.

Images