KR102270793B1 - Around view monitor system for rear open trucks - Google Patents

Abstract

The around-view monitor system for a rear-opening truck in which the loading box rear cover is openable according to an embodiment of the present invention is distributedly disposed on the body of the truck to obtain and provide a front image, a left image, and a right image a plurality of body cameras; at least one cover inside camera disposed inside the cover of the truck to obtain and provide a first rear image; at least one or more outside of the cover camera disposed on the outside of the cover of the truck to obtain and provide a second rear image; a camera calibration unit for repeatedly performing camera calibration in the cover open state and the cover closed state to obtain and store the first image conversion model and the second image conversion model; And in the cover open state, convert the camera images of the main body cameras and the cover inner camera by using the first image conversion model into a first AVM (Around View Monitor) image, and in the cover closed state, the second image conversion model and an AVM image generator converting camera images of the main body cameras and the camera outside the cover into a second AVM image using the .

Description

Around view monitor system for rear open trucks

The present application relates to an around-view monitor system for a rear-opening truck in which the rear cover of the loading box is implemented in an openable form.

The Around View Monitor (AVM) system displays the images around the vehicle captured by the camera attached to the vehicle on the monitor inside the vehicle, allowing the driver to recognize the situation around the vehicle through the monitor image. A system that allows you to drive.

In general, the around-view monitor system displays the top view of the vehicle along with the front image of the vehicle through the monitor, so that the driver can recognize not only the front of the vehicle, but also the left, right, and rear conditions. let there be

However, if the Around View Monitor System is applied to an open rear truck that has a rear cover that can be opened, such as an environmentally friendly vehicle, the operation of the rear open truck, for example, as the rear cover of the open rear truck is opened and closed. It is difficult to properly acquire an image of the rear.

Accordingly, there is a need in the art for a method for normally acquiring and providing an AVM image regardless of whether or not the cover of the open-rear truck is opened in an around-view monitor system for an open-rear truck.

In order to solve the above problems, an embodiment of the present invention provides an around-view monitor system for a rear-opening truck in which the rear cover of the loading box is implemented in an openable form.

The around-view monitor system for the rear open truck includes: a plurality of main body cameras that are distributedly disposed on the body of the truck to acquire and provide a front image, a left image, and a right image; at least one cover inside camera disposed on the inside of the cover of the truck to obtain and provide a first rear image; at least one or more outside of the cover camera disposed on the outside of the cover of the truck to obtain and provide a second rear image; a camera calibration unit for repeatedly performing camera calibration in the cover open state and the cover closed state to obtain and store the first image conversion model and the second image conversion model; And in the cover open state, convert the camera images of the main body cameras and the cover inner camera using the first image conversion model to a first AVM (Around View Monitor) image, and in the cover closed state, the second image conversion model and an AVM image generating unit converting the camera images of the main body cameras and the camera outside the cover into a second AVM image using

After the camera calibration unit repeatedly shoots the calibration plate disposed around the truck while changing the cover state, the calibration plate coordinates extracted from the camera image for each camera image are compared and analyzed with the calibration plate coordinates calculated based on the mathematical model, and a mathematical model After determining the parameters of , by collecting and combining the mathematical models according to the lid state, a first image transformation model corresponding to the lid open state and a second image transformation model corresponding to the lid closed state are generated.

The mathematical model is the process of converting the feature point coordinates defined based on the calibration plate coordinate system to the reference coordinate system standard, the process of converting the feature point coordinates defined by the reference coordinate system standard to the camera coordinate system standard, the feature point coordinates defined by the camera coordinate system standard to the sensor coordinate system It is characterized in that it is an image transformation model that sequentially performs a process of converting to a reference and a process of radially distorting the feature point coordinates defined by the sensor coordinate system, and then converting the radial distortion-converted feature point coordinates into image coordinates.

The around-view monitor system further comprises the image error correction unit for further correcting any one of the first image conversion model and the second image conversion model so that the image error between the first AVM image and the second AVM image is minimized characterized in that

Incidentally, the means for solving the above problems do not enumerate all the features of the present invention. Various features of the present invention and its advantages and effects may be understood in more detail with reference to the following specific embodiments.

According to an embodiment of the present invention, a camera is additionally installed on the inside as well as the outside of the cover located on the rear side of the open-rear truck, and the rear image acquired by the outside or inside camera can be selectively displayed according to the opening and closing of the cover. have. Accordingly, even when the cover of the open-rear truck is opened, the image of the rear can be normally obtained and provided.

In addition, it is possible to acquire and provide AVM images that can always maintain a constant position and angle regardless of whether the loading box cover is opened or not.

1 is a block diagram of an around-view monitor system for a rear-opening truck according to an embodiment of the present invention.
2 is a view showing a camera arrangement according to an embodiment of the present invention.
3 to 6 are diagrams for explaining a camera calibration method according to an embodiment of the present invention.
7 is a diagram for explaining an AVM image generating method according to an embodiment of the present invention.

Hereinafter, preferred embodiments will be described in detail so that those of ordinary skill in the art can easily practice the present invention with reference to the accompanying drawings. However, in describing a preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and functions.

In addition, throughout the specification, when a part is 'connected' with another part, it is not only 'directly connected' but also 'indirectly connected' with another element interposed therebetween. include In addition, 'including' a certain component means that other components may be further included, rather than excluding other components, unless otherwise stated.

1 is a block diagram of an around-view monitor system for a rear open truck according to an embodiment of the present invention.

Referring to FIG. 1 , the around-view monitor system for a rear open truck according to an embodiment of the present invention includes a camera unit 100 , a camera calibration unit 200 , an AVM image generation unit 300 , and an image error correction unit ( 400 ), and a monitor 500 , and the like.

The camera unit 100 includes at least one cover inner camera 120 disposed inside the cover of the truck in addition to the plurality of body cameras 110 distributed and installed in the body of the vehicle, and at least one outside cover disposed outside the cover of the truck. A camera 130 may be further included.

A plurality of main body cameras 110 obtain and provide a plurality of peripheral images by photographing the vehicle surroundings, that is, the front, rear, left and right, respectively, and the cover inner camera 120 provides a rear image when the cover is open. Acquiring and providing, the cover outside camera 130 acquires and provides a rear image when the cover is closed.

For reference, in the case of an open rear truck such as a garbage vehicle, the image from the rear is important. In particular, considering the operation of the cover located on the rear of the garbage vehicle and the position of the operator, the edge area is more important than the center area at the rear of the garbage vehicle . In addition, when the cover located at the rear of the garbage vehicle is opened, the position of the camera installed at the rear is changed, so that a problem in which an image of the rear cannot be properly acquired may occur.

Accordingly, in the present invention, in order to acquire an image of the rear of the truck regardless of whether the cover is opened, cameras are installed on both the outside and inside of the cover located at the rear of the garbage vehicle, respectively.

The camera calibrator 200 considers that the center of gravity of the vehicle may be changed depending on whether the cover is opened, and that the camera external parameter may vary depending on whether the cover is opened even for a camera installed at the same location. , while performing two camera calibrations.

That is, in the cover open state, the camera calibration is performed for the main body cameras 110 and the cover inner camera 120 to obtain and store the first image conversion model, and the main body cameras 110 and the cover outer side in the cover closed state A second image conversion model is acquired and stored by performing camera calibration on the camera 130 . In this case, whether the cover is open may be detected using a vehicle ECU or a vehicle sensor.

The AVM image generating unit 300 generates an AVM image while changing the type of the used camera and the type of the image conversion model according to whether the cover is opened.

That is, in the cover open state, the camera images obtained by the body cameras 110 and the cover inner camera 120 are received, and each of these coordinate systems is converted according to the first image conversion model to unify the coordinate system and then one image. to create a first AVM image.

And in the cover closed state, the camera images obtained through the body cameras 110 and the camera outside the cover 130 are received, and each of these coordinate systems is converted according to the second image conversion model to unify the coordinate system and then into one image. A second AVM image is generated by synthesizing.

In addition, the present invention further comprises an image error correcting unit 400, so that the image error of the first AVM image and the second AVM image is minimized, any one parameter of the first image conversion model and the second image conversion model to be able to correct it.

That is, the center of gravity of the vehicle changes depending on whether the cover is opened, and thus the positions and angles of the first AVM image and the second AVM image can be maintained at a constant value even if the position of the camera is slightly changed accordingly.

2 is a view showing a camera arrangement according to an embodiment of the present invention.

As shown in FIG. 2 , the main body cameras 110 are disposed on the front, left, and right sides of the vehicle to acquire and provide a front image, a left image, and a right image.

The cover inner camera 120 is installed on the inside of the vehicle rear cover or disposed on the vehicle rear surface selectively exposed only when the vehicle rear cover is opened, and selectively acquires and provides a rear image only when the vehicle cover is opened do.

The cover outside camera 130 is disposed on the outside of the vehicle rear cover, and selectively acquires and provides a rear image only when the vehicle cover is closed.

3 is a view for explaining a camera calibration method according to an embodiment of the present invention.

As shown in Figure 3, the camera calibration method of the present invention is largely a calibration plate arrangement and coordinate system definition step (S1), camera photographing step (S2), a feature point calculation step through a mathematical model (S3), a feature point through image processing It consists of an extraction step (S4), a parameter estimation step (S5) of a mathematical model through best fit, and a step (S6) determining an image transformation model.

In step S1, as in FIG. 4, after distributing a plurality of calibration plates around the vehicle (eg, front left, front right, rear left, rear right) of the open rear truck, in the mathematical model for calculating the feature points Defines the coordinate systems to be used.

That is, the camera coordinate system corresponding to the camera body, the calibration plate coordinate system corresponding to the calibration plate, and the reference coordinate system corresponding to the vehicle are defined. In addition, although not shown in FIG. 4 , a camera sensor coordinate system corresponding to a camera sensor that is embedded in the camera body and performs an actual photographing operation is additionally defined.

In step S2, after opening or closing the vehicle cover of the rear open-type vehicle, each of the vehicle front, rear, left, and right images is acquired.

If the cover is open, the body cameras 110 and the cover inside camera 120 are used to obtain front, rear, left, and right images of the vehicle respectively, and if the cover is closed, the body cameras 110 and the cover Each of the front, rear, left, and right images of the vehicle may be acquired using the outside camera 130 .

In step S3, the process of converting the feature points of each calibration plate into image coordinates is repeatedly performed using a mathematical model.

As shown in FIG. 5, the mathematical model of the present invention is the process of first transforming the feature point coordinates defined by the calibration plate coordinate system to the reference coordinate system reference (STEP1), and the feature point coordinates defined by the reference coordinate system are secondarily based on the camera coordinate system Transformation process (STEP2), the process of tertiary transformation of the feature point coordinates defined based on the camera coordinate system based on the sensor coordinate system (STEP3), and finally, after radial distortion transformation of the feature point coordinates defined based on the sensor coordinate system It is an image transformation model that sequentially performs a process (STEP4) of quaternary transformation of feature point coordinates into image coordinates.

The key point coordinates and the reference coordinate system are defined by Equations 1 and 2, and the key point coordinates defined based on the reference coordinate system are converted into coordinates based on the camera coordinate system according to Equation 3.

[Equation 1]

In this case, c is cos, s is sin, 1 of c or s subscripts is an angle rotated along the z-axis, 2 of c or s subscripts are rotated in the x-axis, 3 of c or s subscripts are z-axis The angle rotated by , x is the x-axis coordinate of the calibration plate coordinate system, y is the y-axis coordinate, and z is the z-axis coordinate.

[Equation 2]

[Equation 3]

The feature point coordinates defined based on the camera coordinate system are converted to the sensor coordinate system based on Equation (4).

[Equation 4]

Here, f is the focal length of the camera sensor.

The characteristic point coordinates defined based on the sensor coordinate system are radially distorted according to Equation 5, and the radially distorted characteristic point coordinates are converted into image coordinates according to Equation 6 again.

[Equation 5]

In this case, Ru is the distance from the radial distortion center to the feature point in the image without radial distortion, and Rd is the distance from the radial distortion center to the feature point in the image with radial distortion.

[Equation 6]

In this case, u0 is the x-axis radial distortion center, v0 is the y-axis radial distortion center, kx is a coefficient for converting the x-axis sensor coordinates into image coordinates, and ky is a coefficient for converting the y-axis sensor coordinates into image coordinates.

In step S4, the calibration plate detection and feature point extraction operations are repeatedly performed for each of the images obtained through step S2, thereby extracting the calibration plate feature point coordinates corresponding to each image.

In step S5, as shown in FIG. 6, for each image, by using the nonlinear least-squares method to compare and analyze the transform coordinates of step S3 and the extracted coordinates of step S4, parameters of a mathematical model that minimizes the error between coordinates are estimated. That is, four mathematical models corresponding to each of the front, rear, left, and right images when the cover is open and four mathematical models corresponding to each of the front, rear, left, and right images when the cover is closed are final to create

In step S6, a first image transformation model including four mathematical models corresponding to each of the front, rear, left, and right images when the cover is open is generated, and when the cover is closed, the front, rear, left, A second image transformation model including four mathematical models corresponding to each of the right images is generated.

However, as described above, vehicle inclination occurs due to a change in the center of gravity depending on whether the cover is opened. In this case, the internal parameters of the camera and the relative posture of the installed camera are maintained, but the external parameters of the camera are changed, Accordingly, an image error in which a position and an angle vary between the first AVM image and the second AVM image is generated.

In order to minimize the image error between the first AVM image and the second AVM image, the posture of any one of the reference coordinate systems corresponding to each of the first image transformation model and the second image transformation model is adjusted, and this is reflected in the corresponding image Allows additional adjustment of parameters of the transformation model.

Therefore, in the present invention, as shown in FIG. 7 , in the cover open state, the images acquired by the body cameras 110 and the cover inner camera 120 are converted according to the first image conversion model and then synthesized to the first AVM. Acquire and provide an image, and convert and synthesize the images acquired through the main body cameras 110 and the cover outer camera 130 in the cover closed state according to the second image conversion model to obtain and provide a second AVM image do.

In this case, the first AVM image and the second AVM image may be images having the same position and angle. That is, through the image error correction process, it is possible to prevent the occurrence of image discrepancy depending on whether the cover is opened or not in advance.

The present invention is not limited by the above embodiments and the accompanying drawings. For those of ordinary skill in the art to which the present invention pertains, it will be apparent that the components according to the present invention can be substituted, modified and changed without departing from the technical spirit of the present invention.

100: camera
200: camera calibration unit
300: AVM image generator
400: image error correction unit
500: monitor

Claims (4)

In the around-view monitor system for a rear-opening truck in which the rear cover of the loading box is implemented in an openable form,
A plurality of body cameras that are distributed on the main body of the truck to obtain and provide a front image, a left image, and a right image;
at least one cover inside camera disposed on the inside of the cover of the truck to obtain and provide a first rear image;
at least one or more outside of the cover camera disposed on the outside of the cover of the truck to obtain and provide a second rear image;
a camera calibration unit for repeatedly performing camera calibration in the cover open state and the cover closed state to obtain and store the first image conversion model and the second image conversion model; and
In the cover open state, the camera images of the body cameras and the cover inner camera are converted to a first AVM (Around View Monitor) image by using the first image conversion model, and in the cover closed state, the second image conversion model an AVM image generator for converting the camera images of the main body cameras and the camera outside the cover into a second AVM image using; and
An around-view monitor for an open rear truck including an image error correction unit that additionally corrects any one of the first image conversion model and the second image conversion model so that the image error between the first AVM image and the second AVM image is minimized system. According to claim 1, wherein the camera calibration unit
After repeatedly shooting the calibration plate placed around the truck while changing the cover state, the calibration plate coordinates extracted from the camera image for each camera image are compared and analyzed with the calibration plate coordinates calculated based on the mathematical model to determine the parameters of the mathematical model. Then, by collecting and assembling the mathematical model according to the cover state, the first image conversion model corresponding to the cover open state and the second image conversion model corresponding to the cover closed state are generated. Around View Monitor System. 3. The method of claim 2, wherein the mathematical model is
The process of converting the feature point coordinates defined by the calibration plate coordinate system to the reference coordinate system, the process of converting the feature point coordinates defined by the reference coordinate system to the camera coordinate system, Converting the feature point coordinates defined by the camera coordinate system to the sensor coordinate system standard Around view for an open rear truck, characterized in that it is an image transformation model that sequentially performs a process and a process of radially distorting the feature point coordinates defined based on the sensor coordinate system, and then converting the radial distortion-converted feature point coordinates into image coordinates. monitor system. delete

Images