KR20190062876A - Image processing apparatus and image compression method thereof - Google Patents

Abstract

The present invention relates to an image processing apparatus which can effectively compress an image without degrading the resolution quality for a region of interest in an image, and an image compressing method thereof. To this end, the image processing apparatus comprises: an image obtaining unit obtaining an image through at least one camera installed in a vehicle; a distance measuring unit for detecting a distance between the vehicle and an obstacle around the vehicle; a gear position sensor for obtaining gear information; a steering angle sensor for measuring a steering angle; and a processor for sensing a position of the obstacle through the distance measuring unit, generating an estimated trajectory of the vehicle based on the gear information and the steering angle, and compressing the image using the position of the obstacle and the estimated trajectory by adjusting a compression rate for each pixel based on a pixel significance map after generating the pixel significance map.

Description

[0001] IMAGE PROCESSING APPARATUS AND IMAGE COMPRESSION METHOD THEREOF [0002]

The present invention relates to an image processing apparatus capable of compressing an image efficiently without hindering resolution quality for a region of interest in an image and an image compressing method thereof.

Recently, various systems for assisting driver's safe driving have been applied to vehicles. A driver assistance system such as an automatic parking system, a lane maintenance assist system, and a collision avoidance system analyzes the images of the surroundings of the vehicle obtained through the camera to recognize the surroundings of the vehicle.

Such a driver assistance system can record the peripheral image of the vehicle obtained through the camera when driving and parking the vehicle, or transmit the peripheral image to an external device. At this time, the driver assistance system stores the image data of a large amount and compresses the image using the image compression technique to reduce the communication load.

Conventional image compression technology extracts a motion vector from an input image, identifies a dynamic object, sets a partial area of the dynamic object as a region of interest, and compresses the image so that the compression rate for the region of interest and the compression rate for the dynamic object are different from each other .

However, since the conventional image compression technique extracts motion vectors from all areas of the original image acquired when moving the vehicle, it is difficult to find actual dynamic objects. Therefore, it is difficult to find an actual dynamic object, It is difficult to apply it to the monitoring system. Particularly, since the area of interest in the image acquired by the peripheral monitoring system of the vehicle is the vicinity of the vehicle and the estimated path, there is a limit to the conventional image compression technology for changing the compression rate for the recognized dynamic object by recognizing the dynamic object.

[Literature] KR 101704775 B1

The present invention provides an image processing apparatus and an image compression method capable of efficiently compressing an image without hindering the resolution quality of a region of interest in the image.

According to an aspect of the present invention, there is provided an image processing apparatus including an image acquisition unit that acquires an image through at least one camera mounted on a vehicle, a distance detection unit that detects a distance between the vehicle and an obstacle around the vehicle A gear position sensor for acquiring gear information, a steering angle sensor for measuring a steering angle, and a distance measuring unit for sensing the position of the obstacle and generating an expected trajectory of the vehicle based on the gear information and the steering angle And generating a pixel importance map using the position of the obstacle and the predicted locus, and compressing the image by adjusting a compression ratio for each pixel based on the pixel importance map.

The processor generates a view transformation image by performing a view transformation on an image acquired through the image acquisition unit.

The view-converted image is any one of a top view image, a forward view image, a rear view image, and a side view image.

And the processor compresses the view-transformed image according to the pixel-by-pixel compression ratio.

And the pixel importance map includes pixel importance for each pixel of the image.

The pixel importance is a value between the lower limit pixel importance and the upper limit pixel importance.

The image processing apparatus further comprises a communication unit for transmitting the compressed image under the control of the processor and for checking the data communication speed.

And the processor adjusts the lower-limit pixel importance and the upper-limit pixel importance according to a data communication rate fed back from the communication unit.

Meanwhile, an image compression method of an image processing apparatus according to an embodiment of the present invention includes: acquiring an image through at least one camera mounted on a vehicle; generating an expected trajectory based on gear information and a steering angle of the vehicle , Calculating an obstacle position around the vehicle, generating a pixel importance map based on the predicted locus and the obstacle position, and compressing the image by adjusting a compression ratio for each pixel based on the pixel importance map .

The method may further include generating at least one of a top view image, a front view image, a rear view image, and a side view image by performing a view transformation of the image after acquiring the image.

The method may further include compressing the image, transmitting the compressed image through a communication channel, checking a data communication rate of the communication channel, and adjusting the pixel importance map according to the data communication rate .

And the pixel importance map includes pixel importance for each pixel of the image.

Wherein the step of adjusting the pixel importance map adjusts a lower importance pixel degree and an upper limit pixel importance degree of the pixel importance.

Wherein the step of adjusting the pixel importance map adjusts the pixel importance map when the data communication speed is different from a reference speed in comparison with a data communication speed in transmitting the previous compressed image.

The pixel importance is used to determine a quantization index.

The present invention performs image compression by varying the compression ratio of each pixel in the ROI according to the vehicle situation (expected path and surrounding obstacle location, etc.), and thus does not hinder the resolution quality of the ROI in the image, can do.

1 is a block diagram of an image processing apparatus according to an embodiment of the present invention;
2 is a diagram for explaining a pixel importance map related to the present invention;
FIGS. 3 to 6 are views showing examples of generating a pixel importance map for each situation of a vehicle related to the present invention; FIG.
7 is a flowchart illustrating an image compression method of an image processing apparatus according to an embodiment of the present invention.
8 is an exemplary diagram illustrating image compression according to the present invention;

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the difference that the embodiments of the present invention are not conclusive.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. Also, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

The present invention relates to a technology for efficiently compressing an image without disturbing the resolution quality of a region of interest in an image when a vehicle peripheral monitoring (AVM or SVM) image is stored or transmitted to an external device such as a smart phone. Here, the region of interest refers to the periphery of the vehicle and the expected path.

1 is a block diagram showing an image processing apparatus according to an embodiment of the present invention.

1, the image processing apparatus includes an image acquisition unit 110, a distance measurement unit 120, a gear position sensor 130, a steering angle sensor 140, a communication unit 150, a storage unit 160, And a processor 170.

The image acquisition unit 110 is mounted on the vehicle and acquires a peripheral image of the vehicle. The image obtaining unit 110 may include at least one of a first camera 111, a second camera 112, a third camera 113, and a fourth camera 114. The first camera 111 to the fourth camera 114 are installed on the front, rear, left, and right sides of the vehicle, respectively. In the present embodiment, acquisition of a peripheral image through four cameras 111 to 114 is disclosed. However, the present invention is not limited to this, and five or more cameras may be installed in a vehicle and peripheral images may be acquired through the cameras .

The camera 111, 112, 113, or 114 may be a charge coupled device (CCD) image sensor, a complementary metal oxide semi-conductor CMOS image sensor, a charge priming device (CPD) device image sensor, and the like. The camera 111, 112, 113, or 114 may include at least one of lenses such as a standard lens, an ultra-wide angle lens, a wide angle lens, a zoom lens, a macro lens, a telephoto lens, . The camera 111, 112, 113, or 114 includes an image processor that performs image processing such as noise elimination, color reproduction, image quality and saturation adjustment, and file compression for an image acquired through an image sensor .

The distance measuring unit 120 measures the distance between the vehicle and surrounding objects (vehicle, obstacle, and person, etc.). The distance measuring unit 120 may include at least one of distance sensors such as a radar (a radar) 121 and an ultrasonic sensor 122.

The radar 121 generates an electromagnetic wave in its surroundings, receives electromagnetic waves reflected from nearby objects, and can confirm the distance, direction, and altitude with respect to the surrounding objects.

The ultrasonic sensor 122 generates ultrasonic waves to sense nearby objects and measures the distance between the vehicle and surrounding objects.

The gear position sensor 130 detects the position (gear information) of the selector lever according to the operation of the driver. The position of the shift lever may be classified into parking (P stage), forward (D stage), neutral (N stage), or reverse (R stage). The gear position sensor 130 detects the running state of the vehicle.

The steering angle sensor 140 measures the steering angle of the vehicle. The steering angle sensor 140 is installed in a steering column switch cluster and measures a rotation angle of the steering wheel.

In this embodiment, the position and / or steering angle of the shift lever is sensed through the sensors mounted on the vehicle. However, the present invention is not limited to this, and the position (gear information) As shown in FIG.

The communication unit 150 is a communication unit that allows the image processing apparatus to function as an electronic control unit (ECU) mounted on a vehicle, a vehicle terminal such as navigation and AVN and a mobile terminal such as a smart phone, Etc.) and the like. The communication unit 150 measures the data communication speed (communication speed) of the communication channel and feeds back the measured communication speed to the compression unit 174 of the processor 170.

The communication unit 150 may use a vehicle network and / or a wireless network. The vehicle network is implemented in a Controller Area Network (CAN), a Media Oriented Systems Transport (MOST) network, a Local Interconnect Network (LIN), or an X-by-Wire (Flexray). The wireless network may be implemented with at least one of wireless Internet (e.g. wi-fi), local area communication (e.g. Bluetooth, ZigBee and infrared communication), and communication technology such as mobile communication.

The storage unit 160 may store software programmed by the processor 170 to perform predetermined operations. The storage unit 160 may temporarily store input / output data of the processor 170. The storage unit 160 may store the image data acquired through the image acquisition unit 110. FIG.

The storage unit 160 may be installed inside and / or outside the processor 170. The storage unit 160 may be a flash memory, a hard disk, an SD card (Secure Digital Card), a RAM (Random Access Memory), a ROM (Read Only Memory) web storage), and the like.

The processor 170 controls the overall operation of the image processing apparatus. The processor 170 may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), microcontrollers, and microprocessors.

The processor 170 includes a view conversion unit 171, an obstacle detection unit 172, a locus generation unit 173, and a compression unit 174.

The view conversion unit 171 receives the original image from the image acquisition unit 110 and performs a view conversion. The view conversion unit 171 performs a view conversion on the images acquired through the first camera 111 to the fourth camera 114 to generate a top view image, a forward view image, a rear view image, And generates at least one view-converted image from the side view images. That is, the view conversion unit 171 generates one view conversion image using the four images acquired through the first camera 111 through the fourth camera 114 through the view conversion.

The obstacle detection unit 172 analyzes the view conversion image output from the original image or view conversion unit 171 acquired through the image acquisition unit 110 using the image recognition technology to determine the position of the surrounding obstacle . Alternatively, the obstacle detecting unit 172 may calculate the position of the surrounding obstacle by using the distance measuring unit 120. That is, the obstacle sensing unit 172 measures the distance from the vehicle to the surrounding obstacle through the radar 121 and / or the ultrasonic sensor 122.

The locus generator 173 generates an expected trajectory of the vehicle using the gear information and the steering angle information acquired through the gear position sensor 130 and the steering angle sensor 140. [

The compression unit 174 generates the pixel importance map using the position information of the peripheral obstacle provided from the obstacle sensing unit 172 and the predicted locus information provided from the locus generator 173. [ The pixel importance map includes the importance S (x, y) of each pixel constituting an image frame. The pixel importance is a value within the range of the lower limit pixel importance (lower limit value) S _min to the upper limit pixel importance (upper limit value) S _max .

The compression unit 174 determines the compression ratio for each pixel based on the pixel importance map. The compression unit 174 compresses the input image according to the determined compression ratio for each pixel. At this time, the input image may be a view-converted image output from the view conversion unit 171 or an original image output from the image acquisition unit 110. [

The compression unit 174 may use the pixel importance to determine a quantization index (Q) when performing image compression according to the MPEG standard.

The compression unit 174 may store the compressed image (image data) in the storage unit 160. In addition, the compression unit 174 may transmit (transmit) the compressed image through the communication unit 150.

The compression unit 174 adjusts the pixel importance degree range based on the communication speed fed back from the communication unit 150 after transmitting the compressed image. The compression unit 174 adjusts the pixel importance range when a difference of at least a reference (for example, ± 5%) from the communication speed when the communication speed fed back from the communication unit 150 compresses and transmits the previous frame. Here, as the pixel importance range is adjusted, the compression ratio for each pixel is also adjusted.

The compression unit 174 maintains the previous pixel-by-pixel compression ratio when the feedback communication speed is lower than the communication speed of the previous frame.

In this embodiment, the communication unit 150 is provided outside the processor 170 to measure the communication speed, feed back the processed data to the processor 170, and output the compression rate reference (pixel importance range) based on the communication speed at which the compression unit 174 has been fed back. Is adjusted. However, the present invention is not limited to this, and the communication unit for image data transmission may be mounted in the processor 170 and the compression unit 174 may be fed back with a compression rate criterion corresponding to the communication speed.

FIG. 2 is a diagram for explaining a pixel importance map related to the present invention, and FIGS. 3 to 6 are exemplary diagrams for generating a pixel importance map for each vehicle according to the present invention.

All pixels of an image (e.g., top view image) have respective pixel importance values (pixel importance) S (x, y). The pixel importance map is a set of importance values for each pixel of an image. As mentioned above, the pixel significance S (x, y) can be used to determine the quantization index in image compression according to the MPEG standard.

Pixel Priority S (x, y) has a value between the minimum value S _min and maximum value S _max. Processor 170 of the image processing apparatus does not allow the loss of image by adjusting the minimum value S _min and maximum value S _max of the pixel priority S (x, y) pixel-by-pixel to control the compression ratio in accordance with the communication speed.

The processor 170 of the image processing apparatus can set a pixel importance map for each situation according to the situation of the vehicle. The processor 170 generates (sets) a pixel importance map based on the gear information and the obstacle position information.

2, when the shift lever of the vehicle is located at the P-stage, the pixel importance S (x, y) to a position (pixel) away from the vehicle by a certain distance is the upper limit value S _max , The pixel significance S (x, y) at a position further away from the distant position gradually decreases from the upper limit value S _max to the lower limit value S _min .

The processor 170 generates a pixel importance map in accordance with the expected trajectory shifted by d in a straight line when the vehicle is in the straight running (D-stage) state and the steering angle is 0 degree, as shown in Fig.

On the other hand, the processor 170 generates a pixel importance map based on the expected trajectory when the vehicle is in the straight running state and the steering angle is -30 degrees.

The processor 170 sets the pixel importance map based on the expected trajectory moving backward by d in the case where the shift lever of the vehicle is in the reverse position (R-stage) and no steering operation is performed as shown in FIG.

Meanwhile, the processor 170 expands the pixel importance map in consideration of the expected trajectory when the vehicle is in the reverse gear state and the steering angle is +30 degrees.

The processor 170 generates at least one of the cameras 111 to 114, the radar 121 and the ultrasonic sensor 122 to generate a pixel importance map in consideration of an obstacle position when detecting an obstacle around the vehicle. At this time, increase the pixel importance value of the point (pixel) where the obstacle is located.

As shown in FIG. 5, when the obstacle is located behind the vehicle, the processor 170 sets a high importance value for the pixel in the area where the obstacle is located in the image.

As shown in FIG. 6, if an obstacle is located at the rear of the vehicle and the vehicle is going to attempt to reverse, the processor 170 generates a pixel importance map in consideration of the obstacle position and the expected trajectory of the vehicle.

7 is a flowchart illustrating an image compression method of an image processing apparatus according to an embodiment of the present invention.

First, the processor 170 acquires an image of the surroundings of the vehicle through the image acquisition unit 110 (S110). The processor 170 simultaneously acquires an image through each of the first camera 111 to the fourth camera 114.

The processor 170 performs a view conversion on the acquired image (S120). The view conversion unit 171 of the processor 170 generates a view conversion image by performing a view conversion process on four image frames inputted from the first camera 111 to the fourth camera 114. The view conversion unit 171 may generate at least one of a top view image, a forward view image, a rear view image, and a side view image through a view transformation.

The processor 170 generates an expected trajectory of the vehicle using the gear information and the steering angle information (S130). The processor 170 generates an expected trajectory of the vehicle based on the positional information of the shift lever output from the gear position sensor 130 and the steering angle sensed by the steering angle sensor 140. [

 The processor 170 calculates the position of an obstacle around the vehicle (S140). The processor 170 calculates the position of the obstacle through the image processing of the image (original image) or the view-converted image acquired through the image obtaining unit 110. In addition, the processor 170 may calculate the position of the obstacle using the radar 121 or the ultrasonic sensor 122. Here, the processor 170 may calculate the position of the obstacle by integrating the obstacle position calculating method using the cameras 111 to 114 and the obstacle position calculating method using the distance sensors 121 and / or 122.

The processor 170 synthesizes the predicted locus and the obstacle position information to generate a pixel importance map of the view transformation image (S150). The pixel importance map is used to determine the compression ratio for each pixel in an image.

The processor 170 varies the compressibility of each pixel based on the pixel importance map to compress the view-transformed image (S160). The processor 170 compresses the view-converted image according to the pixel-by-pixel compression ratio.

The processor 170 transmits the compressed image (image data) through the communication channel (S170). The processor 170 transmits the compressed image through the communication unit 150. The processor 170 receives the feedback of the communication speed when the compressed image is transmitted from the communication unit 150.

The processor 170 checks the communication speed after transmitting the compressed image, and confirms whether the contrast rate of the previous frame is changed (S180). The processor 170 confirms whether the communication rate fed back from the communication unit 150 has been changed in contrast to the communication speed when transmitting the compressed previous video.

The processor 170 adjusts the pixel importance range when the communication speed is changed in preparation for the previous image frame transmission (S190). For example, when the communication speed is lower than the reference speed, the processor 170 adjusts S _max and S _min to be low so as to increase the compression rate so that there is no problem in image transmission. On the other hand, when the communication speed increases again, the processor 170 adjusts S _max and S _min so as to lower the compression rate, thereby improving the image quality.

In this embodiment, the processor 170 performs S110 to S190 each time one image frame is input.

In the above-described embodiment, view conversion and image compression of an image are disclosed. However, the present invention is not limited to this, and an original image acquired through a camera may be compressed without performing view conversion.

8 is an exemplary diagram illustrating image compression according to the present invention.

As shown in FIG. 8, when the variable compression is performed by applying the compression technique proposed in the present invention, it can be seen that the data size of the non-compression contrast is greatly reduced.

In addition, compared with the case where the entire area of the image frame is uniformly compressed, the difference in the data size is small, but the predicted path area is clearly visible. That is, the compressed image according to the compression technique of the present invention has no significant difference in the data size compared to the uniformly compressed compressed image, but the resolution quality is improved.

The present invention can reduce the data capacity by varying the compressibility of each pixel of the ROI in the image. For example, when using the remote automatic parking function or the valet parking function, the driver may transmit an image to the driver's mobile terminal so that the driver can check the surroundings of the vehicle in real time. At this time, the communication load can be reduced by compressing the image and reducing the amount of data to be transmitted.

In addition, when the storage space is limited, such as a black box, the storage time can be increased by compressing the image.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

110:
111 to 114: camera
120: distance measuring unit
121: Radar
122: Ultrasonic sensor
130: Gear position sensor
140: steering angle sensor
150:
160:
170: processor
171:
172:
173: locus generator
174:

Claims (17)

An image acquiring unit for acquiring an image through at least one camera mounted on a vehicle,
A distance measuring unit for detecting a distance between the vehicle and an obstacle around the vehicle,
A gear position sensor for acquiring gear information,
A steering angle sensor for measuring a steering angle, and
And a processor for detecting the position of the obstacle through the distance measuring unit, generating an expected trajectory of the vehicle based on the gear information and the steering angle, and compressing the image using the position of the obstacle and the predicted trajectory The image processing apparatus characterized in.
The method according to claim 1,
The processor includes:
And generating a pixel importance map using the position of the obstacle and the predicted locus, and compressing the image by adjusting a compression ratio for each pixel based on the pixel importance map.
3. The method of claim 2,
The processor includes:
Wherein the image processing unit generates a view transformation image by performing a view transformation of the image acquired through the image acquisition unit.
The method of claim 3,
The view-
A back view image, a top view image, a front view image, a rear view image, and a side view image.
The method of claim 3,
The processor includes:
And compresses the view-transformed image according to the pixel-by-pixel compression ratio.
3. The method of claim 2,
The pixel importance map includes:
And a pixel importance for each of all pixels of the image.
The method according to claim 6,
The pixel importance may be expressed as:
And a value between a lower limit pixel importance and an upper limit pixel importance.
8. The method of claim 7,
Further comprising a communication unit for transmitting a compressed image under the control of the processor and checking a data communication speed.
9. The method of claim 8,
The processor includes:
And adjusts the lower-limit pixel importance and the upper-limit pixel importance according to a data communication rate fed back from the communication unit.
Acquiring an image through at least one camera mounted on a vehicle,
Generating an expected trajectory based on the gear information and the steering angle of the vehicle,
Calculating an obstacle position around the vehicle, and
And compressing the image based on the predicted locus and the obstacle position.
11. The method of claim 10,
After obtaining the image,
Further comprising the step of generating at least one of a top view image, a front view image, a rear view image, and a side view image by performing a view transformation of the image.
11. The method of claim 10,
Wherein compressing the image comprises:
Generating a pixel importance map based on the expected trajectory and the obstacle location, and
And compressing the image by adjusting a compression ratio for each pixel based on the pixel importance map.
13. The method of claim 12,
After compressing the image,
Transmitting the compressed image through a communication channel,
Checking a data communication rate of the communication channel, and
And adjusting the pixel importance map according to the data communication rate.
14. The method of claim 13,
The pixel importance map includes:
And a pixel importance of each of all pixels of the image.
15. The method of claim 14,
Wherein adjusting the pixel importance map comprises:
And adjusting the lower-limit pixel importance and upper-limit pixel importance of the pixel importance.
15. The method of claim 14,
Wherein adjusting the pixel importance map comprises:
Wherein the pixel importance map is adjusted when a difference between the data communication speed and a data communication speed at the time of transmitting the previous compressed image is greater than a reference value.
15. The method of claim 14,
The pixel importance may be expressed as:
Wherein the quantization index is used to determine a quantization index.

Images