KR20150145686A - Apparatus for image processing using 3dimension graphic model and method thereof - Google Patents

Abstract

Disclosed are an image processing device using a graphic processing unit (GPU) and a method thereof. The image processing device using a GPU comprises: a collecting module for collecting an image and image data obtained from a car camera, and driving road information obtained through a global positioning system (GPS); a conversion module for extracting an object of the image data from the information collecting module, and converting the image data into a three-dimensional image by using the extracted object; and a display module for displaying the image converted into the three-dimensional image.

Description

TECHNICAL FIELD [0001] The present invention relates to an image processing apparatus using a three-dimensional graphic model and an image processing method using the same,

The present invention relates to an image processing apparatus and method thereof, and more particularly, to an image processing apparatus and method for converting a vehicle front image into 3D using a GPU.

Interest in information processing technology through video is increasing day by day. Particularly, efforts are being made to prevent collision by using an image acquired through a camera mounted on the front or rear of the vehicle.

It is possible to prevent a large traffic accident by knowing the presence of an object (for example, a pedestrian, a vehicle, etc.) that may obstruct the traveling through the camera mounted on the front of the vehicle and the position of the object obstructing the traveling.

However, according to the conventional image processing technique, as shown in FIG. 1, an image processing process including a preprocessing process and a post-process process is performed in order to recognize a forward running vehicle. Therefore, the image display response time is delayed and the driver is provided with the 2D front image, which makes it difficult to detect the distance to the collision object.

In addition, the 2D image provided to the driver has a problem that the perspective and movement of the forward object can not be expressed precisely.

Recently, the semiconductor processing speed is improving, and in the method of increasing only the CPU clock, the direction of the embedded multicore and the GPU are being developed. As the number of applications related to multimedia devices including games increases and diversifies, the progress of processing amount of GPU is being radically improved. Processing on the GPU is faster and more stable than processing by the CPU.

An object of the present invention is to provide a driver with a sophisticated forward image converted into a 3D form by quickly converting a 2D front image provided to a driver into 3D using a GPU (Graphic Process Unit).

It is also an object of the present invention to provide an image processing apparatus and a method thereof that perform real-time image processing by applying a GPU (Graphic Process Unit) having a large number of object processing units to image processing.

In addition, a 2D image, which is a camera image, is rearranged to an area on the space to perform image recognition, thereby providing a driver with a good quality front image.

According to an aspect of the present invention, there is provided an image processing apparatus for collecting image and image data acquired from a camera mounted on a vehicle, traveling road information acquired through a GPS, Extracts an object from the image data from a collection module and an information collection module that extracts information desired by the user by mapping the recognized image in the image captured in the 3D spatial coordinate system and extracts the image data To a three-dimensional image.

In a preferred embodiment, the conversion module includes a separator for separating the background and the object from the collected image, an extractor for extracting the object separated by the separator in the collected image, and a vanishing point included in the collected image, Performing three-dimensional transformation for mapping the calculated distance information to a three-dimensional spatial coordinate system using the extracted object and the retrieved vanishing point, mapping the result to three-dimensional space coordinates, And a conversion unit for converting the image data.

In a preferred embodiment, the display module generates at least one information transfer icon and displays the generated information transfer icon according to the input of the vehicle operator.

In a preferred embodiment, the information transfer icon delivers lane information, obstacle detection information, and driving hazard warning information included in the three-dimensional image acquired from the conversion module to the vehicle driver.

In a preferred embodiment, the conversion module converts object information extracted in a polygon composed of vertices constituting three-dimensional spatial coordinates from the collected image into spatial coordinates to convert the three-dimensional data into spatial coordinates .

According to another aspect of the present invention, there is provided an image processing method including the steps of: collecting image and image data acquired from a vehicle-mounted camera; traveling road information acquired through a GPS; extracting an object from the image data from the information collection module; And converting the image data into a three-dimensional image using the extracted object.

In a preferred embodiment, the image processing method further comprises displaying the image converted into the three-dimensional image.

In the preferred embodiment, the process of converting the image data into the three-dimensional image includes a process of separating the background and the object from the collected image, a process of extracting the object separated by the separator in the collected image, And converting the collected image into 3D using the extracted object and the retrieved vanishing point.

In a preferred embodiment, the process of displaying the converted image displays at least one information transfer icon and the generated information transfer icon in accordance with the input of the vehicle operator.

In the preferred embodiment, the process of converting the collected image to 3D includes a process of extracting a polygon constituting a 3D image from the collected image, and a process of mapping the collected image, which is a 2D image, to the extracted polygon .

In a preferred embodiment, the information transfer icon conveys driving safety information including lane information, obstacle detection information, warning warning during driving, and optimal vehicle speed information included in the converted three-dimensional image to the vehicle driver.

According to the present invention, real-time image processing can be performed using a GPU, thereby improving image processing response time performance.

In addition, it is possible to provide the driver with the information necessary for the safe driving of the vehicle and the forward image in a real-time 3D image so that the driver can more easily perceive the distance to the braking object located ahead, .

In addition, due to the improved image processing speed of the GPU, it is possible to perform the operations required for image processing more quickly, real-time image processing is possible, and accurate recognition and position determination can be performed by performing 2D image matching in 3D space. This increases the speed of the overall image processing operation, thereby enhancing the user's convenience.

1 is a diagram showing a conventional image processing process.
2 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention.
3 is a block diagram showing a detailed configuration of an image processing apparatus according to an embodiment of the present invention.
FIG. 4A is a diagram illustrating a structure of a GPU according to an embodiment of the present invention, and FIGS. 4B to 4C are diagrams illustrating an example of an object extraction process according to an embodiment of the present invention.
FIG. 5 is a diagram for helping an understanding of the 3D image conversion process according to the embodiment.
6 is a flowchart illustrating an image processing method according to an embodiment of the present invention.
7 is a flowchart illustrating an image conversion process according to an embodiment of the present invention.
8 is a diagram showing a configuration of a computer apparatus in which an image processing method according to an embodiment of the present invention can be executed.

The present invention is a result of research carried out with the support of the "RE-EV practical research base construction project" which was funded by the Ministry of Industry and Trade (MOTIE) and supported by the Korea Industrial Technology Development Organization (KIAT).

The present invention is also applicable to a top view within 10 cm of resolution 0043358 with the support of the Ministry of Trade Industry and Energy (MOTIE) and the Korea Institute of Advancement of Technology (KIAT) And a peripheral image information configuration and recognition system capable of panorama view ".

In the present invention, a front image of a 2D type provided to the driver is converted into 3D in real time using a GPU (Graphic Process Unit), thereby providing a driver with a sophisticated forward image converted into a 3D form. Thereby improving the overall response performance of the vehicle display system.

In addition, by providing a video processing apparatus and method in which real-time image processing is performed by applying a GPU (Graphic Process Unit) containing a large number of object processing units to image processing, information necessary for driving a vehicle forward image and safety is provided The 3D image is displayed to the driver in real time so that the driver can more easily recognize the distance to the front braking object and provide the driver with a high quality image.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification.

The term " module ", as used herein, should be interpreted to include software, hardware, or a combination thereof, depending on the context in which the term is used. For example, the software may be machine language, firmware, embedded code, and application software. In another example, the hardware may be a circuit, a processor, a computer, an integrated circuit, a circuit core, a sensor, a micro-electro-mechanical system (MEMS), a passive device, or a combination thereof.

1 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention.

Referring to FIG. 1, an image processing apparatus using a GPU according to an embodiment includes a collection module 100, a conversion module 200, and a display module 300.

The collection module 100 collects image and image data acquired from the vehicle-mounted camera, and travel road information acquired through GPS. Also, an image recognized by an image captured by a camera installed in the vehicle is mapped to a three-dimensional spatial coordinate system to extract information desired by the user.

The conversion module 200 extracts an object of image data from the collection module 100 and converts the image data into a three-dimensional image using the extracted object. Then, the display module 300 displays the image converted into the three-dimensional image.

For example, the conversion module registers object information extracted in a polygon composed of vertices constituting three-dimensional spatial coordinates from the collected image in spatial coordinates, converts the three-dimensional data into spatial coordinates, .

The GPU shown in FIG. 4A used in the present invention is a high-performance processing device for graphics processing. It is difficult to display a high-dimensional image because the calculation result of the CPU (Central Processing Unit) is converted into a picture or character signal and outputted. However, with GPU-based graphics cards, 3D graphics performance is greatly improved, enabling higher-dimensional image display to run more smoothly.

2 is a block diagram showing a detailed configuration of an image processing apparatus according to an embodiment of the present invention.

2, the conversion module 200 of the image processing apparatus according to the embodiment includes a separation unit 210, an extraction unit 230, and a conversion unit 250, An output unit 310, a generating unit 330, and an output unit 350.

The separator 210 separates the background and the object from the collected image. In the embodiment, the reason for separating the background and the object for the 3D conversion is that the objects included in the image collected during the driving are not in the stopped state. Therefore, in order to recognize the relative movement included in the image while the vehicle is running, And separates the object.

Then, the extracting unit 230 extracts the objects separated by the separating unit from the collected images. For example, as shown in FIG. 4B, the extracting unit 230 may search for an object having a vertical component on the basis of the vanishing point, extract the object to be searched, or search for a vertical component of the object on the basis of the vanishing point , And objects are objectsized using the retrieved vertical component. For example, upper, lower, left, and right planes are selected at the vanishing point, and a vertical component for searching for an object on the selected plane is searched. The extraction unit 230 extracts a person as an object, based on a vanishing point or a moving or stopping person having the structure of a face and an arms and legs. As another example, the extracting unit 230 extracts an object, which is contrary to the perspective, as an object, as shown in FIG. 4C.

If the vanishing point can not be retrieved (for example, when a large object such as a truck in front is covered by a vanishing point), the extracting unit 230 extracts the vanishing point based on the previously stored vanishing point directional data Create a virtual vanishing point and extract the objects contained in the image.

The conversion unit 250 searches for the vanishing point included in the collected image, and converts the collected image into 3D using the extracted object and the vanishing point.

For example, the transform unit 250 searches for the vanishing point included in the collected image, performs three-dimensional transformation for mapping the calculated distance information to the three-dimensional spatial coordinate system using the extracted object and the vanishing point that is searched, Maps the result to three-dimensional spatial coordinates, and converts the collected image into three-dimensional coordinates.

 In the embodiment, the conversion module 200 extracts a polygon forming a 3D image from the collected image, maps the collected image, which is a 2D image, to the extracted polygon, and converts the collected image into 3D . A polygon refers to a polygon that is a basic unit constituting a 3D graphic. The polygons of the polygon are gathered together to form a character or a '3D object'. FIG. 5 is a diagram for helping an understanding of the 3D image conversion process according to the embodiment.

The input unit 310 included in the display module 300 is a touch screen type and can output a 3D image that is simultaneously converted to an input and can be provided to a driver. In an embodiment, the generating unit 330 generates at least one information transfer icon according to the input of the driver, and displays the combined information transfer icons through the output unit 350 in combination. At this time, the information transfer icon may provide lane information, obstacle detection information, and danger warning information during driving, which are included in the three-dimensional image acquired from the conversion module 200, to the vehicle driver.

Here, the information transfer icon may be an icon represented by an indicator provided to the user, such as a speed of 30 km / h 60 km / h 90 km / h, a front obstacle including a person, have.

Hereinafter, an image processing method using a GPU according to another embodiment of the present invention will be described with reference to the drawings.

6 is a flowchart illustrating an image processing method according to an embodiment of the present invention.

In step S100, image and image data acquired from the vehicle-mounted camera and traveling road information acquired through GPS are collected. In step S200, an object is extracted from the collected image data. In step S300, the image data is converted into a three-dimensional image using the extracted object.

7 is a flowchart illustrating an image conversion process according to an embodiment of the present invention.

In order to convert the collected image into a three-dimensional image, a process of separating the background and the object from the collected image is performed in step S310. Thereafter, in step S320, a process of extracting an object is performed. In step S330, a process of converting the collected image into three-dimensional image is performed using the extracted object and vanishing point.

In the embodiment, the process of converting the collected image into 3D may include a process of extracting a polygon constituting a 3D image from the collected image, as shown in FIG. 5, and a process of extracting a polygon It can be done by mapping images.

Thereafter, the converted image is provided to the driver through the display module 300.

The display module 300 displays driving safety information including lane information, obstacle detection information, danger warning information during driving, and optimal vehicle speed information included in the converted three-dimensional image on a front panel, Lt; / RTI > In addition, the driving safety information can be provided to the driver in the form of sound including a voice of a person (for example, detection of a pedestrian 10 meters ahead, please pay attention).

As described above, in the embodiment of the present invention, the front image of the 2D form provided to the driver is converted into 3D in real time using a GPU (Graphic Process Unit), thereby providing the driver with a sophisticated forward image converted into the 3D form . Thereby improving the overall response performance of the vehicle display system.

In addition, by providing a video processing apparatus and method in which real-time image processing is performed by applying a GPU (Graphic Process Unit) containing a large number of object processing units to image processing, information necessary for driving a vehicle forward image and safety is provided The 3D image is displayed to the driver in real time so that the driver can more easily recognize the distance to the front braking object and provide the driver with a high quality image.

Meanwhile, the image processing method according to the embodiment of the present invention can be implemented in a computer system or recorded on a recording medium. 8, the computer system includes at least one processor 121, a memory 123, a user input device 126, a data communication bus 126, a user output device 127, And may include a storage 128. Each of the above-described components performs data communication via the data communication bus 122. [

The computer system may further include a network interface 129 coupled to the network. The processor 121 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 123 and / or the storage 128.

The memory 123 and the storage 128 may include various forms of volatile or nonvolatile storage media. For example, the memory 123 may include a ROM 124 and a RAM 125. Therefore, the order management method using the electronic table according to the embodiment of the present invention can be implemented by a method executable by a computer.

Meanwhile, the image processing method according to the embodiment of the present invention described above can be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording media storing data that can be decoded by a computer system. For example, there may be a ROM (Read Only Memory), a RAM (Random Access Memory), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device and the like. The computer-readable recording medium may also be distributed and executed in a computer system connected to a computer network and stored and executed as a code that can be read in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. Therefore, the scope of the present invention should not be limited by the illustrated embodiments, but should be determined by the scope of the appended claims and equivalents thereof.

Claims (11)

The navigation system collects image and image data acquired from the vehicle-mounted camera and traveling road information acquired through GPS, maps the recognized image in the image captured by the camera installed in the vehicle to the 3D spatial coordinate system, A collecting module for extracting the collected data;
A conversion module for extracting an object from the image data from the information collection module and converting the image data into a three-dimensional image using the extracted object; And
To the image processing apparatus.
2. The apparatus of claim 1,
A separator for separating a background and an object from the collected image;
An extracting unit for extracting an object separated by the separating unit in the collected image; And
Dimensional space for mapping the calculated distance information to a three-dimensional space coordinate system using the extracted object and the retrieved vanishing point, and performs a three-dimensional transformation for mapping the calculated distance information to a three-dimensional space coordinate system, A transform unit for transforming the collected image into three-dimensional coordinates by mapping to coordinates;
And an image processing unit for processing the image.
The display device of claim 1, wherein the display module
Generating at least one information transfer icon for transferring the lane information, the obstacle detection information, and the driving-danger warning information included in the three-dimensional image acquired from the conversion module to the vehicle driver according to the input of the vehicle driver, And displays the generated information transfer icon.
2. The apparatus of claim 1,
And the object information extracted in a polygon composed of vertices constituting three-dimensional spatial coordinates from the collected image is registered in spatial coordinates to convert the three-dimensional data into spatial coordinates.
A process of collecting image and image data acquired from a vehicle-mounted camera and traveling road information acquired through GPS;
Extracting an object from the image data from the information collection module; And
Converting the image data into a three-dimensional image using the extracted object; And an image processing method.
6. The image processing method according to claim 5,
Displaying an image converted into the three-dimensional image; Further comprising the steps of:
6. The method of claim 5, wherein the step of converting the image data into a three-
Separating the background and the object from the collected image;
Extracting a separated object from the collected image;
Searching for a vanishing point included in the collected image, and converting the collected image to 3D using the extracted object and the vanishing point that is searched;
Wherein the image processing method comprises the steps of:
The method as claimed in claim 6, wherein the step of displaying the converted image comprises:
And generates at least one information transfer icon according to an input of the vehicle driver and displays the generated information transfer icon.
8. The method of claim 7, wherein the step of converting the collected image into 3D comprises:
Extracting a polygon constituting a 3D image from the collected image; And
Mapping the collected image as a 2D image to the extracted polygon; of
Wherein the image processing method comprises the steps of:
The method of claim 8,
And transmits the driving safety information including the lane information, the obstacle detection information, the warning warning during driving, and the optimal vehicle speed information included in the converted three-dimensional image to the vehicle driver.
11. The recording medium according to any one of claims 5 to 10, wherein the program for executing the image processing method is recorded and decoded by a computer.

Images