KR101874352B1 - VMS, In-vehicle terminal and intelligent transport system including the same for efficient transmission of traffic information on the road - Google Patents

Abstract

The present invention discloses an in-vehicle terminal for efficient traffic information delivery from a traffic electronic display and a traffic light, and an intelligent traffic system including the same. According to an embodiment of the present invention, the in-vehicle terminal comprises: an input unit which acquires an image photographed by a vehicle; a control unit which recognizes display information of a roadside traffic device including a traffic electronic display and a traffic light from the acquired image, converts the recognized display information into traffic information, determines a dangerous situation from the display information, and generates a danger warning signal; an output unit which outputs the traffic information or the danger warning signal to the inside of a vehicle; a first communication unit which delivers or receives the traffic information or the danger warning signal to/from a control server; and a second communication unit which delivers or receives the traffic information or the danger warning signal to or from surrounding vehicles.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an intelligent transportation system including a traffic signboard, an in-vehicle terminal, and an intelligent transportation system for efficiently transmitting traffic information,

The present invention relates to traffic signal recognition and transmission technology.

It is important to understand the surrounding situation, for example, the flow of roads or nearby vehicles or pedestrians when the vehicle is driven, but it is important to grasp the traffic information through the traffic device installed on the roadside. For example, it is necessary to monitor the state of a traffic light signal indicating the direction of the vehicle. If the driver fails to recognize the traffic light accurately and quickly by selling a sight for a short time or carelessly, it can lead to a traffic accident, which can cause not only a car but also a big loss of life.

One of the traffic devices installed on the roadside is the Variable Message Sign (VMS). Traffic signboards provide road users with real-time information on traffic volume, traffic accidents, congestion zones, weather conditions and construction controls. It is equipment to improve efficiency of traffic flow and safety of traffic.

Although there are roadside traffic devices such as traffic signs and traffic lights, many accidents occur due to driver's carelessness and complicated traffic environment. Also, as in the case of a downtown area where vehicles are concentrated, there are frequent occurrences of not recognizing a roadside traffic device due to obstruction of the roadside traffic device, limitation of the camera angle of view, and inability to recognize a remote roadside traffic device.

According to an embodiment of the present invention, an intelligent traffic system including a traffic signboard, a vehicle terminal, and an intelligent traffic system for efficiently transmitting traffic information is proposed. More particularly, the present invention relates to a method and apparatus for recognizing and analyzing information displayed through a roadside traffic device such as a traffic signboard and a traffic light in front of a running vehicle, and transmitting the information to a user in the form of a voice signal or vibration. We propose an intelligent transportation system that can accurately grasp the dangerous situation in real time and can prepare for the danger situation in advance.

An intelligent traffic system according to an embodiment includes a roadside traffic device including at least one of a traffic light board and a traffic light, an in-vehicle terminal for recognizing information displayed from the roadside traffic equipment and processing and outputting the recognized information, The terminal recognizes the display information of the roadside traffic apparatus from the obtained image, converts the recognized display information into traffic information, determines a dangerous situation from the display information, and generates a danger warning signal A first communication unit for transmitting traffic information or a danger warning signal to the control server or receiving a traffic information or a danger warning signal from the control server, And a second communication unit for receiving the signal from the neighboring vehicle.

The output unit transmits traffic information or a danger warning signal to the user in the vehicle. The first communication unit provides the traffic information or the danger warning signal to the control server through the network and transmits the traffic information or the danger warning signal to the surrounding vehicle through the control server. Traffic information or a danger warning signal can be transmitted directly to the vehicle.

According to an exemplary embodiment of the present invention, there is provided an image processing apparatus including a preprocessor for removing noise of image data sequentially input on a frame-by-frame basis and extracting a region of interest, a color processing unit for separating a color space in the extracted region of interest, A first image analyzing unit for recognizing the display information of the roadside traffic apparatus through at least one of color, position, and shape, and a second image recognizing display information of the roadside traffic apparatus through optical character recognition on the extracted ROI And a signal conversion unit for converting the recognized display information through at least one of the first image analysis unit and the second image analysis unit to traffic information or a danger warning signal.

The second image analyzing unit may scan the area of interest of the image frame data in the form of interlaced scanning and recognize the display information of the roadside traffic device through optical character recognition (OCR).

The control unit interpolates pixels in the frame by performing image warping to move the position of each pixel of the preprocessed input image frame data differently to generate reconstructed image data from which the error has been removed, And an image warping unit for providing the image to the second image analysis unit. The image warping unit calculates a vertical intersection position calculation for calculating the position u of the vertical intersection point of the input image pixel V (x, y) and the control line Li with respect to each control line Li with respect to each pixel V (x, y) (X, y) and a vertical displacement h of the control line Li, and a corresponding position V '((x, y)) in the reconstructed image using the vertical displacement h and the vertical intersection position u. a distance calculator for calculating a distance d between the input image pixel V (x, y) and the control line Li, a distance calculator for calculating a distance d between the input image pixel V (x, y) (X, y) and a reconstructed image pixel V ', and calculates displacement values of the input image pixel V (x, y) and the reconstructed image pixel V' using the distance d between the control line Li _x t, _y t) to calculate input by using the cumulative calculation unit, and a calculated cumulative displacement _(x t, _y t) and controlling the weight of the line to zero (X, Y) corresponding to the image pixel V (x, y) and for copying the reconstructed image pixel V (X, Y) to the input image V And a position calculation unit.

The signal conversion unit converts the display information into text information indicating a traffic situation or a dangerous situation, and converts the converted text information into an output signal corresponding to at least one of a voice signal, a video signal, and a vibration signal, and provides the output signal to the output unit . The output signal can be activated in the vehicle steering wheel or in the vehicle chair.

The terminal in the vehicle further includes a third communication unit for transmitting traffic information or a danger warning signal to the traffic light board. The information transmitted from the in-vehicle terminal to the traffic light board is different from the information obtained from the display information of the traffic light board, It may be information obtained by processing information obtained from the display information of the electric sign board.

According to another aspect of the present invention, there is provided an intelligent traffic system comprising: a roadside traffic device including at least one of a traffic light signboard and a traffic light; an image acquisition unit that recognizes information displayed on the roadside traffic device from the acquired image, A first in-vehicle terminal that converts information into traffic information, generates a danger warning signal in the event of a dangerous situation, and outputs traffic information or a danger warning signal to the control server while transmitting the traffic information or the danger warning signal to the control server; And a second in-vehicle terminal for outputting traffic information or a danger warning signal received from the control server in the vehicle.

According to another embodiment of the present invention, there is provided an intelligent traffic system comprising: a roadside traffic device including at least one of a traffic light signboard and a traffic light; an image acquisition unit that recognizes information displayed on the roadside traffic device from the acquired image, A first in-vehicle terminal that converts the display information into traffic information, generates a danger warning signal in the event of a dangerous situation, and outputs traffic information or a danger warning signal to the terminal in the second vehicle while outputting the traffic information or the danger warning signal; And an in-vehicle terminal that outputs an information or danger warning signal in the vehicle.

According to an embodiment, the display information of the roadside traffic apparatus such as a traffic light or a traffic light board can be recognized and the recognized display information can be efficiently delivered to the in-vehicle user. At this time, traffic information informing the traffic situation and danger warning signal indicating the danger situation are generated and transmitted, so that the traffic information can be grasped in real time and prepared in advance for the dangerous situation. In addition, it can be extended to vehicle-to-infrastructure and vehicle-to-vehicle communications to enable efficient information delivery in a variety of ways.

Furthermore, the image processing speed can be rapidly realized by the capturing speed of the input image through the image warping method. Since the vehicle runs at a very high speed, the input image captured in real time in the running car may be distorted due to poor image quality. According to an embodiment, image warping is applied to an input image to correct the input image, and the image can be restored by eliminating the error. When the image analysis is performed on the reconstructed image, it is possible to perform the image analysis at a high speed according to the capture speed of the input image.

In addition, by scanning in the interlaced scanning mode and performing optical character recognition (OCR), data with high accuracy can be obtained in real time. In other words, by performing the optical character recognition (OCR) by scanning the input image information in the interlaced scanning mode, it is possible to process at a higher speed, and highly accurate data can be obtained.

FIG. 1 is a block diagram of an intelligent traffic system according to an embodiment of the present invention.
FIG. 2 is a detailed configuration diagram of an in-vehicle terminal of FIG. 1 according to an embodiment of the present invention;
FIG. 3 is a detailed configuration diagram of the control unit of FIG. 2 according to an embodiment of the present invention,
FIG. 4 is a detailed configuration diagram of the image warping unit of FIG. 3 according to an embodiment of the present invention,
FIGS. 5A through 5E are views for explaining an image warping method according to an embodiment of the present invention;
6 is a flowchart illustrating an efficient traffic information transmission method using an in-vehicle terminal according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. , Which may vary depending on the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification.

Each block of the accompanying block diagrams and combinations of steps of the flowcharts may be performed by computer program instructions (execution engines), which may be stored in a general-purpose computer, special purpose computer, or other processor of a programmable data processing apparatus The instructions that are executed through the processor of the computer or other programmable data processing equipment will generate means for performing the functions described in each block or flowchart of the block diagram.

These computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner so that the computer usable or computer readable memory It is also possible for the instructions stored in the block diagram to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of the flowchart.

And computer program instructions may be loaded onto a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible that the instructions that perform the data processing equipment provide the steps for executing the functions described in each block of the block diagram and at each step of the flowchart.

Also, each block or step may represent a portion of a module, segment, or code that includes one or more executable instructions for executing the specified logical functions, and in some alternative embodiments, It should be noted that functions may occur out of order. For example, two successive blocks or steps may actually be performed substantially concurrently, and it is also possible that the blocks or steps are performed in the reverse order of the function as needed.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the following embodiments of the present invention may be modified into various other forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to enable those skilled in the art to more fully understand the present invention.

1 is a configuration diagram of an intelligent traffic system according to an embodiment of the present invention.

1, an intelligent transport system (ITS) 1 includes an in-vehicle terminal 2, a roadside traffic apparatus 3, a control server 4, and a network apparatus 5. The embodiment of FIG. 1 includes three in-vehicle terminals (2-1, 2-2, 2-3), two roadside traffic apparatuses (3-1, 3-2) and two network apparatuses (5-1, 5-2). However, the number and arrangement of the respective components are not limited thereto.

The in-vehicle terminal 2 is a terminal provided in the vehicle. The in-vehicle terminal 2 acquires an image photographed in the vehicle. To this end, the in-vehicle terminal 2 may include a video image pickup device such as a camera or may be connected to an external video image pickup device. The external image capturing apparatus may be, for example, a camera mounted on a black box. The image photographing apparatus can acquire an image including the roadside traffic apparatus installed on the road side by continuously photographing the front of the vehicle. The input image is continuously received during driving and can be received in a single frame unit. A frame is a picture taken in a short time which forms a moving picture. In the image processing, an image is usually processed in a frame unit.

The roadside traffic device 3 is a device installed on the road side to notify a traffic situation and includes a traffic light sign (VMS) 3-1 and a traffic light 3-2. A traffic signboard (VMS) (3-1) is an electronic device that provides traffic information in text and graphics form at traffic congestion intersections of roads and the like. Traffic information is, for example, road traffic situation, arrival time, traffic forecast, risk situation, and weather conditions. Road traffic conditions include traffic volume, traffic accidents, traffic congestion, and construction. The traffic lights (3-2) may be automobile traffic lights, pedestrian traffic lights, and the like. The signal displayed through the traffic light 3-2 may be a color such as red (stop display), green (threshold display), an arrow indicating the direction of left turn, straight ahead, etc., Or may be in the form of a number indicating the remaining time information before.

The in-vehicle terminal 2 recognizes the signals displayed on the various types of traffic light display boards (VMS) 3-1 and 3-2 and converts the recognized display signals into traffic information. The display signal includes characters or signals that are visible through the traffic light display (VMS) 3-1, the traffic light 3-2, or the like. If traffic information identifies a dangerous situation, a danger warning signal can be generated.

The in-vehicle terminal 2 provides an intelligent transportation service that automatically transmits traffic information, a danger warning signal, etc. to reduce various incidents and smooth communication. The in-vehicle terminal 2 transmits the traffic information and the danger warning signal in various ways. For example, the delivery method is a method of delivering to a user in the form of a voice signal or vibration in a vehicle in which the terminal 2 is installed in the vehicle, a method of delivering it to the control server 4 via the network, How to deliver it to a vehicle, and how to deliver it directly to a nearby vehicle.

The in-vehicle terminal 2 according to one embodiment supports V2X. For example, the in-vehicle terminal 2 supports vehicle to infra (V2I). According to the vehicle-to-infrastructure communication, the real time traffic situation, the unexpected situation, the traffic flow control status, and the like with the outside, for example, the control server 4 through the infrastructure such as the terminal 2 in the vehicle and the network device 5 installed in the road Exchange information. Traffic conditions can be accident, construction, traffic volume, and so on. As another example, the in-vehicle terminal 2 supports vehicle to vehicle (V2V). According to the vehicle-to-vehicle communication, information such as forward traffic information, front and rear situations between the vehicles, the headway distance, the traveling speed, and the collision alert are exchanged.

The control server 4 is connected to a previously registered in-vehicle terminal 2 to monitor and control the in-vehicle terminal 2 and to transmit information between the in-vehicle terminals 2. When the control server 4 is located remotely, it is connected to the terminal 2 in the vehicle via the network device 5. For example, in the case where information is transmitted between terminals 2 in the vehicle, the control server 4 transmits traffic information or a danger warning signal received from the first terminal 2-1 to the second terminal 8-2 . When the control server 4 receives an image from the in-vehicle terminal 2, it can generate traffic information or a danger warning based on the received image and provide it to the terminal 2 in the vehicle. For example, it analyzes the image received from the first terminal (2-1) to recognize the display information of the roadside traffic apparatus (3), converts the recognized display information into traffic information, and generates a danger warning signal do. And transmits the generated traffic information or danger warning to the first terminal (2-1). Further, it may transmit it to the in-vehicle terminals 2-2 and 2-3.

The network device 5 is a device for connecting the terminal 2 and the control server 4, for example, a cellular base station, a Wi-Fi AP, and the like. Or a sensor installed on the road. In this case, the control server 4 is connected to the terminal 2 in the vehicle through the sensor network.

The V2I-based intelligent traffic system 1 transmits traffic information or a danger warning signal through communication between the terminal 2 and the control server 4. For example, the first terminal (2-1) acquires an image photographed by the vehicle, recognizes the roadside traffic apparatus (3) display information from the obtained image, and displays traffic information or a danger warning signal And outputs it to the car. At this time, the first terminal (2-1) transmits the obtained image or generated traffic information or a danger warning signal to the control server (4). The control server 4 receives the video, traffic information or danger warning signal from the first terminal 2-1, generates traffic information or a danger warning signal from the acquired image when the video is acquired, And transmits traffic information or a danger warning signal to the terminals 2-2 and 2-3 in the nearby cars. The terminals 2-2 and 2-3 in the nearby vehicles output traffic information or a danger warning signal received from the control server 4 in the vehicle.

The V2V-based intelligent traffic system (1) transmits traffic information or danger warning signals between terminals (2) in the vehicle. For example, the first terminal (2-1) acquires an image photographed by the vehicle, recognizes the roadside traffic apparatus (3) display information from the image obtained from the obtained image, and displays the traffic information Or a danger warning signal, and transmits it to the terminals 2-2 and 2-3 in the vicinity of the vehicle while outputting the signal in the vehicle. The terminals 2-2 and 2-3 in the nearby vehicles output traffic information or a danger warning signal received from the first terminal 2-1 in the car.

FIG. 2 is a detailed configuration diagram of the in-vehicle terminal of FIG. 1 according to an embodiment of the present invention.

1 and 2, the in-vehicle terminal 2 includes an input unit 20, a control unit 22, an output unit 24, a first communication unit 26-1, a second communication unit 26-2, 3 communication unit 26-3 and a memory 28. [

The input unit 20 receives a user command and provides it to the controller 22. The input unit 20 also acquires an image of the front of the driving vehicle. At this time, the input unit 20 can directly acquire an image through a camera, and can also receive a captured image from a camera. The camera is placed inside or outside the vehicle to capture the image of the front. The camera may include the same camera as the camera used, for example, in a common navigation system, a vehicle black box system, and the like.

The output unit 24 outputs the information processed by the control unit 22 and the information received through the first communication unit 26-1 and the second communication unit 26-2 in the vehicle. The information can be traffic information or a danger warning signal. The output format may be a voice signal, a video signal, or a vibration type. The voice signal may include a warning tone and is output through a speaker. The video signal is displayed on the screen through the display device. In the case of vibration, vibration can be generated from the vehicle steering wheel or the vehicle chair and transmitted to the user. Examples of traffic information output by voice signals include "Keep safe distance 20m ahead", "Bypass the route because traffic is congested", "Bypass road is 00", "It is an unexpected situation. Please stop. "The danger situation is to immediately transmit a danger warning signal to the vehicle or a nearby vehicle to proactively respond to a dangerous situation.

The control unit 22 recognizes the display information of the roadside traffic apparatus 3 from the image acquired by the input unit 20 and converts the recognized display information into traffic information. And generates a critical alarm signal if the recognized indication information is in a dangerous situation. The detailed configuration of the control unit 22 will be described later with reference to Fig.

The first communication unit 26-1 is a module that communicates with the control server 4, and the second communication unit 26-2 is a module that communicates with nearby vehicles. For example, the first communication unit 26-1 transmits traffic information or a danger warning signal to / from the control server 4. The second communication unit 26-2 transmits traffic information or a danger warning signal to the surrounding vehicles or receives the signals from the surrounding vehicles. The third communication unit 26-3 is a module that directly communicates between the in-vehicle terminal 2 and the VMS 3-2. The VMS 3-2 receives information from the control server 4 under the control of the control server 4 and outputs the information to the VMS 3-2 via the communication with the VMS 3-2, -2). ≪ / RTI > For example, if a vehicle recognizes that an accident occurs due to communication with a nearby vehicle and is in an urgent situation, the vehicle can directly transmit it to the VMS 3-2 without going through the control server 4. The traffic information obtained from the display information of the VMS 3-2 may be different from the information transmitted from the in-vehicle terminal 2 to the VMS 3-2. For example, the information transmitted from the in-vehicle terminal 2 to the VMS 3-2 may be information obtained by processing the traffic information obtained from the display information of the VMS 3-2. The processing can be performed by reflecting the information that the terminal 2 in the vehicle can directly obtain from the field.

The in-vehicle terminal 2 according to an embodiment transmits traffic information or a danger warning signal in various ways. For example, the output unit 24 transmits traffic information or a danger warning signal to a user in the vehicle, and the first communication unit 26-1 transmits traffic information or a danger warning signal to the control server through the network, The second communication unit 26-2 transmits the traffic information or the danger warning signal directly to the surrounding vehicle through the first communication unit 4 and the second communication unit 26-2.

The memory 28 stores information necessary for performing operations in the in-vehicle terminal 2 and information generated according to the performance of the operations. The memory 28 stores pattern information such as a shape pattern, a color pattern, and a signal pattern of the roadside traffic apparatus 3 in order to recognize the display information of the roadside traffic apparatus 3 from the image received through the input unit 20 . At this time, the control unit 22 can recognize the traffic information by comparing the inputted image data with the pattern information stored in the memory 28. [ For example, the control unit 22 compares the image data with a traffic light pattern to detect a traffic light existing in front of the vehicle, compares the color pattern detected from the image data with the color pattern of the traffic light stored in the memory 28 Generates guidance information for guiding the driving direction according to the traveling direction, and outputs the guide information to the output unit 24 through the output unit 24 do.

3 is a detailed configuration diagram of the control unit of FIG. 2 according to an embodiment of the present invention.

1 to 3, the control unit 22 may include a pre-processing unit 220, an image analysis unit 222, and a signal conversion unit 226, and may further include an image warping unit 224. The image analyzing unit 222 may include a first image analyzing unit 222-1 and a second image analyzing unit 222-2.

The pre-processing unit 220 preprocesses the input image data to remove noise. For example, when the luminance is too bright or dark, it can be compensated and a noise removing operation or the like can be performed. In addition, unnecessary top, bottom, right and left marginal regions can be removed from the image data to extract ROI only. Since the image capturing apparatus captures the front of the vehicle in motion, there is a high probability that the roadside traffic apparatus 3 will be located in the upper portion of the screen. Therefore, if the input frame is the first frame or the roadside traffic device 3 is not detected in the immediately preceding frame, the pre-processing unit 220 can designate the upper part of the inputted frame as the region of interest above the half-point in the Y-axis direction.

If the roadside traffic device 3 has been detected in the previous frame, it is possible to designate an area extended by a predetermined range from the area where the roadside traffic device 3 is detected in the previous frame as the area of interest. The size of the roadside traffic apparatus 3 photographed in front of the traveling vehicle is smaller than that of the roadside traffic apparatus 3 as the distance between the roadside traffic apparatus 3 and the vehicle is reduced It is because it grows. The way of extending or extending can be changed by the speed of the vehicle, the resolution of the camera, and other factors.

The first image analyzing unit 222-1 separates the color space within the extracted ROI and extracts the color space from the separated color space through at least one of the color, As shown in FIG. For example, the first image analyzing unit 222-1 separates five color spaces (Yellow, Red, Green, White, and Blue) from the input image frame data. Furthermore, in the separation step, the color space can be converted from the RGB (Red: Green: Green: Blue) region to the HSL (Hue: Saturation: Saturation: Lightness) region. However, in order to more effectively detect the color of the roadside traffic device 3, the RGB region is referred to as the HSL region . The formula for changing the color space from the RGB area to the HSL area uses a known method.

For example, in the case of recognizing the display information of the first image analyzing unit 222-1, the signal lamp 3-2 is recognized in the photographed image, and the color of the lamp illuminated by the recognized signal lamp 3 is read . That is, it detects a color (red, green, yellow, etc.) that is illuminated in the traffic light 3-2. In the case of a green signal, whether or not it is a left-turn signal is also recognized by recognizing the shape of the green signal and recognizing the display information of the traffic light 3-2 in front of the vehicle.

The second image analyzing unit 222-2 recognizes the display information of the roadside traffic apparatus 3 through optical character recognition on the extracted ROI. For example, the signal displayed on the traffic light board 3-1 is identified through optical character recognition. The second image analyzing unit 222-2 according to an embodiment scans the extracted region of interest of the image frame data by an interlaced scanning method and transmits the image data to the roadside traffic apparatus through Optical Character Recognition (OCR) And recognizes the display information of the display device (3). In the interlaced scanning method, the entire surface is scanned by scanning two lines in a row in order to scan one screen. In order to recognize accurate information as data in real time, the input image information is scanned in an interlaced scanning manner, By performing character recognition (OCR), processing can be performed at a higher speed, and highly accurate data can be obtained.

The signal converting unit 226 converts the recognized display information into traffic information through the first image analyzing unit 222-1 and the second image analyzing unit 222-2, And generates an alarm signal. Traffic information and danger warning signals are output in the car. The signal converting unit 226 converts the display information into text information indicating a traffic situation or a dangerous situation, converts the converted text information into at least one of a voice signal, a warning sound, and a vibration signal, and outputs the converted text information to the output unit 24 to provide. Text to speech technology can convert text to speech using a text to speech (TTS) technique and output it through a speaker. The beep or vibration signal may be activated in the vehicle steering wheel or in the vehicle chair. For example, when a dangerous situation occurs, the vehicle chair may be vibrated so that the user is immediately aware of the danger.

For example, in the case of a dangerous situation alarm, when a character "O.J.A. area accident occurred" is recognized from a character displayed on a traffic signboard of a vehicle, a danger warning signal is immediately generated and immediately transmitted to the vehicle or a nearby vehicle Allows you to proactively respond to dangerous situations. As a delivery method, a warning sound can be outputted, a vibration can be applied to a vehicle chair, and a warning sound and vibration can be transmitted together. Also, please keep a safe distance of 20m in front of you, please avoid the route because traffic is congested, and the bypass road is 00. Quot ;, " Please stop suddenly. "In another example, if the signal lamp located in front of the vehicle is a circular green, the vehicle is stationary, and the directional signal of the vehicle is not a left- The driver of the vehicle receives the red signal of the traffic light and stops in a situation where the traffic light is changed to a green signal in a situation where the vehicle is stationary, As another example, the number displayed in the traffic light can be recognized as the remaining time, and the remaining time can be output as a voice signal.

The image warping unit 224 performs image warping for moving the position of the input image frame data by pixels differently, thereby generating restored image data by interpolating pixels in the frame. Image warping is one technique of geometric processing that moves the position of a pixel. Geometric processing, such as enlargement or reduction of an image, obtains a uniform return result by applying a constant rule for all pixels. On the other hand, image warping can vary the degree of movement for each pixel, so that it can have the same effect as bending an image drawn on a rubber plate arbitrarily. In order to perform the image analysis processing in real time, there is a technical time restriction that the processing speed must be fast as much as the capturing speed of the input image. To solve this problem through image warping of the image warping part 224. That is, since the vehicle runs at a very high speed, an input image captured in real time in a running car may be distorted due to poor image quality. The image warping unit 224 according to an embodiment can correct the input image by applying image warping to the input image, and recover the image by eliminating the error. When the image analysis is performed on the reconstructed image, it is possible to perform the image analysis at a high speed according to the capture speed of the input image.

The image warping unit 224 receives the preprocessed image through the pre-processing unit 220, performs image warping to remove the error, and provides the reconstructed image with the error removed to the image analysis unit 222. The detailed configuration of the image warping unit 224 will be described later with reference to FIG.

4 is a detailed configuration diagram of the image warping unit of FIG. 3 according to an embodiment of the present invention.

4, the image warping unit 224 includes a vertical intersection position calculation unit 2240, a vertical displacement calculation unit 2241, a corresponding pixel position calculation unit 2242, a distance calculation unit 2243, a weight calculation unit A cumulative calculation unit 2245, and an input image pixel position calculation unit 2246.

The vertical intersection position calculation unit 2240 calculates the position u of the vertical intersection of the input image pixel V (x, y) and the control line Li with respect to each control line Li with respect to each pixel V (x, y) of the input image do. The vertical displacement calculation unit 2241 calculates the vertical displacement h of the input image pixel V (x, y) and the control line Li. The corresponding pixel position calculation unit 2242 obtains a corresponding position V '(x', y ') in the reconstructed image using the position u and the vertical displacement h of the vertical intersection point. The distance calculator 2243 calculates the distance d between the input image pixel V (x, y) and the control line Li. The weight calculation unit 2244 calculates the weight of the control line using the distance d between the input image pixel V (x, y) and the control line Li. The cumulative calculation unit 2245 obtains a displacement value between the input image pixel V (x, y) and the reconstructed image pixel V 'and calculates the displacement accumulation (t _x , t _y ) thereof. Input image pixel position calculating section (2246) is reconstructed image pixels corresponding to the displacement accumulation (t _x, t _y), and using a control weight of the line input image pixel V (x, y) calculated in the cumulative calculation section (2245) The position of V (X, Y) is calculated and the restored image pixel V (X, Y) is copied to the input image V (x, y) pixel.

5A to 5E are reference views for explaining an image warping method according to an embodiment of the present invention. 5A is a reference view showing a reconstructed image and an input image for explaining a method of finding a vertical intersection point existing in a control line, and FIG. 5B is a view for explaining a method of obtaining a vertical intersection point existing outside the control line FIG. 5C is a reference diagram for explaining a method of calculating a distance between a pixel and a control line, FIG. 5D is a reference diagram for explaining calculation of a position of a vertical intersection point between a point and a control line, 5E is a reference diagram for explaining the vertical displacement calculation process of the pixel.

1. Correspondence relationship between restored image and input image

For image warping, first describe how to reconstruct the reconstructed image. One of the image warping methods is to use control lines or control points. In order to obtain the warping result, the positions to be mutually corresponded between the restored image and the input image are described as control lines or control points. In addition to the control lines and control points, various methods such as a mesh, a polygon, and the like can be used to represent the correspondence relationship between the restored image and the input image.

2. Warping technique using control line

A method of performing warping in a case where a correspondence relationship between a restored image and an input image is described using a control line will be described below.

First, the vertical intersection point between the pixel and the control line is obtained as shown in FIG. 5A. Then, warping is performed by using the displacement information between the pixel and the vertical intersection point and the position information of the vertical intersection point in the control line and using the reverse mapping. Referring to FIG. 1A, the control line PQ in the input image corresponds to the control line P'Q 'in the restored image, and the pixel V of the input image is copied from the pixel V' in the restored image. In Fig. 3A, PQ and P'Q 'correspond to control lines, and V and V' correspond to pixels.

In order to calculate the position of the pixel V 'corresponding to the pixel V by the backward mapping, the position of C' within P'Q 'is first found to be the same as the relative position of C in the control line PQ. Then we find a point V 'away from C' by the displacement between C and V.

The vertical intersection of the pixel and the control line may not exist inside the control line as shown in FIG. 5B. In this case, the pixel positions of the restored image are searched by the same method as in the case where the vertical intersection point is inside the control line.

Although the movement position of a pixel can be calculated simply for one control line, several control lines are used in actual warping. When there are multiple control lines, each control line affects every pixel of the image.

We can use the weight as shown in Equation 1 as a method for reflecting that one pixel is influenced by several control lines. As shown in Equation (1), the longer the control line becomes, the larger the weight becomes, and the closer the distance between the pixel and the control line becomes, the larger the weight becomes.

... (수학식 1)

... (1)

In Equation (1), p is a value that determines the degree to which the length of the control line affects the weight, and has a value between 0 and 1. a is a value used to prevent division by zero and has a very small value. b is a value that adjusts the degree of change of the weight according to the change of the length and the distance of the control line, and usually a value between 0.5 and 2.0 is used. When this value is increased, the pixels are less influenced by the control lines at a greater distance.

The distance between the pixel and the control line is defined differently depending on the position of the vertical intersection point as shown in FIG. 5C. If the vertical intersection point is inside the control line, the distance between the pixel and the vertical intersection point is used. If the vertical intersection point is outside the control line, the point near the pixel and the distance between the pixels are used.

If there are several control lines, the algorithm for obtaining the corresponding pixel of the restored image for each pixel of the input image is as follows.

1) Position calculation of vertical intersection point

As a first operation, the position of the vertical intersection with each control line L _i is obtained for each pixel V (x, y) of the input image. It is assumed in mannandago pixel V (x, y) is the control point is made perpendicular to the line Li _C (x _c, y _c), as shown in Figure 5d. And the distance from the end point P (x _i , y _i ) of the control line L _i to the vertical intersection point C (x _c , y _c ) is denoted by u.

... (수학식 2)

... (Equation 2)

The value of u can be obtained from the end points P and Q of the control line and the position V of the pixel as shown in Equation (2). In the case of x = x _i and y = y _i (V = P) in the equation (2), when the numerator becomes 0 and the value of u becomes 0 and x = x _{i + 1} and y = y _{i +1} = Q), the denominator and the numerator become the same, and the value of u becomes 1.

The value of u has the following range depending on the position of the vertical intersection point.

u <0: Vertical intersection point outside P

u = 0: vertical intersection point = P

0 <u <1: Vertical intersection point inside the control line

u = 1: vertical intersection point = Q

u> 1: Vertical intersection point outside Q

2) Calculation of vertical displacement from control line

In the second operation, the vertical displacement h from each control line L _i is obtained for the pixel V (x, y) of the input image. The vertical displacement h of the pixel from the control line is as shown in Fig. 5E. The vertical displacement h is a value indicating the position of a pixel on a straight line that crosses the control line and passes through the pixel, and has the following range.

Vertical displacement h =

1) <0: if the pixel is below the control line

2) 0: When the pixel is on the control line

3)> 0: If the pixel is above the control line

In Fig. 5 (a), the value of h is larger than 0 because the pixel V is above the control line. 5 (b), the value of h is smaller than 0 since the pixel V is located below the control line.

The absolute value of the vertical displacement h is the distance between the vertical intersection point and the pixel. Here, the reason why the absolute value is not used for the vertical displacement h is to know the position of the control line and the pixel (code division).

The equation for obtaining the vertical displacement h is given by the following equation (3).

... (수학식 3)

... (3)

3) Corresponding pixel position calculation in reconstructed image

The position u and the vertical displacement h of the vertical intersection with each control line L _i are obtained for each pixel V (x, y) of the input image, and then the value of V (x, y) (X ', y') of the corresponding reconstructed image. If "the amount of the end point coordinate (x _i, L _i of the control line in the reconstructed image corresponding to the control line of the input image is called L _i, y _i ') and _{(x i + 1', y} i +1 ') V '(x', y ') as shown in Equation (4).

... (수학식 4)

... (Equation 4)

4) the distance between the pixel and the control line

The distance d between the pixel and the control line can be obtained by the following equation (5). I.e., u value at this time is smaller than 0, the pixel V (x, y) and P (x _i, y _i) when the distance, u value from greater than 1 pixel V (x, y) and Q (x _{i + 1} , y _{i +1} ), and when the u value is between 0 and 1, the absolute value of the vertical displacement h is obtained.

... (수학식 5)

... (Equation 5)

5) Calculation of control line weights

After obtaining the distance d between the pixel and the control line, the weight of the control line is obtained by using Equation (6). In Equation (6), a, b, and p constant values can be generally used. For example, a = 0.001, b = 2.0, and p = 0.75.

... (수학식 6)

... (Equation 6)

6) Accumulation of displacement from the corresponding pixel of the reconstructed image

The pixel V '(x', y ') of the reconstructed image corresponding to the pixel V (x, y) of the input image is obtained for each control line L _i and the weight is obtained, The product of the displacements and weights between V and V 'is calculated and accumulated in the variables t _x and t _y .

... (수학식 7)

... (7)

7) Corresponding pixel location calculation of reconstructed image

After calculating the displacement values of the pixels of the reconstruction image corresponding to the pixels V (x, y) of the input image with respect to each control line and calculating the sum (t _x , t _y ) thereof, V ( (x, y) of the reconstructed image corresponding to x, y.

... (수학식 8)

... (8)

After the pixel V (X, Y) corresponding to V (x, y) is finally obtained, the value of the V (X, Y) pixel of the restored image is copied to the V (x, y) pixel of the input image .

6 is a flowchart illustrating an efficient traffic information transmission method using an in-vehicle terminal according to an embodiment of the present invention.

Referring to FIG. 6, the terminal in the vehicle acquires an image 610 in front of the vehicle, recognizes 620 a roadside traffic device from the captured image, and recognizes 620 the display information of the recognized roadside traffic device. The roadside traffic system includes traffic signboards (VMS) and traffic lights. Interlaced scanning optical character recognition (OCR) technology, image warping technology, etc. can be used to improve the accuracy and speed of recognition of the display information of the roadside traffic apparatus. Next, it is determined whether the recognized display information is information for notifying a dangerous situation (step 640). If the detected dangerous situation is a dangerous situation, a danger warning signal is generated and outputted to the car (650). In contrast, if it is not a dangerous situation, the display information is converted into traffic information and the converted traffic information is output in the car (660). The danger warning signal and traffic information may be output in the form of at least one of a voice signal, a video signal, and a vibration signal. At the same time as the output, the in-vehicle terminal transmits traffic information or a danger warning signal to the control server or transmits it to the nearby vehicle (670). The control server can transmit the received traffic information or a danger warning signal to the surrounding vehicles.

The embodiments of the present invention have been described above. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

1: intelligent traffic system 2: vehicle terminal
3: Road traffic device 4: Control server
5: Network device 20: Input
22: control unit 24: output unit
26-1: first communication unit 26-2: second communication unit
26-3: third communication unit 28: memory
220: Preprocessing section 222: Image analysis section
222-1: first image analyzing unit 222-2: second image analyzing unit 222-2:
226: Signal conversion unit 224: Image warping unit
2240: vertical intersection position calculation unit 2241: vertical displacement calculation unit
2242: Corresponding pixel position calculation unit 2243: Distance calculation unit
2244: Weight calculation unit 2245: Cumulative calculation unit
2246: input image pixel position calculation unit

Claims (11)

A road side traffic device including at least one of a traffic signboard and a traffic light; And
An in-vehicle terminal for recognizing information displayed from a roadside traffic apparatus and processing and outputting the recognized display information; / RTI &gt;
The in-
An input unit for acquiring an image photographed by the vehicle;
A controller for recognizing display information of the roadside traffic apparatus from the obtained image, converting the recognized display information into traffic information, determining a dangerous situation from the display information, and generating a danger warning signal;
An output unit for outputting traffic information or a danger warning signal in the vehicle;
A first communication unit for delivering traffic information or a danger warning signal to the control server or receiving the signal from the control server; And
A second communication unit for transmitting traffic information or a danger warning signal to the surrounding vehicles or receiving the signals from the surrounding vehicles; / RTI &gt;
The control unit
A preprocessor for removing noise of image data sequentially input frame by frame and extracting a region of interest;
A first image analyzer for separating the color space in the extracted ROI and recognizing the display information of the roadside traffic device through at least one of color, position and shape of the color signal from the separated color space;
A second image analyzing unit for recognizing display information of the roadside traffic apparatus through optical character recognition on the extracted ROI;
A signal converter for converting the recognized display information through at least one of the first image analyzer and the second image analyzer into a traffic information or a danger warning signal; And
The image processing unit performs image warping in which the movement of the position is shifted for each pixel of the preprocessed input image frame data, thereby interpolating pixels in the frame to generate reconstructed image data from which the error has been eliminated, An image warping unit provided to the image analysis unit;
And an intelligent traffic system. The method according to claim 1,
The output unit transmits traffic information or a danger warning signal to a user in the vehicle,
The first communication unit provides the traffic information or the danger warning signal to the control server through the network and transmits it to the surrounding vehicles via the control server,
And the second communication unit transmits the traffic information or the danger warning signal directly to the neighboring vehicle. delete The apparatus of claim 1, wherein the second image analyzing unit
Characterized by scanning the area of interest of the image frame data in an interlaced scanning manner and recognizing the display information of the roadside traffic device through optical character recognition (OCR). delete The apparatus of claim 1, wherein the image warping unit
A vertical intersection position calculation unit for calculating a position u of a vertical intersection point of the input image pixel V (x, y) and the control line Li with respect to each control line Li with respect to each pixel V (x, y) of the input image;
A vertical displacement calculation unit for calculating a vertical displacement h of the input image pixel V (x, y) and the control line Li;
A corresponding pixel position calculation unit for obtaining a corresponding position V '(x', y ') in the reconstructed image using the position u of the vertical intersection and the vertical displacement h;
A distance calculation unit for calculating a distance d between the input image pixel V (x, y) and the control line Li;
A weight calculation unit for calculating a weight of the control line using the distance d between the input image pixel V (x, y) and the control line Li;
An accumulation unit for obtaining a displacement value between the input image pixel V (x, y) and the reconstructed image pixel V 'and calculating the displacement accumulation (t _x , t _y ) thereof; And
The position of the restored image pixel V (X, Y) corresponding to the input image pixel V (x, y) is calculated using the calculated displacement accumulation (t _x , t _y ) and the weight of the control line, , Y) into an input image V (x, y) pixel;
And an intelligent traffic system. 2. The apparatus of claim 1, wherein the signal conversion unit
And converts the display information into text information indicating a traffic situation or a dangerous situation, and converts the converted text information into an output signal corresponding to at least one of a voice signal, a video signal, and a vibration signal, and provides the output signal to the output unit. system. 8. The method of claim 7,
Wherein the output signal is activated in a vehicle steering wheel or in a vehicle chair. The method according to claim 1,
A third communication unit for transmitting traffic information or a danger warning signal to the traffic light sign board; Further comprising:
Wherein the information transmitted from the in-vehicle terminal to the traffic light board is information obtained by processing the information obtained from the display information of the traffic light board different from the information obtained from the display information of the traffic light board in the in-vehicle terminal. delete delete

Images