KR101199959B1 - System for reconnizaing road sign board of image - Google Patents

Abstract

PURPOSE: A system for recognizing a road sign board in an image is provided to automatically extract a road sign board using color and shape features, thereby recognizing road sign board information. CONSTITUTION: An input unit receives an operation signal from a user. An output unit displays processed data on a screen. A communication interface communicates with an external device through a communication network. A road sign board recognizing unit recognizes road sign board information by analyzing the generated image information. A road sign board information providing unit provides the recognized road sign board information to an image service device. [Reference numerals] (100) Image space information building device; (110) Central control unit; (120) Image information collection unit; (121) Camera; (122) IMU; (123) GPS; (130) Post processing unit; (131) Brightness control module; (132) Image enforcing module; (133) Image attribute information generation module; (140) Communication interface; (150) Database; (AA) Image file; (BB) Image attribute information

Description

Traffic Sign Recognition System in Image {SYSTEM FOR RECONNIZAING ROAD SIGN BOARD OF IMAGE}

The present invention relates to a traffic sign recognition system, and more particularly, to a traffic sign recognition system in an image for automatically extracting and recognizing a traffic sign using color and morphological characteristics in an image acquired through photographing. will be.

In recent years, as information processing and communication technologies are rapidly developed, a method of constructing geographic information as map information data and using it for transportation has been widely used. The map information data is produced based on the geographic information that is actually photographed, and is used for navigation, safety driving, and automatic driving through navigation.

Navigation using the map data is an automatic vehicle navigation device, which provides a voice and monitor screen to provide directions to a destination using a GPS that detects the current location using satellites. Such a navigation device includes a GPS receiver receiving the vehicle's current location information from a GPS satellite, a map DB in which map information such as a traffic sign is registered, and map information according to the current location identified through the GPS receiver. And an operating program for displaying on the screen.

Map information constructed in the map DB of the navigation is generated through measurement of actual geographic information. Since such map information changes according to the establishment of a road or a change of traffic system, the map information is updated regularly or irregularly. shall. The update of the map information of the navigation is usually performed by hand, especially since the traffic signs are constantly changing, the map manager frequently checks the traffic signs while moving to a location on the map, and then checks the traffic signs included in the map information. There was a cumbersome problem of updating information.

In addition, a safety driving device and an automatic driving system applied to a vehicle in recent years are a system for automatically driving a vehicle on behalf of a person. Such an automatic driving system should automatically recognize various traffic signs required for safe driving. The results should be used to cope with various situations.

As such, there has been a demand for a system that automatically extracts traffic signs from images and recognizes them quickly and accurately in navigation, vehicle safety devices, and automatic driving systems.

The present invention has been proposed for automatically extracting and recognizing a traffic sign included in an image so that it can be applied to the automatic updating of a navigation map or the safe driving of an automatic driving system. The present invention provides a traffic sign recognition system for automatically extracting and recognizing a traffic sign included in an acquired image using color and shape characteristics.

In order to achieve the above object, a traffic sign recognition system in an image according to the present invention is a traffic sign recognition system for recognizing a traffic sign included in an image. Wow; Color and shape are separated according to the color and shape characteristics of objects in the image included in the image space information data produced by the image space information construction device, and traffic signs are separated through the color and shape information of the separated objects. It comprises a; traffic sign recognition device for identifying and recognizing.

The apparatus for recognizing a traffic sign includes an image viewer module for selecting a target image of a traffic sign recognition image from the image space information data produced by the image space information building device and displaying the image on the screen, and the color characteristics of the objects included in the selected image. And a sign extraction / recognition module for separating colors according to the shape characteristics of the object in which the color is separated and recognizing the object as a traffic sign through color and shape information of the separated object, and extracting the sign / A traffic sign recognition unit including a result display module for displaying the traffic sign information recognized through the recognition module on the screen is provided.

Here, the sign extraction / recognition module sets an HSV value corresponding to the border and the inner background color of each traffic sign using an HSV color space model, and then uses the HSV value to determine the color of the object from the image to be recognized according to the set HSV value. HSV values corresponding to the border and the inner background color of the traffic sign are set as shown in the following table for each color of the image to be recognized by the Ladybug3 camera of Point Gray Research Co., Ltd. It is preferable.

(0≤color (H) ≤360, 0≤saturation (S) ≤255, 0≤brightness (V) ≤255)

Meanwhile, the sign extraction / recognition module detects the Top / Bottom Component and the Top Line and the Bottom Line of the object whose color is separated, identifies the horizontal state of the detected Top / Bottom Line, and identifies the top / bottom as the horizontal state. The shape of the traffic sign is extracted by classifying the shape of the object according to the number of bottom lines. The top / bottom line is considered to be horizontal when the top / bottom line does not exceed ± 30 ° from the horizontal line. It is preferable to be. Here, the traffic sign is in the form of a circle when the number of Top / Bottom Lines identified as the horizontal state is 0, and in the form of a rectangle when the number of the Top / Bottom Lines identified as the horizontal state is 2, it is identified as the horizontal state. If the number of bottom lines is 1, it is classified as a triangle and if the number of top lines that are recognized as horizontal is 1, it is classified into an inverted triangle and extracted.

In addition, the sign extraction / recognition module detects the Top / Bottom Component and the Top Line and the Bottom Line of the object included in the image to be recognized, and identifies the number and the parallel state of the detected Top / Bottom Line, thereby determining the Top / Bottom Line. The shape of the traffic sign can be extracted by classifying the shape of the object according to the number and parallel state. Here, the traffic sign is in the form of a triangle when there is only one bottom line and no parallel top line exists, and in the form of an inverted triangle when only one top line exists and there is no bottom line parallel thereto. If Top Line and Bottom Line exist in pairs and are parallel to each other, they are classified into a rectangular form and if there is no Top / Bottom Line, they are classified and extracted.

On the other hand, the traffic sign recognition device is provided with a traffic sign information providing unit for providing the traffic sign information recognized through the traffic sign recognition unit to the video service device that requires the traffic sign information for safe driving. The image service device that receives the traffic sign information from the study may be a map DB in which the navigation map information is stored or a safe driving device that performs automatic driving.

Meanwhile, the apparatus for constructing image space information includes an image information collecting unit which collects image information by capturing an image through a camera while moving along an image collecting copper line, and adjusts the brightness of the image collected through the image information collecting unit and adjusts the brightness within the image. It includes a post-processing unit for classifying objects.

Here, the image information collecting unit includes an IMU (Inertial Measurement Unit) and a GPS (Global Positioning System) for measuring posture information and position information of an image photographing a camera, and the post-processing unit adjusts the brightness and the object It is preferable that an image attribute information generation module is provided to generate image attribute information by matching posture information and position information measured through the IMU and GPS with the image information divided by the IMU and the GPS.

In the traffic sign recognition system according to the present invention, it is possible to automatically recognize and recognize the traffic sign by using the color characteristics and the morphological characteristics of the objects included in the image, thereby enabling rapid and accurate traffic sign recognition in the image. There is an effect that can be applied to the automatic update of the map of the vehicle or the vehicle's safe driving system and automatic driving system.

1 is an overall network connection diagram of a traffic sign recognition system in an image according to the present invention;
FIG. 2 is a block diagram of a video spatial information construction apparatus according to the present invention.
3 is a block diagram of a traffic sign recognition apparatus according to the present invention;
Figure 4 is an example of the type of traffic signs recognized through the traffic sign recognition apparatus according to the present invention,
5 is an example of an image having a traffic sign recognized through the traffic sign recognition apparatus according to the present invention,
Figure 6 is an example HSV color space model of the cylinder model applied to the present invention,
7 illustrates an example of a screen in which colors are separated from an image according to the present invention;
8 is a flowchart illustrating a process of classifying a morphological characteristic of an object through traffic sign classification criteria and conditions according to an embodiment of the present invention;
9 to 12 are examples of a case in which the morphological characteristics of the objects extracted according to the embodiment of the present invention are triangles, inverted triangles, rectangles, and circles;
FIG. 13 is a flowchart illustrating a process of classifying a characteristic of an object through traffic sign classification criteria and conditions according to another embodiment of the present invention; FIG.
14 to 17 illustrate an example in which the morphological characteristics of an object extracted according to another embodiment of the present invention are triangles, inverted triangles, rectangles, and circles;
18 is a flowchart illustrating a process of generating image space information through the apparatus for constructing image space information according to the present invention;
19 is a flowchart illustrating a process of extracting and recognizing a traffic sign in an image through the traffic sign recognition apparatus according to the present invention;
20 illustrates an example of a recognition target image displayed on a screen through the image viewer module of the traffic sign recognition unit according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an overall network connection of a traffic sign recognition system in an image according to an exemplary embodiment of the present invention.

As shown in FIG. 1, the traffic sign recognition system according to the present invention uses an image space information construction device 100 for constructing image space information data including a traffic sign and the image space information construction device 100. And a traffic sign recognition apparatus 200 which extracts and recognizes a traffic sign image included in the constructed image spatial information. Meanwhile, the traffic sign information recognized by the traffic sign recognition apparatus 200 may be provided to the image service apparatus 300 requiring the traffic sign information.

The image space information construction device 100 captures an image through an image capturing device such as a camera and stores the image as a file, and captures and registers attribute information such as posture information and position information according to the captured image in a database. Information acquisition device.

The traffic sign recognition apparatus 200 receives image information from the image space information construction apparatus 100 and automatically extracts and recognizes traffic signs included in the image information by classifying them into color characteristics and morphological characteristics. The traffic sign recognition apparatus 200 may be operated as an independent server computer, or may be installed and operated subordinately to the image service apparatus 300.

The video service device 300 is a device that receives the traffic sign information recognized from the traffic sign recognition device 200 and utilizes it. An example of the video service device 300 includes a map DB in which navigation map information is stored. 310, a safety driving device 320 such as an automatic driving system, and the like. The map information registered in the map DB 310 is automatically updated and managed with the traffic sign information recognized by the traffic sign recognition device 200, and the safe driving device 320 is operated by referring to the traffic sign information. It will cope with various situations that occur.

2 is a block diagram of an apparatus for constructing image spatial information according to an embodiment of the present invention.

As shown in FIG. 2, the apparatus 100 for constructing image space information according to the present invention includes an image information collecting unit 120 for collecting image information, and image information collected through the image information collecting unit 120. The communication interface 140 sets up communication with the post-processing unit 130 and the traffic sign recognition device 200 to post-process and transmits the image information processed through the post-processing unit 130 to the traffic sign recognition device 200. And a database 150 for storing the image information processed by the post processor 130 and a central controller 110 for controlling the operation of each component.

The image information collecting unit 120 is installed in a vehicle to photograph an image according to the movement of the vehicle 121 and an IMU (Inertial Measurement Unit) 122 to grasp the attitude information of the vehicle at the time the image is stored. And a GPS (Global Positioning System) 123 for detecting position information of the vehicle at the time when the image is stored. The collection vehicle equipped with the camera 121, the IMU 122, and the GPS 123 is moved according to the planned collection line, and the camera 121, the IMU 122, and the GPS 123 are moved according to the movement of the vehicle. The image and posture information and position information are collected and transmitted to the post processor 130. In the embodiment of the present invention, the ladybug3 camera of Point Gray Research Co., Ltd. is applied to the camera 121, and the image captured by the ladybug3 camera is processed through the post-processing unit 130 and then to the traffic sign recognition apparatus 200. Is sent. On the other hand, the IMU 122 and the GPS 123 provided in the image information collecting unit 120 should include posture information and location information in the image captured by the camera 121 (for example, a navigation map Image information to update the video, etc.), but not in the case of simply grasping the contents of the image in real time (e.g., image information to detect and warn the traffic sign in real time for safe driving). You may not. Accordingly, the IMU 122 and the GPS 123 may be provided or omitted in the image information collecting unit 120 according to the type of the image captured and utilized by the camera 121.

In the present invention, since the traffic sign is automatically extracted and recognized in the captured image, the quality of the image becomes a very important factor. To this end, the post-processing unit 130 post-processes the image information collected through the image information collecting unit 120 to improve the quality of the image and store it in the database 150. The post-processing unit 130 includes an image A brightness control module 131 for adjusting the brightness of the image, an image enhancement module 132 for distinguishing the objects included in the image, and an image property information generation module 133 for generating the property information of the image. The brightness control module 131 is a program module for adjusting the brightness of the image to accurately extract and recognize the traffic sign in the image obtained by the camera 121 of the image information collecting unit 120, image enhancement The module 132 is a program module for distinguishing between objects in the acquired image. In addition, the image attribute information generation module 133 matches the posture information of the IMU 122 and the position information of the GPS 123 one-to-one with the image information undergoing a post-processing process in which the brightness is adjusted and the objects are separated. Program module for generating attribute information. The image file and the image property information are generated through the post-processing unit 130, and the generated image file and the image property information are finally registered in the database 150. Table 1 below shows the image property generated through the post-processing unit. The information is presented in a table.

The communication interface 140 establishes communication with the traffic sign recognition apparatus 200 and transmits the image file and image property information registered in the database 150 to the traffic sign recognition apparatus 200.

The database 150 is a storage device for storing information generated by the image information collecting unit 120 and the post-processing unit 130. The database 150 includes image files and images generated by the post-processing unit 130. Attribute information and the like are stored and managed.

3 is a block diagram of a traffic sign recognition apparatus according to an embodiment of the present invention.

As shown in FIG. 3, the traffic sign recognition apparatus 200 according to the present invention displays an input unit 220 for receiving an operation signal from a user and data processed through the traffic sign recognition device 200 on a screen. Traffic to extract and recognize traffic sign information by analyzing the image information generated through the output unit 230, the communication interface 240 for communicating with the external device through the communication network, and the image space information construction device 100 A traffic sign information providing unit 260 for providing the image service device 300 with traffic sign information recognized through the sign recognition unit 250, the traffic sign recognition unit 250, and the traffic sign recognition device 200. It includes a database 270 that stores the data to be processed through the) and the central control unit 210 for controlling the operation of each component.

The input unit 220 is a data input device such as a mouse or a keyboard that receives operation data from a user, and the output unit 230 is a monitor or a printer that displays data processed by the traffic sign recognition device 200 on a screen. Output device. In addition, the communication interface 240 is a communication device that establishes communication with external devices such as the image space information construction apparatus 100 and the image service apparatus 300 to transmit and receive data.

The traffic sign recognition unit 250 receives image information from the image space information construction apparatus 100 and extracts and recognizes traffic sign information included in the image information. The traffic sign recognition unit 250 includes an image viewer module 251 for selecting a traffic sign recognition target image and displaying it on a screen, and a sign extraction / recognition module for automatically extracting and recognizing a traffic sign included in the recognition target image ( 252 and a result display module 253 for displaying traffic sign information recognized by the sign extraction / recognition module 252 on the screen. In the embodiment of the present invention, the sign extraction / recognition module 252 analyzes the color characteristics and the morphological characteristics of the objects included in the recognition target image displayed on the screen by the image viewer module 251 to analyze the traffic sign image. The extracted traffic sign information is displayed and displayed on the screen through the result display module 253 so that the user can check it.

The traffic sign information providing unit 260 is a program provided to the image service device 300 to utilize the traffic sign information recognized by the traffic sign recognition unit 250. The traffic sign information providing unit 260 converts the traffic sign information recognized through the sign extraction / recognition module 252 into map information and transmits the map information to the map DB 310 of the image service apparatus 300 to transmit map information. The map DB updating module 261 to update and the safe driving provision module 262 which transmits traffic sign information to the safe driving device 320 are provided.

Meanwhile, the traffic sign information extracted and recognized by the traffic sign recognition unit 250 and provided to the image service apparatus 300 through the traffic sign information providing unit 260 is registered and managed in the database 270.

Hereinafter, a method of extracting and recognizing a traffic sign included in an image through the sign extraction / recognition module 252 of the traffic sign recognition unit 250 will be described.

In general, traffic signs are classified into caution signs, regulatory signs, and indication signs. FIG. 4 shows an example of a type of such traffic signs, and FIG. 5 shows an image with a traffic sign. The traffic sign recognition unit 250 needs to distinguish between the traffic sign represented in the image and information other than the traffic sign in order to recognize the traffic sign in the image.

In the embodiment of the present invention, the traffic sign recognition unit 250 recognizes the traffic signs in the image by separating them into shape and color characteristics, and the characteristics of each sign are shown in Table 2 below. Same as

As shown in Table 2, the sign extraction / recognition module 252 of the traffic sign recognition unit 250 extracts the traffic sign image by using the shape and color characteristics of the traffic sign in the image. The image is recognized.

In the embodiment of the present invention, in order to extract and recognize the traffic signs in the image, the characteristics of the color are separated using the HSV color space in the image, and then the shape of the traffic signs that can be extracted from each color using the morphological characteristics. It will separate the characteristics. Thereafter, a recognition target image extracted by using color and morphological characteristics is input to a recognizer such as a neural network or SVM, and the target region is recognized according to a pattern previously learned by the recognizer.

Hereinafter, a process of recognizing the traffic sign using the color and the morphological features of the traffic sign will be described in detail.

First, a method of separating color characteristics using the HSV color space in an image will be described.

In general, since the HSV color space in the image is slightly different according to the characteristics of the camera, it is difficult to limit the range for each color separation in the HSV color space. Therefore, in the present invention, after defining the HSV color space in the image obtained through a specific camera, it sets the color range for color separation in the defined HSV color space to separate the color characteristics through this. In the embodiment of the present invention, the Ladybug3 camera of Point Gray Research Co., Ltd. is used as the specific camera. Hereinafter, HSV color capable of detecting the area of the traffic sign from the digital image acquired using the Ladybug3 camera of Point Gray Research Co. It explains from defining space.

In general, the HSV color space is a way of expressing color, and by arranging colors according to the method, a specific color is designated by using coordinates of Hue, Saturation, and Brightness. Done. The HSV is also called HSB, and this HSV color space is constructed as follows.

-Hue: The color value H means the relative placement angle when the longest blue wave is 0 ° in the color wheel where the visible light spectrum is arranged in a ring shape. Therefore, the H value ranges from 0 ° to 360 ° and 360 ° and 0 ° refer to the same color blue.

-Saturation: The saturation value S represents the degree of concentration when the darkest state of a specific color is 255. A saturation value of 0 represents achromatic colors of the same brightness.

-Brightness (Value, Brightness): Brightness value V or B indicates the degree of brightness when white is set to 255 and black is set to 0.

These values may be mapped to other ranges of values as necessary.

The model representing the HSV color space includes a three-dimensional shape of a cylinder or an inverted conical shape. In an embodiment of the present invention, a range is set using a cylindrical model. Figure 6 shows an example of the HSV color space model of the cylinder model applied to the present invention.

As shown in FIG. 6, the HSV color space is represented by a cylinder according to the definition of the HSV color space cylinder model component. One point on the surface and the inside of the cylinder represents one color. Here, the color value is expressed in degrees and specifies in which direction the specified color is located in the horizontal cross section of the cylinder. In addition, the saturation corresponds to the radius, the achromatic color is located in the center and the outer surface of the cylinder has the darkest saturation. Brightness corresponds to height and becomes brighter up.

Each RGB value in the cylindrical model is scaled in a range of 0 to 1 and converted into H, S, and V values by the following equation (1).

Through Equation 1, H, S, and V values in the range of 0 ≦ V ≦ 1, 0 ≦ S ≦ 1, and 0 ≦ H ≦ 360 can be obtained. In the embodiment of the present invention, the range of the H, S, V values is scaled to a range such as V ← 255 V, S ← 255 S, and H ← H / 2.

The HSV color space can be used as a border color and an internal background color that a traffic sign can have in order to detect an object from a domestic traffic sign in a digital image. The types, shapes, and colors of the traffic signs are described in Table 2 have. According to the traffic signs classified in Table 2, five types (triangle, circle, inverted triangle, octagon, square) of the traffic sign frame and the interior background color are selected, and four kinds of colors (white, red, Blue, yellow) is finally selected to determine the optimal HSV values corresponding to each color. Table 3 below shows an example of HSV values finally selected according to the embodiment of the present invention.

Colors are separated from a given image using the HSV values shown in Table 3, and FIG. 7 illustrates an example of a screen from which colors are separated from an image.

When colors are separated in the image using the HSV color space as described above, the sign extraction / recognition module 252 uses the morphological characteristics of the traffic signs to extract traffic signs from each of the separated color channels.

As classified in Table 2, the traffic signs have the form of triangles (including inverted triangles), circles (including octagons), and quadrangles. A method of extracting traffic signs from color channels using image features By classifying objects or regions of interest divided into colors or edges based on the user's purpose from the image, triangular, inverted triangle, square, and circular forms are simple. In digital images, objects with simple shapes such as road signs and road facilities can be safely and efficiently identified.

In general, the simple shapes of triangles, inverted triangular shapes, squares, and circles to be classified can be determined by using the number of line segments or vertices that constitute each figure. Same as

However, this approach does not always clarify the process of extracting objects based on color or edge information from images and finding elements such as segments and vertices using Hough Transform and least-squares techniques. There is this. In other words, the edge of an object may be sharpened or unclear due to the camera's performance, or even partial differences may appear. In addition, in the process of finding a line segment or a vertex from a color or an edge, the result may be different depending on the threshold setting and environmental factors. This means that a process of detecting a component such as a straight line or a vertex by using an image processing technique from an object extracted from an image includes instability.

Therefore, in the present invention, in order to minimize the instability of the general approach, the criteria used to classify the shape of the object are not based on the number of elements constituting the figure, but a new standard that can represent the morphological characteristics of the object Top / Bottom Component and Top / Bottom Line are defined and used.

Table 5 below is a diagram defining these Top / Bottom Component and Top / Bottom Line.

In an embodiment of the present invention, one of a method of classifying the morphological characteristics of an object that satisfies the definition of Table 5 is to use a method of classifying shapes using only line segments determined to be horizontal or close to horizontal among Top / Bottom Components. These classification criteria and conditions are shown in Table 6 below.

8 is a flowchart illustrating a process of classifying the morphological characteristics of an object through the classification criteria and conditions shown in Table 6, and FIG. 9 is a triangular characteristic of the object. In the case of an inverted triangle, FIG. 11 illustrates an example in which a shape characteristic of an object is a rectangle, and FIG. 12 illustrates an example in which a shape characteristic of an object is a circular shape.

As shown in Table 6 and FIGS. 8 to 12, a line which is determined to be horizontal or close to horizontal in a Top / Bottom Components configuration of an object (for example, within ± 30 ° based on a horizontal line, hereinafter referred to as a horizontal line) The number of horizontal lines is classified into a circle if the number of horizontal lines is 0, and a rectangle if the number of horizontal lines is two. In addition, when the number of horizontal lines is 1, the bottom line is identified, and if there is a horizontal line, it is classified as a triangle, and if there is no horizontal line, it is classified as an inverted triangle. As described above, according to the exemplary embodiment of the present invention, the morphological characteristics of the object are classified through the number of Top / Bottom Lines.

Meanwhile, the method described above in Table 6 and FIGS. 8 to 12 has a condition for generating a Top / Bottom Line and determining whether there is one. In this method, when the object to be classified is rotated more than a certain range, a problem arises in that Top / Bottom Line cannot be generated. That is, when the target object is a rectangle and is rotated more than 30 degrees, the rectangle is classified as a circle because the method cannot generate a top / bottom line.

Therefore, in order to solve this problem, in another embodiment of the present invention, a top / bottom line is formed regardless of an angle, and the shape is classified by evaluating the number of line segments and the parallel information at the same time. These classification criteria are shown in Table 7 below.

FIG. 13 is a flowchart illustrating a process of classifying the morphological characteristics of an object through the classification criteria and conditions shown in Table 7; FIG. 14 is a triangular characteristic of the object; FIG. In the case of an inverted triangle, FIG. 16 illustrates an example in which the object has a circular characteristic shape, and FIG. 17 illustrates an example in which the object has a circular shape.

As shown in Table 7 and FIGS. 13 to 17, the number of parallel lines is determined in the configuration of the Top / Bottom Components of the object. Will be classified. In addition, when the number of parallel lines is one, if the bottom line is a parallel line, it is classified as a triangle, and if the bottom line is not a parallel line, it is classified as an inverted triangle.

As described above, in another embodiment of the present invention, the morphological characteristics of the object are classified through the number of top / bottom lines and parallel information.

When the color characteristics are separated in the image through the above process and the morphological characteristics of the object for the separated color channel are classified, the sign extraction / recognition module 252 recognizes the object information in which the color characteristics and the morphological characteristics are classified. The process of recognizing traffic signs is performed by typing in. That is, the sign extraction / recognition module 252 inputs a recognition target image classified with color and form characteristics into the recognizer. Accordingly, the recognizer provided in the sign extraction / recognition module 252 is input according to the learned pattern. The target area is recognized as a traffic sign. In an embodiment of the present invention, a recognizer such as a neural network or a support vector machine (SVM) is applied to the sign extraction / recognition module 252.

When the traffic sign information is extracted and recognized by the sign extraction / recognition module 252 through the color and shape characteristics of the object included in the image, the result display module 253 is the sign extraction / recognition module 252. Traffic sign information recognized by) is displayed on the screen so that the user can check it.

On the other hand, the traffic sign information in the image recognized by the traffic sign recognition unit 250 is provided through the traffic sign information providing unit 260 as a device requiring the traffic sign information is utilized.

Hereinafter, a process of recognizing a traffic sign through the image space information construction device and the traffic sign recognition device having the above configuration will be described.

18 is a flowchart illustrating a process of generating image space information through the apparatus for constructing image space information according to an embodiment of the present invention.

As shown in FIG. 18, first, when data collection copper lines for constructing image space information are planned (S110), the image information collecting unit 120 moves along the collection lines to collect image spatial information (S120). The collected image spatial information is subjected to a post-processing process such as image brightness adjustment, image enhancement, and image property information generation through the post-processing unit 130 (S130). The image space information post-processed by the post processor 130 is stored in the database 150 as an image file and image property information, thereby finally constructing image space information data.

This series of processes is an example of a case in which the posture information and the position information are included in the image spatial information data. If the posture information and the position information are unnecessary in the image spatial information data, the posture information and the position information are collected and the image attribute information is obtained. The process of generating can be omitted.

19 is a flowchart illustrating a process of extracting and recognizing traffic sign information in an image through a traffic sign recognition apparatus according to an exemplary embodiment of the present invention.

Step S210: First, the traffic sign recognition unit 250 provided in the traffic sign recognition apparatus 200 selects and inputs an image to be recognized from the image information generated through the image space information construction apparatus 100. The input image is displayed on the screen through the image viewer module 251. 20 illustrates an example of a recognition target image displayed on a screen through an image viewer module of a traffic sign recognition unit according to an exemplary embodiment of the present invention.

Steps S220, S230, and S240: When the recognition target image is input, the sign extraction / recognition module 252 of the traffic sign recognition unit 250 first performs color separation of the objects included in the image. Separation is performed using the HSV values of Table 3 described above (S220). In addition, when color separation of an object in the image is made, the sign extraction / recognition module 252 separates the shape of the object from each of the separated color channels, and the shape separation of the object is described in Table 6 or Table 7 above. It is separated using the morphological characteristics of the traffic sign according to (S230). When the color characteristics and the morphological characteristics of the objects in the image are separated through the above process, the sign extraction / recognition module 252 analyzes the object information that is color separation and shape separation through the learned recognizer and recognizes the information as traffic signs. It becomes (S240).

Step 250, S260: traffic sign information recognized by the sign extraction / recognition module 252 is displayed on the screen through the result display module 253 (S250), such traffic sign recognition process until there is no image input It is repeated (S260).

Meanwhile, the traffic sign information recognized through the sign extraction / recognition module 252 of the traffic sign recognition unit 250 may be the map DB 310 of the image service apparatus 300 or the traffic sign information providing unit 260. It is transmitted to the safe driving device (320).

As described above, the traffic sign recognition system according to the present invention generates the image space information through the image space information construction apparatus 100, and extracts and recognizes the traffic sign included in the image through the traffic sign recognition apparatus 200. The traffic sign information to be recognized may be provided to the video service apparatus 300 that needs the information to use the information.

The present invention is not limited to the above-described embodiments and various modifications and variations within the equivalent range of the technical spirit of the present invention and the claims to be described below by those skilled in the art to which the present invention pertains. Of course this can be done.

100: image space information construction device 110: central control unit
120: image information collecting unit 121: camera
122: IMU 123: GPS
130: post-processing unit 131: brightness control module
132: image enhancement module 133: image attribute information generation module
140: Communication interface 150: Database
200: traffic sign recognition device 210: central control unit
220: input unit 230: output unit
240: communication interface 250: traffic sign recognition unit
251: Image viewer module 252: Sign extraction / recognition module
253: result display module 260: traffic sign information provider
261: map DB update module 262: module for providing safe operation
270: database 300: video service device
310: map DB 320: safe driving device

Claims (13)

delete delete delete Selects a traffic sign recognition target image from the image space information data produced by the image space information construction apparatus 100, separates colors according to the color characteristics of the objects included in the selected image, and the color separated object. Traffic comprising a traffic sign recognition device 200 is provided with a traffic sign recognition unit 250 for separating the form according to the morphological characteristics of the object, and recognizes the object as a traffic sign through the color and shape information of the separated object. In the sign recognition system,
The traffic sign recognition unit 250 sets an HSV value corresponding to the border and the inner background color of each traffic sign using an HSV color space model, and then separates the color of the object from the image to be recognized according to the set HSV value. But
The HSV value corresponding to the border and the inner background color of the traffic sign is set as shown in the following table for each color of an image captured by a Ladybug3 camera manufactured by Point Gray Research Co., Ltd. based on a camera setting. Sign Recognition System.

Here, 0≤color (H) ≤360, 0≤saturation (S) ≤255, 0≤brightness (V) ≤255 Selects a target image of a traffic sign recognition image from the image space information data produced by the image space information construction apparatus 100, separates colors according to the color characteristics of the objects included in the selected image, and the object of which the color is separated. Traffic comprising a traffic sign recognition device 200 is provided with a traffic sign recognition unit 250 for separating the form according to the morphological characteristics of the object, and recognizes the object as a traffic sign through the color and shape information of the separated object. In the sign recognition system,
The traffic sign recognition unit 250 detects the Top / Bottom Component and the Top Line and the Bottom Line of the object whose color is separated, identifies the horizontal state of the detected Top / Bottom Line, and identifies the top / bottom as the horizontal state. The shape of the traffic sign is extracted by classifying the shape of the object according to the number of bottom lines.
If the number of Top / Bottom Lines identified in the horizontal state is 0, it is in a circular form. If the number of Top / Bottom Lines identified in the horizontal state is 2, in a rectangle form, the number of Bottom Lines identified in the horizontal state is The traffic sign recognition system, characterized in that the triangular form in the case of one, the number of top lines to be identified in the horizontal state is classified into an inverted triangle form and extracted. 6. The method of claim 5,
The Top / Bottom Line is a traffic sign recognition system, characterized in that it is determined that the top / Bottom Line is in a horizontal state if it does not exceed ± 30 ° relative to the horizontal line. delete delete Selects a target image of a traffic sign recognition image from the image space information data produced by the image space information construction apparatus 100, separates colors according to the color characteristics of the objects included in the selected image, and the object of which the color is separated. Traffic comprising a traffic sign recognition device 200 is provided with a traffic sign recognition unit 250 for separating the form according to the morphological characteristics of the object, and recognizes the object as a traffic sign through the color and shape information of the separated object. In the sign recognition system,
The traffic sign recognition unit 250 detects the Top / Bottom Component, the Top Line and the Bottom Line of the object included in the recognition target image, grasps the number and the parallel state of the detected Top / Bottom Line, and determines the Top / Bottom Line. The shape of the traffic sign is extracted by classifying the shape of the object according to the number of and parallel state.
The traffic sign is triangular when there is only one bottom line and no parallel top line exists. In case of only one top line and there is no parallel bottom line, the traffic sign has an inverted triangle shape. A traffic sign recognition system, characterized in that the bottom line is present in pairs and parallel to each other in the form of a rectangle, if the Top / Bottom Line does not exist, classified into a circular form. delete The method according to any one of claims 4, 6 and 9,
The traffic sign recognition device 200
Provide traffic sign information recognized through the traffic sign recognition unit 250 to the image service device 300 including a map DB 310 or a safety driving device 320 that requires traffic sign information for safe driving. Traffic sign recognition system characterized in that the traffic sign information providing unit 260 is provided. delete The method according to any one of claims 4, 6 and 9,
The video spatial information construction apparatus 100 includes:
An IMU (Inertial Measurement Unit) 122 and a GPS (Global) for capturing an image through a camera 121 while moving along an image collection copper line, and measuring posture information and position information of an image captured by the camera 121. An image information collecting unit 120 including a positioning system 123;
Through the IMU 122 and the GPS 123 to adjust the brightness of the image collected through the image information collecting unit 120, to classify the object in the image, the image information of the brightness is adjusted and the object is divided And a post-processing unit (130) having an image attribute information generation module (133) for generating image attribute information by matching the measured posture information and position information.

Images