KR102133913B1 - Digital display board system using object recognition based on deep learning for providing customized information of object and method for controlling thereof - Google Patents

Abstract

The present invention relates to a guide display board system for providing customized information for each object. According to an embodiment of the present invention, the guide display board system for providing customized information for each object using object recognition based on deep learning comprises: an image photographing unit generating detected image information; an image processing unit separating the background and the object from the detected image information; an object recognition unit classifying objects in accordance with the type; an information collection unit storing information on detected image information and objects; an information processing unit generating an artificial intelligence model based on the detected image information and generating customized information for each object corresponding to the model and the identified object; and an information display unit outputting the customized information for each object.

Description

DIGITAL DISPLAY BOARD SYSTEM USING OBJECT RECOGNITION BASED ON DEEP LEARNING FOR PROVIDING CUSTOMIZED INFORMATION OF OBJECT AND METHOD FOR CONTROLLING THEREOF}

The present invention relates to an information display system and a control method for providing customized information for each object using deep learning-based object recognition, and in particular, performs learning on the surrounding environment based on information collected through various sensing means, and results of learning The present invention relates to an information display board system for providing customized information for each object that can output traffic-related guide information suitable for the type and situation of the identified object.

As artificial intelligence-related technologies such as machine learning, deep learning, and neural networks are newly illuminated, attempts are being made to apply them to various fields. In particular, in the deep mind challenge match in 2016, this trend is accelerating as the game between Alpha Go, the Go game of AI, and Lee Sedol, the representative article of the Republic of Korea, have caused global repercussions.

It is no exaggeration to say that AI technology could be further improved by the advancement of IT technology as well as the evolution of various processing techniques including neural networks. Since detection means already installed in various parts of society, such as CCTV, have already produced a huge amount of data in real time. By using them as data for learning AI, we will be able to create solutions for our safety and daily convenience. It is expected.

Recently, attempts to combine artificial intelligence and safety issues are underway. By providing intelligently to cars or pedestrians based on various information that can be detected on the road, it will be possible to prevent accidents or provide various useful information.

However, the devices that provide traffic-related information nowadays are not intelligent and merely collect information.

On the other hand, as a cause of traffic accidents, various factors such as negligence and drinking can be considered, but it acts as a major cause of overspeed on the road. Currently, there are more and more places where speed is limited due to various needs, such as a child protection area, and usually all roads have their own speed limit. Above all, the driver must make efforts to refrain from speeding on his own, but the speed limit is often not observed, and thus, in terms of traffic safety and accident prevention, a speed measuring device or speed marking facility is installed in a designated area of the road. It is done.

Current speed display facilities simply show the speed of a vehicle approaching the detection area sequentially, and thus are insufficient to draw the user's attention. In addition, speed information is provided for all vehicles passing through the detection area. However, when a speeding vehicle and a vehicle not passing through the detection area at a similar time pass through the detection area, there is a problem in that it is currently impossible to determine which vehicle is output. . In particular, it can be said that it is difficult to substantially induce the speed limit of the vehicle since there is no warning about the vehicle that is continuously driving at the speed.

The present invention is designed to solve the above-mentioned problems, and has an object to provide a guide information display system for providing customized information for each object using deep learning-based object recognition for intelligently providing traffic information, vehicle speed information, and the like. have.

According to an embodiment of the present invention, in a personalized information providing guide electronic board system using deep learning-based object recognition including at least one information providing guide electronic board, the information providing guide electronic board captures an image An image capturing unit that generates information, an image processing unit that performs pre-processing on the detected image information and separates a background and an object from the captured image, recognizes objects separated from the background, and classifies the recognized objects by type Object recognition unit, a surrounding environment detection unit that detects changes in the surrounding environment to generate surrounding environment information, the detected image information, the object classified for each type, and an information collection unit for storing the surrounding environment information, the surrounding environment information, and Information for performing learning and model generation based on at least one of the sensed image information and generating customized information for each object, which is customized information corresponding to the type of the object, based on the identified object and the learning and model generation results. A information display unit for outputting at least one of audio, light, or image of the customized information for each object may be provided, and a guide system for providing customized information for each object using deep learning-based object recognition may be provided.

Here, the object recognition unit divides the recognized object into a pedestrian and a vehicle, and the information processing unit includes the number of pedestrians over time and the number of pedestrians using a pedestrian crossing over time based on information stored in the information collection unit. Pedestrian-related statistics and vehicle-related statistics including the number of cars over time and the average speed of the vehicle over time are calculated, and a traffic information statistical model over time is generated based on the pedestrian-related statistics and the vehicle-related statistics When the pedestrian object is recognized from the detected image information, the pedestrian customized information is generated based on the time when the pedestrian object is recognized and the traffic information statistical model, and the pedestrian customized information is output through the information display unit. And, when the vehicle object is recognized from the detected image information, the vehicle object is generated based on the time and the traffic information statistical model, the vehicle object is recognized, and outputs the vehicle customized information through the information display unit do.

Here, the information processing unit, the traffic information statistics corresponding to the time when the number of cars of the traffic information statistical model corresponding to the time when the object recognized as a pedestrian exceeds a preset limit number of cars, the object recognized as the pedestrian When the average speed of the vehicle of the model exceeds a preset limit vehicle speed, a message prompting attention when using a pedestrian crossing is included in the pedestrian customized information.

Here, the information processing unit, the number of pedestrians using the pedestrian crossing of the traffic information statistical model corresponding to the time when the object is recognized as a vehicle exceeds the preset number of pedestrians using a limited number of pedestrian crossings, or the time when the object is recognized as the vehicle If the number of pedestrians included in the set night time zone and the number of pedestrians in the traffic information statistical model corresponding to the time when the vehicle object is recognized exceeds a preset number of night pedestrians, a message prompting attention to pedestrians in the vehicle customized information To include.

Here, the object recognition unit further divides the pedestrian into an adult and a minor, and the information processing unit includes the number of pedestrians as adults over time, the number of pedestrians as minors over time, and the number of adults using crosswalks over time. Pedestrian-related statistics including the number of minors using the pedestrian crossing according to the generation, and the number of minors as the minors of the traffic information statistical model corresponding to the time when the object being the vehicle was recognized exceeds the preset limit of the minors, and When the number of minors using the pedestrian crossing of the traffic information statistical model corresponding to the time when the object is recognized exceeds the preset number of minors using the limited pedestrian crossing, the vehicle customized information is urged to pay attention to pedestrians who are minor when passing the pedestrian crossing Message.

Here, the external sensing device includes a temperature sensing means and an air quality measurement means, the ambient environment information includes information about the temperature and information about the air quality, the information processing unit, the pedestrian personalized information or the vehicle personalized information Includes information about the temperature and information about the air quality.

Here, the information collecting unit further stores information on a parking stop prohibition time range, which is a range of the prohibition time for parking according to time, and the information processing unit is vehicle number information, which is information on a number of objects that are automobiles, based on the detected image information. Extracting, but, the information processing unit, when the range of the time that the vehicle is an object that has been stopped from the time the vehicle starts to stop is included in the forbidden parking time range, the vehicle being stopped is determined as a parking violation vehicle, Through the information display unit, the vehicle number information of the vehicle violating the parking lot and a message urging the prohibition of the parking lot are also output.

Here, the information display system for providing customized information for each object using the deep learning-based object recognition detects a change in the surrounding environment, generates surrounding environment information, and captures an image of a preset detection area of the road to generate sensed image information. An external sensing device; Providing customized information for each object using the deep learning-based object recognition relays transmission and reception of signals, information, or data between devices included in the information display system, and traffic-related output information based on the surrounding environment information and the detected image information And further comprising a relay server for generating, the information providing guide display panel outputs at least one of the output information and the object-specific customized information, the relay server, based on the detection image information to pass through the detection area At least one object is individually identified, vehicle number information that is information about a vehicle object number is extracted, vehicle speed information that is information about a vehicle object speed corresponding to the vehicle number information is extracted, and the The vehicle number information of at least one vehicle passing through the detection area at a speed exceeding the speed limit of the detection area is selected as respective speeding vehicle number information, and an individual vehicle corresponding to the speeding vehicle number information is detected by the individual vehicle. The vehicle speed information when passing through is selected as respective excess speed information, and the speeding vehicle number information and the excess speed information are output through the information providing information display board.

Here, the information-based information display system for personalized information using object recognition based on deep learning includes a plurality of external sensing devices and a plurality of information providing information display boards, and one external sensing device and one information providing information display board. Forming a road detection group of the, the relay server, when outputting the output information generated based on the detection image information from the external detection device through the information providing guide electronic board included in the same road detection group as the external detection device Output the output information.

Here, the sensing area of the external sensing device is located at a predetermined distance from the information providing information board included in the same group as the external sensing device, and the relay server includes the positioning distance and the excess speed information or vehicle speed information. Based on the information, it is time to start outputting information, and information output start time and time to maintain output of output information are calculated, and the speeding vehicle number information and the excess speed information are calculated through the information providing information display board. When outputting, the information is output from the start time of the information output for the information output maintenance time.

Here, with respect to the information display area, which is a virtual range area capable of recognizing the output information output from the information providing information display board, the information output start time is an entry time, which is a time when the vehicle is predicted to enter the information display area, The information output holding time is calculated based on the difference between the entry time and the entry time, which is the time when the vehicle is expected to leave the information display area.

Here, the information providing guide display board includes light output means for outputting information through light, and outputs output information as light through the light output means, and an area of the information display area is an area where the information providing guide display board is located. It is determined based on the ratio of the light intensity of the light output means to the light intensity of, and the relay server maintains a constant area of the information display area by adjusting the light intensity of the light output means according to the light intensity.

Here, the relay server, based on the detection image information, the detection area entry speed information that is vehicle speed information when entering the detection area, the detection area driving speed information and detection area that is vehicle speed information when driving the detection area Calculates the detection area advance speed information, which is vehicle speed information when entering from, and at least one of the detection area entry speed information, the detection area driving speed information, and the detection area advance speed information exceeds the speed limit of the detection area If it is, the speed information indicating the highest speed is selected as the excess speed information.

Here, the relay server predicts the trend of the speed of the target vehicle based on the detection area entry speed information, the detection area driving speed information, and the detection area entry speed information, and is based on the predicted speed trend. By calculating the entry time and the entry time.

Here, the guide system for providing personalized information for each object using the deep learning-based object recognition includes a plurality of road detection groups, and the information providing guide display board for the first road detection group is an external detection device of the second road detection group. Located in a detection area, the relay server may include a first average speed and a second average value of vehicle speed information determined from first detection image information captured by an external sensing device of the first road detection group. A traffic condition between the first detection area and the second detection area is determined based on a difference between a second average speed which is an average value of vehicle speed information determined from second sensing image information captured by an external sensing device, and the traffic The information output start time and the information output maintenance time of the information providing guide display board of the second road detection group are determined based on the situation, but the second average speed is less than the value obtained by subtracting the first average speed by a preset speed value. When it is small or when the second average speed is less than a predetermined ratio compared to the first average speed, output information for prompting attention is output through the information display guide board of the second road detection group.

Here, the information display system for providing customized information for each object using the deep learning-based object recognition includes a plurality of road detection groups, and the relay server corresponds to the same vehicle number information in a road detection group having a predetermined number of or more consecutively. When it is determined that the vehicle to be passed at a speed exceeding the speed limit of each detection area, the vehicle number information is selected as a dangerous vehicle number, a vehicle corresponding to the dangerous vehicle number is selected as a dangerous vehicle, and the dangerous vehicle The dangerous vehicle number and all excess speed information of the vehicle corresponding to the dangerous vehicle number are outputted through the information providing information display board of the other road detection group that recognizes the number.

Here, the information providing guide electronic board includes light output means for outputting information through light, and outputs output information as light through the light output means, and the relay server is operated at a speed at which the dangerous vehicle exceeds the speed limit. As the number of detection areas passed increases, the size of the font of the dangerous vehicle number output through the information providing information display board increases, the intensity of light output to output the dangerous vehicle number increases, and the dangerous vehicle number An emphasis effect for emphasizing is added, but the emphasis effect includes the number of borders, the thickness of the border, and the number of flashing frequencies.

Here, the light output means outputs light corresponding to a plurality of colors, and the relay server outputs through the information providing information display board as the number of detection areas that the dangerous vehicle passes through at a speed exceeding the speed limit increases. Make the color of the dangerous vehicle number closer to red.

Here, the relay server, the map information including information on the topography, buildings, and roads for each location, the location information of the external sensing device, which is information about the location of each external sensing device, and the location of each information providing guide display board The output device location information, which is information, is stored, and a predicted path determined to be driven by the dangerous vehicle is calculated based on the map information and the location of the external sensing device corresponding to the detection area where the dangerous vehicle has passed, and the output device location Based on the information, at least one information providing information board located in the expected route is selected, and a message warning that a dangerous vehicle is approaching from the rear is output through the selected information providing information board.

Here, the guide system for providing personalized information for each object using the deep learning-based object recognition includes a plurality of external sensing devices and a plurality of information providing guide billboards, and the relay server is related to terrain, buildings, and roads for each location. Map information including information, location information of the external sensing device, which is information about the location of each external sensing device, and output device location information, which is information about the location of each information providing guide display board, are stored, and the path of the emergency vehicle is stored. When the emergency vehicle route information, which is information and the emergency vehicle number information, which is vehicle number information of the emergency vehicle, is received, the lane of the emergency vehicle is determined based on the detection image information including the emergency vehicle number information, and the emergency vehicle Based on the route information, the map information, and the location information of the output device, at least one information providing guide display board that is determined to pass through the emergency vehicle is selected, and driving of the emergency vehicle is displayed through the selected information providing guide display board. Information and information indicating the lane of the emergency vehicle are output.

Here, the relay server generates at least one or more types of output information based on information received from the outside, the surrounding environment information, and the detected image information, but output information indicating an emergency situation has a first priority, and an emergency situation The safety-related output information has a second priority, the speed-related output information has a third priority, the rest of the output information has a fourth priority, and for the natural number n, the n-th priority is the nth It means that the priority is high, and when there is no output information corresponding to a higher priority, output information corresponding to a lower priority is output through the information providing information display board.

Here, the output information corresponding to the first priority, the relay server detects the occurrence of an accident based on the information received from the outside, the surrounding environment information and the detection image information, the occurrence of the accident based on the detection result The output information including the information informing the information or the information informing that the emergency vehicle is driving, and the output information corresponding to the second priority, each of the output information at a speed exceeding the speed limit of the detection area of a predetermined number or more Includes information indicating that the vehicle passing through the detection area is approaching, and the output information corresponding to the third priority, the speeding vehicle of the vehicle passing through the detection area at a speed exceeding the speed limit of one detection area The output information corresponding to the fourth priority includes number information and excess speed information, and includes at least one of information on convenience facilities around the information providing information display board, event guide information, weather information, and news.

According to another embodiment of the present invention, a control method of a guide information display system for providing information customized for each object using deep learning-based object recognition, the information providing guide display image capturing image to generate detection image information; The information providing guide electronic board performing pre-processing on the detected image information and separating a background and an object from the captured image; The information providing guide electronic board recognizing an object separated from the background, and classifying the recognized object by type; Generating a surrounding environment information by detecting a change in the surrounding environment by the information providing guide electronic board; The information providing guide electronic board receiving wired/wireless communication with the outside to receive information from the outside; Storing the detected image information, the object classified for each type, and the surrounding environment information by the information providing guide electronic board; The information providing guide electronic board performing learning and model generation based on at least one of the surrounding environment information and the sensed image information; Generating customized information for each object, which is customized information corresponding to the type of the object, based on the identified object, the learning and model generation results, by the information providing guide electronic board; And the information providing guide display board outputting the customized information for each object as at least one of audio, light, or video. A method of controlling a customized information providing guide display system for each object using deep learning-based object recognition. This can be provided.

According to an embodiment of the present invention, it is possible to automatically recognize an object and intelligently provide traffic-related information suitable for each situation according to whether the object is a pedestrian or a vehicle.

Further, according to an embodiment of the present invention, only information regarding the speed of the speeding vehicle may be selected and provided.

In addition, according to an embodiment of the present invention, it is possible to effectively calculate a start time and a hold time for outputting information on speed.

In addition, according to an embodiment of the present invention, a dangerous vehicle is automatically selected, and a warning for the driver of the dangerous vehicle is performed by warning of the emergence of the dangerous vehicle through the information display guide board of the road where the dangerous vehicle is expected to pass. And increase awareness of surrounding drivers. This can also prevent traffic accidents.

In addition, by outputting information about the emergence of the emergency vehicle and the lane in which the emergency vehicle is in use, it is possible to quickly secure a passage through which the emergency vehicle can pass, thereby helping to resolve an emergency situation.

1 is a diagram illustrating a control method of a guide electronic display system for providing customized information for each object using deep learning-based object recognition according to an embodiment of the present invention.
2 is a diagram illustrating a control method of a guide electronic display system for providing customized information for each object using deep learning-based object recognition according to another embodiment of the present invention.
3 is a diagram illustrating a guide electronic display system for providing customized information for each object using deep learning-based object recognition according to an embodiment of the present invention.
4 is a view showing an information providing guide electronic board according to an embodiment of the present invention.
5 is a diagram showing an example in which an information providing information display board is applied to a road.
6 is a view showing a process of performing an information providing guide electronic board learning and model generation according to an embodiment of the present invention.
7 is a view showing an example of customized information for each object output through the information providing information display board.
8 is a view showing a road detection group according to an embodiment of the present invention.
9 is a diagram illustrating a method of predicting a trend of speed according to an embodiment of the present invention.
10 is a view showing an aspect in which a plurality of road detection groups are disposed according to an embodiment of the present invention.
11 is a view showing a method of selecting a dangerous vehicle according to an embodiment of the present invention.
12 is a diagram illustrating an example of output information output through an information providing guide electronic board according to an embodiment of the present invention.
13 is a diagram illustrating an example of output information output through an information providing guide electronic board according to another embodiment of the present invention.
14 is a diagram illustrating a method of selecting an information providing guide electronic board for outputting information related to a dangerous vehicle according to an embodiment of the present invention.
15 is a view showing a method of selecting an information providing guide electronic board for outputting emergency vehicle-related information according to an embodiment of the present invention.

The present invention relates to an information display system and a control method for providing customized information for each object using deep learning-based object recognition, and in particular, performs learning on the surrounding environment based on information collected through various sensing means, and results of learning The present invention relates to an information display board system for providing customized information for each object that can output traffic-related guide information suitable for the type and situation of the identified object.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

In the following specification, the'guide system for providing customized information for each object using object recognition based on deep learning' may be simply described as'system'. Also, the'road sensing group' may be simply described as'group'. In addition, the term'location' may mean an actual location that can be specified by specific GPS coordinates or the like. In addition, the'information providing information display board' may be described as a'information display board' or'information output device' or'output device'. In addition, the'traffic information statistical model' may be described as a'model'.

1 is a diagram illustrating a control method of a guide electronic display system for providing customized information for each object using deep learning-based object recognition according to another embodiment of the present invention.

According to an embodiment of the present invention, the information providing guide electronic board may capture the image to generate sensed image information (S110 ). Then, the information providing guide electronic board may perform pre-processing on the sensed image information and separate a background and an object from the captured image (S120). Then, the information providing guide electronic board may recognize an object separated from the background, and classify the recognized object by type (S130). In addition, the information providing guide electronic board may detect changes in the surrounding environment and generate surrounding environment information (S140 ). In addition, the information providing guide electronic board may perform wired/wireless communication with the outside to receive information from the outside (S150). In addition, the information providing guide electronic display board may store the detected image information, the object classified for each type, and the surrounding environment information (S160). Then, the information providing guide electronic board may perform learning and model generation based on at least one of the surrounding environment information and the sensed image information (S170 ). Then, the information providing guide electronic board may generate customized information for each object, which is customized information corresponding to the type of the object, based on the identified object, the learning and model generation results (S180). In addition, the information providing guide electronic board may output customized information for each object as at least one of audio, light, or video (S190).

According to this embodiment of the present invention, it is possible to automatically recognize an object and intelligently provide traffic-related information suitable for each situation according to whether the object is a pedestrian or a vehicle.

And, Figure 2 is a view showing a control method of a guide information display system for providing customized information for each object using deep learning-based object recognition according to another embodiment of the present invention.

According to an embodiment of the present invention, in a control method of a guide electronic display system for providing customized information for each object using deep learning-based object recognition, an external sensing device captures an image of a preset detection area of a road to generate detection image information (S310) can be. In addition, the relay server may individually identify at least one object passing through the detection area based on the detection image information (S320 ). In addition, the relay server may extract vehicle number information that is information about the number of an object that is a vehicle (S330). In addition, the relay server may extract vehicle speed information that is information on the speed of an object that is a vehicle corresponding to the vehicle number information (S340 ). Then, the relay server may select vehicle speed information of at least one vehicle passing through the detection area at a speed exceeding the speed limit of the detection area as respective speeding vehicle number information (S350 ). In addition, the relay server may select vehicle speed information as individual excess speed information when an individual vehicle corresponding to each speeding vehicle number information passes through the detection area (S360 ). Then, the relay server may output the speeding vehicle number information and the excess speed information through the information providing information display board (S370).

Through the above-described series of processing processes, the information display system for providing customized information for each object using deep learning-based object recognition can select and output only information about the speed of the speeding vehicle and can instill awareness in the drivers of the speeding vehicle. .

Description of each step will be described in detail through the drawings below.

3 is a diagram illustrating a guide electronic display system for providing customized information for each object using deep learning-based object recognition according to an embodiment of the present invention.

According to an embodiment of the present invention, a personalized information providing guide display system (hereinafter referred to as a system) for each object using deep learning based object recognition may include at least one information providing guide display board 101 (102). In addition, according to the method of implementing the present invention, the system may include at least one external sensing device 201 and 202, at least one information providing guide electronic board 101 and 102, and a relay server 300. According to FIG. 3, the system is illustrated as including two external sensing devices 201 and 202 and two information providing guide electronic boards 101 and 102, but the present invention is not limited thereto. Each device can transmit/receive mutual information/signal/data through a wired/wireless communication network, and transmit/receive information to/from facilities or institutions outside the system (e.g., traffic, police, hospital, firefighting related organizations) 800 This is possible.

The external sensing devices 201 and 202 may generate sensing image information by capturing an image of a preset sensing area on the road or around the road through image capturing means such as a camera. Here, the sensing region may be all range regions included in the sensing image information or may be actual range regions corresponding to a part of the sensing image information. Alternatively, the detection area may be a polygon-shaped area where each vertex is specified by coordinates such as GPS. In addition, the external sensing devices 201 and 202 may include various sensing means such as an air quality sensing means, a temperature sensing means, a sound recording means, an illuminance sensor, and a heat/infrared sensor, through which changes in the environment around the road or road It is possible to detect and generate surrounding environment information as a result of the detection. The external sensing devices 201 and 202 may transmit the generated sensing image information or surrounding environment information to the relay server 300, the information providing guide electronic board 101, 102, or an external agency 800 through a wired/wireless communication network. have. The external sensing devices 201 and 202 may be installed on a rod-like structure like a traffic light or may be installed inside a roadside storage box. Alternatively, the external sensing devices 201 and 202 may be provided as portable and utilized at a desired position by the side operating the system, or the installation position may be variably adjusted. However, the aspect in which the external sensing devices 201 and 202 are installed is not limited thereto.

Information providing information display board (101, 102) is a device for providing information to a car or a pedestrian, it not only serves as an information providing device, but also independently learns based on the results of sensing and sensing the surrounding environment. And modeling, and may provide traffic-related information suitable for each object based on the learning and modeling results and the detected object type (whether it is a car or a pedestrian). According to a preferred embodiment of the present invention, the information providing guide electronic boards 101 and 102 take an image to generate sensing image information, perform pre-processing on the sensing image information, separate a background and an object from the captured image, and , Recognizing an object separated from the background, and distinguishing the recognized object by type. In addition, the information providing guide electronic boards 101 and 102 may detect changes in the surrounding environment to generate the surrounding environment information, and may perform wired and wireless communication with the outside to receive information from the outside. In addition, the information providing guide display boards 101 and 102 may store the detection image information, the object classified for each type, and the surrounding environment information. Also, the information providing guide electronic boards 101 and 102 may perform learning and model generation based on at least one of the surrounding environment information and the sensed image information, and are based on the identified object, the learning and model generation results. By doing so, customized information for each object, which is customized information corresponding to the type of the object, may be generated. In addition, the information providing guide electronic boards 101 and 102 may output customized information for each object as at least one of audio, light, and video. In addition, the information providing guide electronic boards 101 and 102 may output at least one of output information generated by the relay server 300 and customized information for each object. The information providing information display board will be described in more detail in FIG. 4.

The information providing information display boards 101 and 102 may be installed on the road or around the road. The information providing information display boards 101 and 102 may be installed on a rod-like structure like a traffic light or may be installed inside a roadside storage box. Alternatively, the information providing guide electronic boards 101 and 102 may be provided as portable and utilized at a desired position by the side operating the system, or the installation position may be variably adjusted. However, the aspect in which the information providing guide electronic boards 101 and 102 are installed is not limited thereto.

The relay server 300 may manage and control the overall operation of the system. In addition, the relay server 300 may relay transmission/reception of signals, information, or data between devices included in the guide information display system providing customized information for each object using the deep learning-based object recognition. The relay server 300 may be implemented in the form of software, such as a system program, or in a hardware form such as a mobile terminal, a server, or a PC, or may be implemented in a form in which software and hardware are combined. The relay server 300 outputs the output information based on the received environment information, detection image information, control/request signal received from an external organization 800, weather, event, announcement/information, news, and the like. Can be created. In addition, the generated output information may be transmitted to the information providing guide electronic boards 101 and 102.

Meanwhile, the relay server 300 may include information storage means. The information storage means outputs map information including location-specific topography, buildings, and road information, location information of each external sensing device, location information of the external sensing device, and information about the location of each information providing information display board You can also store device location information. In addition, the information storage means includes the speed limit for each detection area, the location/shape and size of the information display area for each information providing information board, the location/shape and size of the detection area, the vehicle number information, the vehicle speed information, the speeding vehicle number information, and the excess. You can also store speed information, dangerous vehicle numbers, and dangerous cars. In addition, the relay server 300 may utilize information stored in the information storage means when generating output information.

According to a preferred embodiment of the present invention, one external sensing device and at least one information providing guide electronic board may form one road sensing group (hereinafter, a group). According to FIG. 2, group G1 includes an external sensing device 201 and an information providing guide electronic board 101, and group G2 includes an external sensing device 202 and an information providing guide electronic panel 102. In FIG. 2, only two groups are shown, and each group includes only one external sensing device and one information providing guide display board, but this is only an example for easily explaining the contents of the present invention.

According to an embodiment of the present invention, the relay server 300 may individually identify at least one object passing through the sensing area based on the sensing image information. Here, identification means recognizing an independent object or person as an object separately from surrounding backgrounds and structures, and as a result of the identification, each object may be assigned a unique identification code. The relay server 300 may additionally extract the type of each object (such as a person or a car) after identification of the object. According to a preferred embodiment of the present invention, the relay server 300 may detect the speed and direction of movement of the identified object in real time. In particular, the relay server 300 may extract vehicle number information that is information about the number of objects that are automobiles. In addition, the relay server 300 may extract vehicle speed information that is information on the speed of an object that is a vehicle corresponding to the vehicle number information.

The above-described process of object identification, number extraction, and speed extraction may be implemented through processes such as image recognition, computer vision, and image signal processing. In addition, the moving direction of each object included in the sensing image information based on the installation position of the external sensing devices 201 and 202 photographing the sensing area, the installation direction, the distance between the sensing area and the installation position of the external sensing device, and the like, The speed (eg, calculated based on the actual distance per unit pixel in the sensed image information image data) may be performed.

In addition, the relay server 300 may recognize vehicle number information of at least one vehicle passing through the detection area at a speed exceeding the speed limit of the detection area, and the corresponding vehicle number information as respective speeding vehicle number information. Can be selected. In addition, the relay server 300 may select vehicle speed information as individual excess speed information when an individual vehicle corresponding to each speeding vehicle number information passes through the detection area. Thereafter, the relay server 300 may include the speeding vehicle number information and the excess speed information in the output information, and may transmit the output information to the information providing information display board. According to a preferred embodiment of the present invention, if the above-mentioned speeding vehicle number information and the excess speed information are calculated based on information generated by the external sensing device 201, the output information is to be transmitted to the information providing information display board 101 In addition, the information providing guide electronic board 101 may output the received output information in the form of light or sound.

4 is a view showing an information providing guide electronic board 100 according to an embodiment of the present invention. In FIG. 4, a power supply unit for supplying power for driving and operating the information providing guide electronic board 100 is omitted.

According to FIG. 4, the information providing guide electronic board 100 includes an image photographing unit 110, an image processing unit 120, an object recognition unit 130, a surrounding environment detection unit 140, an information collection unit 160, and an information processing unit It may include a 170 and the information display unit 180. According to the method of implementing the present invention, the information providing guide electronic board 100 may further include a communication unit 150, a plurality of components may be integrally provided, or some components may be omitted. In addition, each component of the information providing guide electronic board 100 may be implemented with at least one chip, processor, module, and circuit board. Alternatively, each component may be implemented in hardware, software, or a mixture of hardware and software.

The image capturing unit 110 may capture the image and generate sensed image information. The image capturing unit 110 may include an imaging element/sensor for image capturing. According to the method of implementing the present invention, the image photographing unit 110 may further include a means for detecting infrared rays, and accordingly, it may perform photographing through infrared rays even at night when there is insufficient light, and as a result, infrared photographing It is also possible to generate sensing image information including an image. The image photographing unit 110 may be implemented through at least one camera, or may generate sensed image information including a panoramic image obtained by merging the detection areas photographed by each camera into one. In this case, the detection area of the image capturing unit 110 may reach 360 degrees in a horizontal or vertical direction.

The image processing unit 120 may be a hardware chip, circuit, device, or software filter/program capable of processing image signals/data/information. The image processing unit 120 may perform pre-processing on the sensed image information generated by the image capturing unit 110. Here, the pre-processing may include applying a filter to adjust the brightness, color tone, color, contrast, resolution, size of the detection image information, or to remove noise. In addition, the image processing unit 120 may separate the background and the object from the sensed image information/recorded image. According to a preferred embodiment, the image processing unit 120 may separate the background and the object based on the difference between the unchanged region and the changing region during a preset detection period in the two-dimensional image represented by the detected image information, but the background and the object may be separated. The method of separating is not limited to this. Here, the separation of the background and the object may be performed by the information processing unit 170 or the object recognition unit 130, which will be described later.

The object recognition unit 130 recognizes an object separated from the background, and classifies the recognized object by type. The object recognition unit 130 may classify the type of the object into a pedestrian and a vehicle, and the pedestrian may be divided into an adult and a minor. However, the types of objects that the object recognition unit 130 can distinguish are not limited thereto. The object recognition unit 130 may perform feature extraction on the 2D image data corresponding to the object separated from the background, and perform classification for each object type based on the extracted feature. However, the manner in which the object recognition unit 130 distinguishes and recognizes objects is not limited to the above.

The surrounding environment detection unit 140 may detect a busy environment, and as a result, may generate surrounding environment information. The surrounding environment detection unit 140 may include at least one of an air quality sensing means, a temperature sensing means, a sound recording means, an illuminance sensor, and a heat/infrared sensor, through which a change in a road or an environment around the road is sensed, As a result of detection, it is possible to generate information about the surrounding environment. According to a preferred embodiment of the present invention, the ambient environment information may include information about temperature and air quality.

The communication unit 150 may include a wired or wireless communication module, through which it can perform wired and wireless communication with the outside. Through this, the communication unit 150 may transmit/receive information/data/signals with external organizations, external devices, and external networks.

The information collection unit 160 may store information, and for this purpose, the information collection unit 160 may include physical hardware (chip, hard disk, SSD) or software virtual storage space and memory as information storage means. . Preferably, the information collecting unit 160 includes the detected image information, the objects classified by the type, the surrounding environment information, the information received by the communication unit from the outside, and the parking stop prohibition time range, which is a range of the prohibition of parking stop over time. You can store information about it.

The information processing unit 170 is a core component capable of controlling the operation of the information providing guide electronic board 100, and may perform learning and model generation based on at least one of the surrounding environment information and the sensed image information. In addition, the information processing unit 170 may generate customized information for each object, which is customized information corresponding to the type of the object, based on the identified object, the learning and model generation results. Here, the customized information for each object may include vehicle customized information or pedestrian customized information. In addition, the information processing unit 170 may also calculate vehicle speed information, which is information about a vehicle's object speed, and vehicle number information, which is information about a vehicle's object number, based on the sensed image information. Meanwhile, the information processing unit 170 may include information about the temperature and information about the air quality in the pedestrian personalization information or the automobile personalization information.

The information display unit 180 may output information/signal/data. In particular, the information display unit 180 may output various information generated by the information processing unit 170 (customized information for each object) or received from the outside (output information). In addition, the information output by the information display unit 180 is output information generated by a relay server, sensing image information generated by an external sensing device or surrounding environment information, weather received from an external organization, event, announcement/information , News, and the like.

The information display unit 180 may output information/signal/data through at least one output means. For example, the information display unit 180 may include sound output means such as a speaker, through which information may be output as sound. According to a preferred embodiment of the present invention, the information display unit 180 may include light output means for outputting information through light. Here, the light output means may be a panel in which a plurality of lamps/bulbs (such as LEDs) are integrated, or a display panel of a method such as LCD, LED, OLED. Through this, the information display unit 180 may output information in the form of light. In this case, the light output means may output light corresponding to a plurality of colors, and may simultaneously output light of various colors.

5 is a view showing an example in which the information providing guide electronic board 100 is applied to the road. A and SA shown in dotted lines in FIGS. 5(a) and 5(b) are a sensing area and an external sensing device 201, 202 that can be photographed/detected by an image capturing unit or a surrounding environment sensing unit of the information providing guide electronic board 100 ). In addition, in FIG. 5(a), although the detection area of the information providing guide display board 100 is shown as a circle centering on the information providing guide display board 100, the aspect of the detection area of the information providing guide display board 100 is limited thereto. Does not work.

According to a preferred embodiment of the present invention, the information providing guide electronic board 100 constantly detects an object in the detection area, and information related to the detection (sensing image information, surrounding environment detection information, object related information, etc.) ). In addition, the information providing guide electronic board 100 may classify each object into a pedestrian or a vehicle. In the situation of Fig. 5 (a), the information providing guide electronic board 100 can distinguish the object 500 as a vehicle and the objects 901 and 902 as a pedestrian, respectively. Furthermore, the information providing information display board 100 may further classify pedestrians as adults and minors, and accordingly object 901 as adults and object 902 as minors. In addition, the information providing information display board 100 may distinguish whether or not a pedestrian is moving on a pedestrian crossing. Here, the information providing guide electronic board 100 may separately detect/extract the background corresponding to the crosswalk from the captured sensing image information. In addition, the information providing guide electronic board 100 may extract a moving direction of each object, a moving speed of each object, and the like. Such object classification can be implemented with various image signal processing and computer vision technologies. In addition, the information providing guide electronic board 100 can provide customized information for each object corresponding to the classification of each object according to whether the recognized object is a pedestrian or a vehicle, and accordingly, the customized information for the vehicle 500. , Pedestrian customized information for pedestrians 901 and 902 may be output. In the situation of FIG. 5, the pedestrian and the vehicle are simultaneously located in the detection area, so that the customized information for both objects may be simultaneously output.

5(b) and 5(c) show a specific case in which information providing guide electronic boards 101 and 102 are installed on a road where several lanes exist. In particular, in the situation of FIGS. 5(b) and 5(c), each external sensing device is generated in connection with at least one external sensing device other than the image capturing unit and the surrounding environment sensing unit that the information providing guide electronic board itself includes. An example of providing customized information for each object based on one sensed image information is illustrated.

First, according to FIG. 5(b), each direction of the road is composed of two lanes, and can be detected for each lane by the external sensing devices 201 and 202 individually assigned to each lane. And, based on the external sensing image information photographed by the external sensing device 201, the information providing guide electronic board 101 can generate and output customized information for each object suitable for the identified object (eg, cars 501 and 502). , The information providing guide electronic display board 102 may generate and rule customized information for each object based on the external sensed image information captured by the external sensing device 202.

In FIG. 5(b), it is assumed that the car 501 is in a speeding operation, and the car 502 is in a constant speed operation or a situation where a pedestrian crossing (not shown) is ahead. Accordingly, the information providing guide electronic board 101 can generate and provide customized information corresponding to the situation of the two cars.

FIG. 5( c) shows customized information or output information for each object that can be output from the information providing guide electronic board 101. According to a preferred embodiment of the present invention, the information providing guide display board may include a plurality of information display units, and each object generated based on detection image information of different external sensing devices through each information display unit Information may be output through different information display units. According to FIGS. 5(b) and 5(c), the information providing guide electronic board 101 may be provided in a horizontally long form, and the information display unit 181 on the left side may target information on the primary lane in which the vehicle 501 is driving. Output, and the information display unit 182 on the right may output information to a secondary lane in which the vehicle 502 is running. In the example of FIG. 5(b), the car 501 is traveling at a speed of 88 km/h, and the information providing guide electronic board 101 that recognizes this is the car 501 through the information display unit 181 on the left side of the car 501. It outputs a message warning whether or not the vehicle is overspeed, information on the detected speed, and a vehicle speed number 1234, which is the number of the vehicle 501. In the example of FIG. 5(b), since the car 502 is traveling at a constant speed, a message related to speeding is not displayed for the car 502, but by recognizing the direction in which the car 502 travels, to a pedestrian crossing located in the direction of the car 502 A message urging attention is output through the information display unit 182 on the right side of the information providing information display board 101.

6 is a view showing a process of performing an information providing guide electronic board learning and model generation according to an embodiment of the present invention. In FIG. 6, the horizontal axis represents time, and the vertical axis represents the number of pedestrian crossing users detected in the detection area of the information display guide board, and the black dot is the number of pedestrian crossing users detected over time.

In the following description, for ease of explanation, a series of processes performed and processed by the information processing unit will be described as being performed by an information providing information display board that is a device including the information processing unit.

According to a preferred embodiment of the present invention, the information providing guide display board may perform learning and model generation based on at least one of ambient environment information and sensed image information. Alternatively, the information providing guide electronic board may perform learning and model generation based on information (corresponding to big data) stored in the information collection unit. Here, the learning and model generation performed by the information display guide board correspond to the concept of learning and artificial intelligence model generation commonly used in the fields of artificial intelligence, machine learning/deep learning. That is, the information providing information display board includes at least one or more layers including a preset number of nodes or neurons as an artificial intelligence model or a neural network model (hereinafter, a model). can do. In addition, the information providing information display board provides information as a result value by multiplying a preset number of input values by a preset weight, and adding a sum of at least a portion of each input value multiplied by weight and a preset bias. It can be delivered to a specific node of the layer of the guide sign (that is, entered as the value of each node). The input value, each weight, and the connection relationship between the result value and the node may be provided in various ways. If the information display guide board has only one layer, the value of each node, or the value corresponding to the result of applying a specific processing process/function to the value of each node, outputs each of the artificial intelligence model of the information display guide board It becomes the value. If the information providing information board has a plurality of layers, the information providing information board may multiply the value of each node by a preset weight, and add a preset bias to the sum of at least one multiplied result value to obtain the next layer. It can be input to a specific node. This process can be applied to other nodes in different layers in a similar manner. The above-described process may be repeated depending on the number of layers included in the information providing information display board model. The information display guide display board may calculate a final result value by applying a specific processing process such as softmax to at least one result value of the model.

Hereinafter, an embodiment in which learning is performed in an information providing information display board will be described. For example, the information providing information display board may input a large amount of sensed image information (so-called big data) as an input value of the model. Here, when the sensed image information is image (or image) information/signal/data composed of a plurality of pixels, a value of a pixel extracted by a preset number from the image information may be input as an input value of the model. When the value of the pixel is input to the model of the information providing information display board, it is possible to derive a preset number of output values according to the above-described process. Here, the model generates two output values, for example, the first output value of the model is a value representing the degree of identifying the object of the image by the vehicle, and the second output value of the model identifies the object of the image as a pedestrian. It is a value indicating the degree, and it can be assumed that the maximum value of the output value is 1 and the minimum value is 0. In this case, if the first output value is 1 and the second output value is 0.1, it means that the object included in the image is identified as the vehicle with respect to the input sensing image information. The information providing information display board may update the weights and biases of the model based on the output value to improve the accuracy of the model (or proceed with optimization of the model).

Here, the learning of the information providing guide electronic board may update the weight and bias of the model according to supervised learning. For example, it may be assumed that the input sensing image information described above includes a vehicle image, and that “automobile” is specified as a correct answer or label. If the weight and bias of the model are properly set, the first output value should be close to 1 and the second output value should be close to 0 for the sensed image information including the correct answer "automobile". However, if the weight and bias of the model are not properly set, the information display guide board displays the difference between the two output values for the detected image information including the correct answer "automobile" and the output value corresponding to the correct answer {1, 0}. Calculation (in the field of artificial intelligence, this may be a process of calculating cost), and based on this, error backpropagation may be performed to update the weight and bias of the model. However, the method in which the information providing guide electronic board performs the update of the weight and bias of the model is not limited to this, and other methods such as an unsupervised learning method may be used. And the learning is continuously performed based on the newly collected information even while the information display guide board is in operation, so that the accuracy of the model can be improved.

Through the above-described process, the information providing information display board can generate at least one model and perform learning related to the model. As described above, the information providing guide display board may perform learning and model generation to accurately identify objects, or may perform learning and model generation to identify the number of types of each object. Alternatively, the information providing information display board may receive the time when the image was taken as an additional input value, and as a result, may be included in the detection area (ie, included in the detection image information) according to the time or the location (eg, lane). ) Can also perform learning that estimates the number of each object, etc., and generate a model accordingly. If the information providing information display board receives a signal requesting to generate a specific model from the outside, model generation and learning corresponding to the received information may be performed (for example, the number of objects that are buses is estimated. Model). Alternatively, the information providing guide electronic board may generate a model for estimating the moving direction and speed of the object, and perform learning based on information stored in the information collection unit.

In addition, according to a preferred embodiment of the present invention, the information providing guide display board includes information stored in the information collection unit-sensing image information, objects classified by type (identified objects), and number of pedestrians over time based on surrounding environment information. , Pedestrian-related statistics including the number of pedestrians using a pedestrian crossing over time can be calculated. In addition, the information providing guide electronic board can calculate automobile-related statistics including the number of automobiles over time and the average speed of automobiles over time based on the stored information. In addition, the information providing information display board may generate a traffic information statistical model over time based on the pedestrian-related statistics and the vehicle-related statistics.

Here, the traffic information statistical model according to the time is a model generated and optimized as a result of learning based on information stored in the information collection unit, and object-related information (eg, the type of the identified object, the object as an input value) If information on the number, or the image data corresponding to the object itself) and time (the time when the object is identified or the type is distinguished) is input, the object will be located in the detection area of the information display guide board or external sensing device. The number of expected objects can be created as an output value.

Alternatively, the traffic information statistical model may generate, as an output value, an expected average value of the speed at which the object passes through the detection area of the information display guide board or the external sensing device when the object-related information and time-related information are input as input values. Can.

Alternatively, the traffic information statistical model outputs the number of pedestrians expected to use the pedestrian crossing within the detection area of the information display guide board or the external sensing device when the object-related information and time-related information are input as input values. Can be created.

On the other hand, preferably, the information providing information display board can recognize the number, speed, location, moving direction, etc. of each object, and based on this, includes the number of pedestrians over time and the number of pedestrians using the pedestrian crossing over time. Pedestrian-related statistics and vehicle-related statistics including the number of vehicles over time and the average speed of the vehicle over time may be calculated. For example, the information providing information display board can calculate the number of pedestrians using a pedestrian crossing according to each unit time (for example, 1 minute) with respect to the time during the day, and the calculation of the number is performed for days or months. can do. As a result, the information display guide board can calculate the number of pedestrians using the pedestrian crossing according to the unit time indicating the statistical distribution. In addition, the information providing information display board may generate a traffic information statistical model over time based on the pedestrian-related statistics and the vehicle-related statistics. Here, the information providing guide electronic board may calculate an expected statistical value corresponding to the time based on the type of the object and the time the object is identified/recognized.

6 illustrates a process of calculating a traffic information statistical model based on the number of pedestrian crossings over time. FIG. 6(a) shows the number of pedestrian crossing users detected by the information providing information display board as a black circle with time. The information providing information display board can generate a traffic information statistical model based on the acquired/stored information, and thus corresponds to a trend line M corresponding to the number of pedestrians crossing over time as shown in FIG. 6(b). The model can be calculated. Accordingly, when information regarding a “crosswalk user (object as a pedestrian)” and time is input to the corresponding model, the number of persons estimated to use the crosswalk at the time may be output as an output value. Here, the model may be periodically optimized/updated/updated based on sensing image information collected in real time.

On the other hand, in Figure 6 (c), it can be assumed that the information providing information display panel identifies the vehicle at time t4. In this case, the information providing information display board can calculate the number of treadmill users/statistical value val4 corresponding to the trend line M according to the time t4 and the traffic information statistical model. In other words, this means that at the time t4, there is a high possibility that a pedestrian corresponding to val4 exists in the pedestrian crossing around the information display guide board.

In addition, when the object providing the information is recognized as the pedestrian object from the detected image information, the information providing guide electronic display board generates pedestrian customized information based on the time when the object as the pedestrian is recognized and the traffic information statistical model, and the pedestrian through the information display unit. Custom information can be output. Alternatively, the information providing guide electronic board generates vehicle personalized information based on the time when the vehicle object is recognized and the traffic information statistical model when the vehicle object is recognized from the detected image information, and the vehicle is displayed through the information display unit. Custom information can be output.

Suppose that in the situation of FIG. 6, a separate traffic information statistical model (trend line) generated based on the number of pedestrians over time is provided. According to an embodiment of the present invention, the information providing guide display board is a limited number of pedestrians using a pedestrian crossing where a number of pedestrians using a pedestrian crossing (val4) of the traffic information statistical model corresponding to a time t4 when a vehicle object is recognized is preset. The number of pedestrians in the traffic information statistical model exceeding (th) or corresponding to the time t4 when the vehicle object is recognized is included in a preset night time zone, and the vehicle object is recognized (t4) When the city) exceeds the preset number of night pedestrians (not shown), a message prompting attention to pedestrians may be included in the vehicle customized information. That is, when the object is a car, attention may be required to pedestrians at intersections or pedestrian crossings, and particularly, pedestrians crossing a pedestrian crossing or walking at night need attention. Through the above-described embodiment, the information providing information display board can output vehicle customized information to warn the pedestrian about the vehicle, and through this, it is possible to prevent a pedestrian-related automobile accident in a pedestrian crossing.

As another example, in FIG. 6, the vertical axis is the average speed of the vehicle over time, and the trend line M corresponding to the traffic information statistical model calculated based on the statistical distribution of the average speed of the vehicle over time is calculated. Let's assume. In addition, although not shown in FIG. 6, the vertical axis is the number of cars over time, and another trend line is calculated from the traffic information statistical model calculated based on the statistical distribution of the number of cars over time. In this case, according to a preferred embodiment of the present invention, the information providing guide display board has a predetermined number of cars (not shown) in which the number of cars (not shown) of the traffic information statistical model corresponds to the time t4 when the object as a pedestrian is recognized ( (Not shown), and if the average speed (val4) of the vehicle of the traffic information statistical model corresponding to the time t4 when the object that is the pedestrian is recognized exceeds the preset limit vehicle speed (th), the pedestrian The customized information may include a message urging attention when using a pedestrian crossing. That is, when a pedestrian is detected at the time t4, the information providing guide display board is more likely to have a greater number of cars than a specific limit value for the pedestrian, and the average speed of the car is also likely to be above a certain limit value. By taking caution, you can prevent accidents on pedestrian crossings.

As another example, the information providing information display board includes pedestrians including the number of pedestrians as adults over time, the number of pedestrians as minors over time, and the number of adults using pedestrian crossings over time. Related statistics can be generated. And the information providing information display board can generate a traffic information statistical model based on the pedestrian-related statistics. In addition, the information providing guide display board includes the traffic corresponding to the time at which the vehicle object is recognized, and the number of minor pedestrians of the traffic information statistical model corresponding to the time at which the vehicle object is recognized exceeds the preset limit of the number of minor pedestrians. When the number of minors using the pedestrian crossing of the information statistical model exceeds the preset number of minors using the limited pedestrian crossing, the vehicle customized information may include a message urging attention to the minor pedestrian when passing the pedestrian crossing, and accordingly The driver of the car raises the boundary for the minors around, and as a result, the traffic accident of the minors can be prevented.

7 is a view showing an example of customized information for each object output through the information providing information display board. According to an embodiment of the present invention, the information collection unit of the information providing information display board may store information on the parking stop prohibition time range, which is a range of the parking stop prohibition time over time. In addition, the information providing guide display board may extract vehicle number information, which is information about a number of an object that is a vehicle, based on the detected image information. In this case, the information providing guide display board may determine that the vehicle being stopped is a parking violation vehicle when the range of the time at which the object, which is the vehicle, remains stationary from the time the vehicle starts to stop, is included in the forbidden time range. 7 illustrates a situation in which a vehicle with the number 1234 is determined to be a parking violation vehicle. In addition, the information providing guide electronic display board may output a message for urging the prohibition of parking and the vehicle number information of the vehicle that violates the parking through the information displaying unit 180. When the message according to the example of FIG. 7 is displayed, the driver of the car that violates the parking lot feels a psychological burden from a message that directly refers to the number of his vehicle, thereby releasing the illegal parking state of his vehicle, and It is likely to move the car. In addition, the police or the like to control the illegal parking of the vehicle may immediately control the illegal parking of the vehicle based on information and messages output from the information display unit 180.

8 is a view showing a road detection group according to an embodiment of the present invention. In FIG. 8, it is assumed that the information providing guide electronic board 100 is installed on the side of the road, and only the image capturing means of the external sensing device 200 is shown, and the remaining structures for installation and fixing are omitted. In FIG. 8, SA denotes a sensing region of the external sensing device 200, P100b denotes a location where the information providing information display board 100 is installed, or a point where the information displaying region ends, and OA denotes an information displaying region, D denotes an arrangement distance, which is a distance between the detection area SA and the location of the information providing information display board 100. In FIG. 8, one end of the information display area P100b is shown as the location where the information providing information display board 100 is installed, but this is for convenience of description, and information is displayed by the installation location of the information providing information display board 100. The area OA is limited or vice versa, and the installation position of the information providing guide electronic board 100 is not limited by the information display area OA.

According to an embodiment of the present invention, one external sensing device and at least one information providing guide electronic board may form one road sensing group. 8 shows that the external sensing device 200 and the information providing guide display board 300 are included in the same road sensing group. In addition, when the relay server outputs output information generated based on the sensed image information obtained from the external sensing device, the output information may be output through the information providing guide electronic board included in the same road sensing group as the external sensing device. have. Accordingly, output information generated based on the surrounding environment information generated by the external sensing device 200 of FIG. 8 may be output through the information providing guide electronic board 300.

According to a preferred embodiment of the present invention, the sensing area SA of the external sensing device 200 is located at a predetermined distance from the information providing guide electronic board 100 included in the same group as the external sensing device. Can be located. Then, the relay server calculates the information output start time, which is the time to start outputting the output information and the time to maintain the output of the output information, which is the time to start outputting the output information based on the arrangement distance D and the excess speed information or vehicle speed information. Can. The relay server may output the speeding vehicle number information and the excess speed information during the information output maintenance time from the information output start time when outputting the speed information through the information providing guide electronic board 100.

Here, with respect to the information display area OA, which is a virtual range area capable of recognizing output information output from the information providing guide electronic display board 100, the information output start time is when the vehicle 500 enters the information display area ( P100a) is an entry time, which is a time predicted to be performed, and the information output holding time may be calculated based on a difference between an entry time and an entry time, which is a time predicted that the vehicle will leave the information display area (P100b).

For example, in the situation of FIG. 8, the car 500 maintains a speed of 36 km/h, the distance corresponding to the OA is 72 m, the D is 144 m, and the time when the car 500 passes P200b is 15 o'clock. You can assume that it is on time. In this case, since the vehicle 500 moves at a speed of 10 m/s, the time at the moment of entering the information display area OA (P100a) becomes 15:2, which is the entry time and the start time for information output. . In addition, since the advance time, which is the time when the vehicle 500 passes all the information display areas OA (P100b), is 15 seconds and 4 seconds, it can be said that the information output holding time is 2 seconds.

The information display area (OA) effectively means a range of positions where a driver or a pedestrian is expected to be able to check information output from the information providing information display board 100, which is the output included in the information providing information display board 100 It may be influenced by the type of means, the installation direction of the output means, the intensity of light or sound of the output means, the surrounding weather/brightness (illuminance), and the like.

When it is assumed that the information providing guide electronic board 100 includes light output means for outputting information through light and outputs output information as light through the light output means, the area of the information display area OA is the It can be determined based on the ratio of the intensity of light of the light output means to the illuminance of the region where the information providing guide electronic board 100 is located. Here, the information providing guide electronic board 100 may be equipped with a sensor for measuring ambient light and a sensor for measuring light intensity of the light output means, and transmit the measured result to a relay server.

For example, when the result of measuring the ambient illuminance is 5 (unit omitted), and the light intensity of the light output means is 10 (unit omitted), the ratio between the two values is 1:2. In this ratio, it can be assumed that the area of the information display area OA is attracted. If the result of measuring the illuminance is 10 because the surrounding illuminance increases, the ratio between the two values becomes 1:1, and accordingly, the area of the information display area OA may be reduced. That is, even if the brightness of the smartphone screen is the same, the information display area (OA) is evaluated differently on the same principle as the difference between visibility in the middle of the day and visibility in an unlit room. When the area of the information display area OA is reduced, the information output start time may be relatively slow or the information output maintenance time may be reduced (the time required to pass through the corresponding area is reduced as the information display area decreases). In general, since the information providing information display board 100 installed in a specific place expects to output information under the same conditions, the possibility of recognizing the information even when the situation changes to the surroundings, and the same visibility when outputting information through the light output means It is expected to be maintained, and the time at which information is output also needs to be maintained. Therefore, the relay server may maintain the area of the information display area constant by adjusting the light intensity of the light output means according to the illuminance. In the previous example, when the ambient illuminance increases to 10, the relay server may increase the light intensity of the light output means to 20 so that the ratio of the two values becomes 1:2, and accordingly, the visibility and information output start time And the information output holding time can be maintained.

9 is a diagram illustrating a method of predicting a trend of speed according to an embodiment of the present invention. In FIG. 9, P200a is an entry point of the detection area SA, P200b is an entry point of the detection area SA, and P100c represents a specific point in the detection area SA. According to an example, P200c may have position coordinates corresponding to an average value of the position coordinates of P200a and P200b, but embodiments of the present invention are not limited thereto. The graph at the bottom in FIG. 9 represents the speed of the vehicle 500 according to time, ta is time when passing through the P200a point, tb is time when passing through the P200b point, and tc is when passing through the P200c point. Means the time of tx, tx is a specific time, and the white bar above tx is the estimated speed at tx. va, vc, and vb refer to the speed of the vehicle 500 detected at P200a, P200c, and P200b, respectively.

Referring to FIG. 8, it is assumed that the vehicle is traveling at 36 km/h by way of example, but there is no practical case where the vehicle maintains the same speed. Therefore, it is necessary to calculate the information output start time and the information output maintenance time according to the speed of the vehicle more realistically.

According to an embodiment of the present invention, the relay server is based on the detection image information, when the vehicle enters the detection area (P200a), the vehicle speed information, the detection area entry speed information (va), and when driving the detection area (P200c) It is possible to calculate the sensing area driving speed information vc, which is the vehicle speed information, and the sensing area advance speed information vb, which is the vehicle speed information when entering the P200b from the sensing area. The relay server can continuously measure the speed of a specific object based on the sensed image information, so it is possible to calculate such a value.

According to an embodiment of the present invention, the relay server may select information regarding the number and speed of the vehicle passing through the detection area at speeds exceeding the speed limit of the detection area, respectively, as speeding vehicle number information and excess speed information. Accordingly, the relay server is the highest if at least one of the detection area entry speed information (va), the detection area driving speed information (vc) and the detection area entry speed (vb) information exceeds the speed limit of the detection area The speed information indicating the speed can be selected as the excess speed information. In FIG. 9, when vc and vb are speeds exceeding the speed limit, the relay server may select vb, which is the higher speed, as the speed information.

In addition, the relay server may predict a trend of the speed of the target vehicle based on the detection area entry speed information, the detection area driving speed information, and the detection area entry speed information. In FIG. 9, est is a curve showing the trend of the selected speed based on three speeds (va to vc), and can be calculated through techniques such as least squares method, curve fitting with polynomials, and the like. In addition, the relay server may calculate the entry time and the entry time based on the trend of the predicted speed. That is, the relay server is based on the predicted speed trend (est), placement distance, and points P100a and P100b of the information display area (OA) of FIG. 8, such that the vehicle 500 enters and exits the information display area (OA). Time can be calculated (distance or position over time of the vehicle can be calculated by integrating the graph at the bottom of FIG. 9 with respect to time).

According to the method of carrying out the present invention, the relay server may perform a machine learning process for calculating the optimal information output start time and information output retention time based on the speed and arrangement distance of the vehicle passing through the detection area. For example, it is possible to measure various speeds of thousands or tens of thousands of vehicles passing at various speeds, and collect survey results for the appropriateness of information output start time and information output maintenance time from drivers of each vehicle. Thereafter, the survey results are classified according to the speed category of each vehicle (for example, a speed of passing 10km/h or more and less than 20km/h, a speed of 20km/h or more and less than 30km/h, ...), By performing machine learning using the survey result values according to the categories, it is possible to calculate the optimal information output start time and information output maintenance time for each speed category.

10 is a view showing an aspect in which a plurality of road detection groups are disposed according to an embodiment of the present invention. In FIG. 10, an external sensing device included in the same road detection group as each information providing guide board is omitted. In FIG. 10, vid1 to vid3 refer to first to third detection image information captured in each detection area SA1 to SA3 (or taken by an external detection device in each road detection group).

According to an embodiment of the present invention, the system may include a plurality of road detection groups. In addition, the information providing guide display board of a specific road detection group may be located in the detection area of the other road detection group. In FIG. 10, examples of three road detection groups are illustrated, and the information providing guide electronic board 101 (hereinafter, the first external detection device) of the first road detection group is the detection area SA2 of the second road detection group. Sensing area), and the information providing guide electronic board 102 (hereinafter, the second external sensing device) of the second road sensing group is located in the sensing area SA3 (hereinafter, the third sensing area) of the third road sensing group.

In this case, the relay server photographs the first average speed which is the average value of the vehicle speed information determined from the first detection image information captured by the external detection device of the first road detection group and the external detection device of the second road detection group. A traffic condition between the first detection area and the second detection area may be determined based on a difference between a second average speed which is an average value of vehicle speed information determined from the second sensed image information. For example, when the second average speed is very low compared to the first average speed, it may be determined that a traffic jam has occurred around the second detection area SA2. Then, the relay server may determine the information output start time and the information output maintenance time of the information providing guide display board of the second road detection group based on the traffic condition. For example, as opposed to the above-described example, when the second average speed is higher than the first average speed, it may be determined that the traffic situation around the second detection area is better than the traffic condition around the first detection area. Accordingly, it can be predicted that the speed of the vehicle will increase further as it passes through the second detection area. In this case, since the corresponding vehicle can enter the information display area of the second road detection group more quickly and pass faster, a faster information output start time and a shorter information output maintenance time can be calculated.

Meanwhile, the relay server may have the second average speed less than a value obtained by subtracting a preset speed value (for example, 30 km/h) from the first average speed, or the second average speed is compared to the first average speed. If it is less than a set ratio (for example, 30%), it is possible to output the output information urging attention ahead through the information providing information display board of the second road detection group, and through this, the vehicle according to the sudden change/deterioration of road traffic conditions It can prevent accidents such as rear collision.

On the other hand, when the information providing information display board is arranged in the configuration as shown in FIG. 10, the traffic situation between each information providing information display board or the speed of the cars can be collected in real time/continuously, and the entire traffic situation information based on this can be generated. In addition, the entire traffic situation information may be output through each information providing information display board.

11 is a view showing a method of selecting a dangerous vehicle according to an embodiment of the present invention. In FIG. 11, the speed limits of the detection areas SA1 to SA3 are 80 km/h, 80 km/h, and 60 km/h, respectively, and the detected speeds of the vehicle 500, number 1234, in each detection area are 120 km/h and 115 km/h, respectively. , 112 km/h.

According to an embodiment of the present invention, the relay server passes a vehicle corresponding to the same vehicle number information in a road detection group having a predetermined number of or more consecutively (hereinafter, a limit number of areas) at a speed exceeding the speed limit of each detection area When it is determined that the vehicle has been determined, the vehicle number information may be selected as a dangerous vehicle number, and a vehicle corresponding to the dangerous vehicle number may be selected as a dangerous vehicle. In the example of FIG. 11, when the number limit of the number of zones is 3, since the vehicle 500 continuously exceeds each detection zone at a speed exceeding the speed limit of all three detection zones, the number 1234 of the vehicle 500 is a dangerous vehicle It is selected by number, and the vehicle 500 may be selected as a dangerous vehicle.

According to another embodiment of the present invention, the relay server may extract the number of times a vehicle object changes a lane from the sensing image information of each sensing area. Accordingly, when a lane of a specific number of times or more is changed (for example, 5) in the detection area of a predetermined number or more (for example, 3) continuously, the number of the corresponding car is similarly selected as a dangerous vehicle number, and the corresponding car is selected. You can also classify it as a dangerous car.

The relay server may output the dangerous vehicle number and all excess speed information of the vehicle corresponding to the dangerous vehicle number through the information display guide board of the other road detection group that recognizes the dangerous vehicle number. That is, a dangerous vehicle enters the detection area of another road detection group, not the road detection group corresponding to the three detection areas of FIG. 11, and the relay server detects the vehicle number 1234 from the detection image information of the other road detection group In one case, it is possible to immediately output the previous excess speed information (speeds detected in the detection areas SA1 to SA3 120km/h, 115km/h, 112km/h) through the information providing information board of the other road detection group, Through this, it is possible to instill awareness in the driver of the automobile 500. If the car 500 is all speeding in the detection area of FIG. 11 and the lane change is performed more than a specific number of times, the relay server detects the number of the car 500 (dangerous vehicle number) and the car 500. Information can be automatically transmitted to external organizations such as the National Police Agency.

12 and 13 are diagrams illustrating examples of output information output through an information providing information display board according to an embodiment of the present invention. Here, FIG. 12 is an example of a situation in which the number of areas limit value is 3 and FIG. 13 is a number of areas limit value of 2.

In particular, FIG. 12 shows an example of output information output through the information providing guide electronic board when the case of FIG. 11 occurs. That is, in FIG. 11, the vehicle 500 has passed at all speeds in all three detection areas, and thus, is selected as a dangerous vehicle. In this case, when the vehicle 500 enters the new detection area, the information providing guide board of the road detection group corresponding to the detection area includes speeding vehicle number information (dangerous vehicle number in the case of a dangerous vehicle) and the vehicle 500 All the excess speed information of can be output, and accordingly, as shown in FIG. 12, the speeds detected in each of the detection areas SA1 to SA3 and the dangerous vehicle number 1234 can be output together.

On the other hand, by outputting information differentially according to the number of times in violation of the speed limit of each detection area, it is possible to instill awareness in the driver of the dangerous vehicle. According to another embodiment of the present invention, the relay server increases the size of the font of the dangerous vehicle number output through the information providing information display board as the number of detection areas that the dangerous vehicle passes through at a speed exceeding the limited speed increases. And, the intensity of light output to output the dangerous vehicle number increases, and an emphasis effect for highlighting the dangerous vehicle number may be added. Here, the emphasis effect may include the number of borders, the thickness of the borders, and the number of flashes.

13, the vehicle 1234 passed through the detection areas SA1 to SA3 at a speed, and the vehicle 5678 passed the detection areas SA2 and SA3 at a speed. As described above, in the example of FIG. 13, since the number limit of the number of areas is 2, both the cars 1234 and 5678 may be classified as dangerous cars. At this time, according to an embodiment of the present invention, as the number of detection areas that a dangerous vehicle passes through at a speed exceeding a limited speed increases, the size of the font of the dangerous vehicle number output through the information providing information display board may increase. Accordingly, it can be confirmed that the size of the font of the dangerous vehicle number 1234 is greater than the size of the font of the dangerous vehicle number 5678.

On the other hand, the relay server may make the color of the dangerous vehicle number output through the information providing information display board closer to red as the number of detection areas that the dangerous vehicle passes through at a speed exceeding the speed limit increases. Accordingly, according to FIG. 13, a dangerous vehicle number 1234 of a vehicle that has passed through three detection areas at a speed is closer to red than a dangerous vehicle number 5678 of a vehicle that has passed two detection areas at a speed, and 5678 is relatively yellow. It is output in a color close to. Through this, information related to a vehicle that violates the speed limit more seriously can be aroused to the driver of the vehicle.

Meanwhile, in FIG. 13, it is illustrated that the two vehicle-related information is output through the same area, but the embodiment of the present invention is not limited thereto, and in the example of FIG. 13, the number 1234 related information having a greater number of speeding violations is output. The area may be larger than the area outputting the number 5678-related information, and through this, the information related to the number 1234 may be further emphasized.

14 is a diagram illustrating a method of selecting an information providing guide electronic board for outputting dangerous vehicle-related information according to an embodiment of the present invention. In FIG. 14, solid lines denote roads on map information, numbers 201 to 205 denote positions of the first external sensing device to the fifth external sensing device, and numbers 101 to 105 denote first information providing information display boards to fifth information It means the location of the information display board. As in the previous figure, car 500 is a dangerous car and number 1234 is a dangerous vehicle number.

According to another embodiment of the present invention, the relay server includes map information including information on topography, buildings, and roads for each location, location information on the external sensing device, which is information about the location of each external sensing device, and information provision guide It is possible to store output device location information, which is information about the location of the scoreboard.

In relation to the above-mentioned dangerous vehicle, the relay server may calculate the predicted route that the dangerous vehicle is determined to travel based on the map information and the location of the external sensing device corresponding to the detection area through which the dangerous vehicle has passed. According to FIG. 14, the relay server is the third external sensing device 203 where the device that has generated the most dangerous vehicle-related sensing image information is the risk as shown in FIG. 14 based on the time passing through the sensing area of the external sensing device. The position of the vehicle 500 may be specified. In addition, the relay server may calculate a path predicted by the dangerous vehicle 500 to proceed based on the specified location. Then, the relay server may select at least one information providing information display board located in the expected path based on the output device position information, which is information about the location of each information providing information display board. According to FIG. 14, the relay server may predict that the dangerous vehicle 500 will proceed in the direction of the fourth external sensing device 204 and the fourth information providing guide display board, and accordingly, may select the fourth information providing guide display board. In addition, the relay server may output a message warning that a dangerous vehicle is approaching from the rear through the selected information providing information display board. That is, in the example of FIG. 14, a message warning of the approaching of a dangerous vehicle may be output through the fourth information providing guide electronic board 104. Dangerous vehicles are likely to continuously speed up, and dangerous vehicles, which are faster than other vehicles, may attempt to overtake cars around the fourth information providing billboard 104 from the rear. As described above, by propagating information about a speeding vehicle entering from behind a region where a speeding vehicle is expected to enter, a vehicle accident can be prevented in advance.

15 is a view showing a method of selecting an information providing guide electronic board for outputting emergency vehicle-related information according to an embodiment of the present invention. In FIG. 15, the dotted line means the path R of the emergency vehicle 600. Descriptions of the same items as in FIG. 15 to FIG. 14 will be omitted.

Emergency vehicles (fire trucks, police cars, ambulances, etc.) on the road need to reach their destinations quickly, and the transmission and propagation of traffic-related information is required.

According to an embodiment of the present invention, the relay server receives emergency vehicle route information, which is information about the route of the emergency vehicle, and emergency vehicle number information, which is vehicle number information of the emergency vehicle (from an external agency, etc.) through a wired/wireless communication network. Can be. In this case, the relay server may determine the lane of the emergency vehicle based on the detection image information including the emergency vehicle number information at the latest. According to the situation of FIG. 15, an external sensing device that generates sensing image information including emergency vehicle number information is the first external sensing device, and a relay server is based on the sensing video information generated by the first external sensing device. By doing so, the emergency vehicle 600 may detect a driving lane (eg, one lane). In addition, the relay server may select at least one information providing guide display board that the emergency vehicle is determined to pass based on the received emergency vehicle route information, map information, and output device location information. First, as in the case of FIG. 14, the relay server may predict the position of the emergency vehicle 600 based on the position of the external sensing device 201 that has passed the last time and the time that passes the sensing area. Then, based on the received route information R, it can be determined that the emergency vehicle 600 will pass through the second, third, and fourth information providing guide signs later. In addition, the relay server may output information indicating the driving of the emergency vehicle and information indicating the lane of the emergency vehicle through the selected information providing guide electronic board. That is, the relay server can output the output information indicating that the emergency vehicle 600 will travel in the first lane through the second, third, and fourth information providing information billboards, thereby emptying the first lane on the path R. You can urge other drivers.

Meanwhile, the relay server may generate at least one or more types of output information based on information received from the outside, the surrounding environment information, and the detected image information. According to a preferred embodiment of the present invention, output information indicating an emergency situation has a first priority, output information related to safety excluding an emergency situation has a second priority, and output information related to speeding is a third priority. With, the remaining output information may have a fourth priority. Here, for the natural number n, the nth priority may mean that the nth priority is high. In addition, when there is no output information corresponding to a higher priority, the relay server may cause the output information corresponding to the lower priority to be output through the information providing information display board. That is, the system can be operated more effectively by allowing information to be output based on information having higher priority.

Here, output information corresponding to each priority is as follows.

First, the output information corresponding to the first priority detects the occurrence of an accident based on the information received from the outside, the surrounding environment information, and the detected image information, and notifies the occurrence of the accident based on the detection result. Output information including information or information indicating that an emergency vehicle is driving may be included. In other words, such urgent information needs to be quickly propagated between drivers driving on the road and can be given priority over any other information.

The output information corresponding to the second priority may include information indicating that a vehicle (ie, a dangerous vehicle) that has passed through each of the detection regions is approaching at a speed exceeding the limiting speed of the detection region of a preset number or more. This will prevent car accidents.

In addition, the output information corresponding to the third priority includes speeding vehicle number information and excess speed information of the vehicle passing through the detection area at a speed exceeding the speed limit of one detection area, and the driver violating the speed limit Can be output to awaken the targeted awareness.

Finally, the output information corresponding to the fourth priority may be information including at least one of information about convenience facilities around the information providing information display board, event guide information, weather information, and news. Through this, it is possible to provide various useful information to drivers.

The above-described priorities and detailed examples for each priority are only examples, and the type and output method of the output information may be implemented in different ways.

According to an embodiment of the present invention, it is possible to automatically recognize an object and intelligently provide traffic-related information suitable for each situation according to whether the object is a pedestrian or a vehicle.

Further, according to an embodiment of the present invention, only information regarding the speed of the speeding vehicle may be selected and provided.

In addition, according to an embodiment of the present invention, it is possible to effectively calculate a start time and a hold time for outputting information on speed.

In addition, according to an embodiment of the present invention, a dangerous vehicle is automatically selected, and a warning for the driver of the dangerous vehicle is performed by warning of the emergence of the dangerous vehicle through the information display guide board of the road where the dangerous vehicle is expected to pass. And increase awareness of surrounding drivers. This can also prevent traffic accidents.

In addition, by outputting information about the emergence of the emergency vehicle and the lane in which the emergency vehicle is in use, it is possible to quickly secure a passage through which the emergency vehicle can pass, thereby helping to resolve an emergency situation.

Although the present invention has been described above through specific embodiments, those skilled in the art will be able to make modifications and changes without departing from the spirit of the present invention. Therefore, it should be interpreted that a person belonging to the technical field to which the present invention pertains can be easily inferred from the detailed description and examples of the present invention.

100: Information providing information display board
200: external sensing device
300: relay server

Claims (23)

In the information display system for providing information customized for each object using deep learning-based object recognition including at least one information providing information display board,
The information providing guide electronic board,
An image capture unit that captures an image and generates sensed image information,
An image processing unit that performs pre-processing on the detected image information and separates a background and an object from the captured image,
An object recognition unit that recognizes an object separated from the background and divides the recognized object into a pedestrian or a vehicle,
A surrounding environment detection unit that detects changes in the surrounding environment and generates surrounding environment information,
An information collection unit for storing the detected image information, the object classified as the pedestrian or the vehicle, and the surrounding environment information,
Based on at least one of the surrounding environment information and the sensed image information, learning and model generation are performed, and customized information for each object, which is customized information corresponding to a type of object, based on the recognized object and the learning and model generation results. Information processing unit to generate,
Includes an information display unit for outputting the customized information for each object as at least one of audio, light, or video,
The information processing unit,
Based on the information stored in the information collection unit
Pedestrian-related statistics including the number of pedestrians over time and the number of pedestrians using pedestrian crossings over time, and
Calculate car-related statistics including the number of cars over time and the average speed of cars over time,
Create a traffic information statistical model over time based on the pedestrian-related statistics and the vehicle-related statistics,
When an object as a pedestrian is recognized from the detected image information,
Pedestrian personalized information is generated based on the time when the pedestrian object is recognized and the traffic information statistical model, and the personalized personal information is output through the information display unit.
When the vehicle object is recognized from the detected image information,
Based on the time at which the vehicle object is recognized and the traffic information statistical model, vehicle-specific information is generated, and the vehicle-specific information is output through the information display unit, for each object using deep learning-based object recognition. Information display system for customized information. According to claim 1,
The information processing unit,
The number of cars in the traffic information statistical model corresponding to the time when the object, which is a pedestrian, is recognized, exceeds a preset limit number of cars,
When the average speed of the vehicle of the traffic information statistical model corresponding to the time when the object as the pedestrian is recognized exceeds a preset limit vehicle speed,
A system for providing customized information for each object using deep learning-based object recognition, characterized by including a message prompting attention when using a pedestrian crossing in the pedestrian customized information. According to claim 1,
The information processing unit,
The number of pedestrians using the pedestrian crossing of the traffic information statistical model corresponding to the time when the vehicle object is recognized exceeds the preset number of pedestrians using the limited pedestrian crossing,
When the time at which the vehicle object is recognized is included in a preset night time zone, and when the number of pedestrians in the traffic information statistical model corresponding to the time at which the vehicle object is recognized exceeds the preset number of night pedestrians,
A system for providing customized information for each object using deep learning-based object recognition, characterized in that a message for prompting attention to pedestrians is included in the vehicle customized information. According to claim 3,
The object recognition unit further divides the pedestrian into an adult and a minor,
The information processing unit,
Generate pedestrian-related statistics including the number of pedestrians as adults over time, the number of pedestrians as minors over time, and the number of adults using pedestrian crossings over time.
The number of pedestrians as minors of the traffic information statistical model corresponding to the time when the vehicle object is recognized exceeds a preset limit of the minors pedestrian,
When the number of minors using the pedestrian crossing of the traffic information statistical model corresponding to the time when the vehicle object is recognized exceeds the preset number of minors using the limited pedestrian crossing,
A system for providing customized information for each object using deep learning-based object recognition, characterized by including a message urging attention to a minor pedestrian when passing a pedestrian crossing in the vehicle customized information. According to claim 1,
The surrounding environment sensing unit includes a temperature sensing means and an air quality measurement means,
The surrounding environment information includes information about temperature and air quality,
The information processing unit,
Information system for providing personalized information for each object using deep learning based object recognition, characterized in that the pedestrian personalized information or the automobile personalized information includes information about the temperature and information about the air quality. According to claim 1,
The information collecting unit further stores information on the forbidden parking time range, which is a range of the forbidden parking time over time,
The information processing unit extracts vehicle number information, which is information about a number of an object that is a vehicle, based on the detected image information,
The information processing unit,
When the range of the time at which the object that is the vehicle has been stopped from the time at which the vehicle started to stop is included in the forbidden parking stop time range,
The stopped vehicle is determined as a parking violation vehicle,
A system for providing information customized for each object using deep learning-based object recognition, characterized by outputting together with the information display unit a message for urging the prohibition of parking and the vehicle number information of the vehicle violating the parking. According to claim 1,
The information-based information display system that provides customized information for each object using the deep learning-based object recognition
An external sensing device that senses a change in the surrounding environment to generate surrounding environment information, and captures an image of a preset sensing area of the road to generate sensing image information;
Providing customized information for each object using the deep learning-based object recognition relays transmission and reception of signals, information, or data between devices included in the information display system, and traffic-related output information based on the surrounding environment information and the detected image information It further includes a relay server to generate,
The information providing guide electronic board outputs at least one of the output information and the customized information for each object,
The relay server,
At least one object passing through the detection area is individually recognized based on the detection image information,
Extract vehicle number information, which is information about the number of objects that are automobiles,
Extracting vehicle speed information that is information about the speed of an object that is a vehicle corresponding to the vehicle number information,
The vehicle number information of at least one vehicle passing through the detection area at a speed exceeding the speed limit of the detection area is selected as respective speeding vehicle number information,
The vehicle speed information when an individual vehicle corresponding to each speeding vehicle number information passes through the detection area is selected as each excess speed information,
Personalized information providing information display board system for each object using deep learning based object recognition, characterized in that outputting the speeding vehicle number information and the excess speed information through the information providing information display board. The method of claim 7,
The guide system for providing customized information for each object using the deep learning-based object recognition includes a plurality of external sensing devices and a plurality of information providing guide billboards,
One external sensing device and one information providing information display board form one road sensing group,
The relay server,
When outputting output information generated based on sensed image information from an external sensing device, the deep learning based feature is characterized in that the output information is output through an information providing guide electronic board included in the same road sensing group as the external sensing device. Information display system that provides customized information for each object using object recognition. The method of claim 8,
The sensing area of the external sensing device is located at a predetermined distance from the information providing information billboard included in the same group as the external sensing device,
The relay server,
Based on the arrangement distance and excess speed information or vehicle speed information, calculate an information output start time, which is a time to start outputting output information, and a time to maintain output of output information, which is a time to start outputting output information,
When the speeding vehicle number information and the excess speed information are output through the information providing information display board, personalization of each object using deep learning-based object recognition, characterized in that it is output from the start time of the information output to the time of maintaining the information output. Information providing information display system. The method of claim 9,
Information display area, which is a virtual range area that can recognize output information output from the information display board,
The information output start time is an entry time, which is a time predicted for the vehicle to enter the information display area,
The information output maintenance time is calculated based on the difference between the entry time and the entry time, which is the time when the vehicle is expected to leave the information display area, and provides information for providing customized information for each object using deep learning-based object recognition. system. The method of claim 10,
The information providing guide electronic board includes light output means for outputting information through light, and outputs output information as light through the light output means,
The area of the information display area is determined based on the ratio of the light intensity of the light output means to the illuminance of the area where the information providing information display board is located,
The relay server adjusts the intensity of light of the light output means according to the illuminance to maintain a constant area of the information display area, and provides a customized information for each object using deep learning-based object recognition. . The method of claim 10,
The relay server,
Based on the detected image information, the detection area entry speed information that is vehicle speed information when entering the detection area, the detection area driving speed information that is vehicle speed information when driving the detection area, and the vehicle speed when entering the detection area Calculate information on the speed of entry into the detection area, which is information,
When at least one of the detection area entry speed information, the detection area driving speed information, and the detection area advance speed information exceeds the speed limit of the detection area, the speed information indicating the highest speed is selected as the excess speed information Information display system that provides customized information for each object using deep learning based object recognition. The method of claim 12,
The relay server,
Predict the trend of the speed of the target vehicle based on the detection area entry speed information, the detection area driving speed information and the detection area entry speed information,
Based on the trend of the predicted speed, information for providing information customized for each object using deep learning-based object recognition, wherein the entry time and the entry time are calculated. The method of claim 9,
The information display system for providing customized information for each object using the deep learning-based object recognition includes a plurality of road detection groups,
The detection area of the second road detection group is located after the detection area of the first road detection group based on the driving direction of the vehicle on the road,
The information display guide board of the first road detection group is located in the detection area of the external detection device of the second road detection group,
The relay server,
The first average speed which is the average value of the vehicle speed information determined from the first detection image information taken by the external detection device of the first road detection group and the second detection image information taken by the external detection device of the second road detection group A traffic condition between the first detection area and the second detection area is determined based on a difference between a second average speed which is an average value of vehicle speed information determined from
The information output start time and the information output maintenance time of the information providing guide display board of the second road detection group are determined based on the traffic condition,
When the second average speed is less than a value obtained by subtracting the first average speed from a predetermined speed value, or when the second average speed is less than a predetermined ratio compared to the first average speed, information on the second road detection group is provided. An information display system that provides customized information for each object using deep learning-based object recognition, which also outputs output information that prompts attention through the electronic display. The method of claim 8,
The information display system for providing customized information for each object using the deep learning-based object recognition includes a plurality of road detection groups,
The relay server,
When it is determined that a vehicle corresponding to the same vehicle number information has passed at a speed exceeding the speed limit of each detection area in the road detection group having a predetermined number or more,
The vehicle number information is selected as a dangerous vehicle number, and a vehicle corresponding to the dangerous vehicle number is selected as a dangerous vehicle,
Deep learning-based object recognition characterized by outputting all the excess speed information of the vehicle corresponding to the dangerous vehicle number and the dangerous vehicle number through the information display guide board of the other road detection group recognizing the dangerous vehicle number. Information display system that provides customized information for each object used. The method of claim 15,
The information providing guide electronic board includes light output means for outputting information through light, and outputs output information as light through the light output means,
The relay server,
As the number of detection zones through which the dangerous vehicle has passed at a speed exceeding the speed limit increases,
The size of the font of the dangerous vehicle number output through the information display electronic board increases,
The intensity of the light output to increase the dangerous vehicle number is increased,
An emphasis effect was added to highlight dangerous vehicle numbers,
The emphasis effect includes a number of borders, the thickness of the border, the number of flashing frequency, information display system for providing information customized for each object using deep learning-based object recognition. The method of claim 16,
The light output means outputs light corresponding to a plurality of colors,
The relay server,
As the number of detection areas that a dangerous car passes at a speed exceeding a speed limit increases, the color of the dangerous vehicle number output through the information display guide board approaches to red, using deep learning-based object recognition. Information display system that provides customized information for each object. The method of claim 15,
The relay server,
Map information, including information on topography, buildings, and roads for each location, location information on external sensing devices, which is information on the location of each external sensing device, and output device location information, which is information on the location of each information providing guide display board Save it,
Based on the map information and the location of the external sensing device corresponding to the detection area through which the dangerous vehicle has passed, an estimated route determined to be driven by the dangerous vehicle is calculated,
Selecting at least one information providing information display board located in the expected path based on the output device location information,
Through the selected information providing guide electronic board, a message for warning that a dangerous vehicle is approaching from the rear is output. A guide system for providing personalized information for each object using deep learning-based object recognition. The method of claim 7,
The guide system for providing customized information for each object using the deep learning-based object recognition includes a plurality of external sensing devices and a plurality of information providing guide billboards,
The relay server,
Map information, including information on topography, buildings, and roads for each location, location information on external sensing devices, which is information on the location of each external sensing device, and output device location information, which is information on the location of each information providing guide display board Save it,
When receiving emergency vehicle route information, which is information on the route of the emergency vehicle, and emergency vehicle number information, which is vehicle number information of the emergency vehicle,
Lastly, the lane of the emergency vehicle is determined based on the detected image information including emergency vehicle number information.
Based on the emergency vehicle route information, the map information and the output device location information, and screening at least one information providing information display board that is determined to pass the emergency vehicle,
Through the selected information providing information display board, the information indicating the driving of the emergency vehicle and the information indicating the lane of the emergency vehicle are output. The method of claim 7,
The relay server generates at least one kind of output information based on information received from the outside, the surrounding environment information, and the detected image information,
The output information indicating the emergency situation has a first priority,
Output information about safety excluding emergency situations has a second priority,
Output information about speeding has a third priority,
The remaining output information has a fourth priority,
For the natural number n, the nth priority means that the nth priority is high,
When there is no output information corresponding to the priority of the upper level, the output information corresponding to the priority of the lower level is provided through the information providing information display board. Billboard system. The method of claim 20,
The output information corresponding to the first priority,
The relay server detects the occurrence of an accident based on information received from the outside, surrounding environment information, and detection image information, and output information including emergency information based on the detection result or an emergency vehicle is traveling. Contains information that informs you that
The output information corresponding to the second priority,
It includes information informing that vehicles passing through the detection areas are approaching at a speed exceeding the limiting speed of the detection areas of a predetermined number or more,
The output information corresponding to the third priority,
Includes speeding vehicle number information and excess speed information of the vehicle passing through the detection area at a speed exceeding the speed limit of one detection area,
The output information corresponding to the fourth priority,
Information-providing information-based information display system that provides information customized for each object using deep learning-based object recognition, which is information that includes at least one of information about convenience facilities, event information, weather information, and news. In the control method of the information display system that provides customized information for each object using deep learning-based object recognition,
Generating an image sensing information by photographing an image by the information providing information display board;
The information providing guide electronic board performing pre-processing on the detected image information and separating a background and an object from the captured image;
Recognizing an object separated from the background, and classifying the recognized object into a pedestrian or a vehicle;
Generating a surrounding environment information by detecting a change in the surrounding environment by the information providing guide electronic board;
The information providing guide electronic board receiving wired/wireless communication with the outside to receive information from the outside;
Storing the detected image information, the object classified as the pedestrian or the vehicle, and the surrounding environment information by the information providing guide electronic board;
The information providing guide electronic board performing learning and model generation based on at least one of the surrounding environment information and the sensed image information;
Generating customized information for each object, which is customized information corresponding to the type of the object, based on the recognized object, the learning and model generation results, by the information providing guide electronic board; And
The information providing guide electronic board includes the step of outputting the customized information for each object as at least one of audio, light, or video,
Step of performing the learning and model generation,
Pedestrian-related statistics, including the number of pedestrians over time and the number of pedestrians using a pedestrian crossing over time, based on the information stored by the information providing information display board.
Calculate car-related statistics including the number of cars over time and the average speed of cars over time,
Create a traffic information statistical model over time based on the pedestrian-related statistics and the vehicle-related statistics,
When an object as a pedestrian is recognized from the detected image information,
In the step of generating the personalized information for each object, generating the personalized information for pedestrians based on the time when the information providing information display board is the pedestrian object and the traffic information statistical model, and outputting the information is the information providing information display board Outputs the pedestrian customized information,
When the vehicle object is recognized from the detected image information,
The information providing guide electronic board generates vehicle personalized information based on the time when the object as the vehicle is recognized and the traffic information statistical model, and the outputting step is characterized in that the information providing guide electronic board outputs the vehicle customized information. Control method of guide information display system that provides customized information for each object using deep learning based object recognition. delete

Images