KR102541407B1 - Realtime traffic information provision system for autonomous driving based on c-its mec - Google Patents

Abstract

The present invention relates to a real-time traffic information providing system for autonomous driving based on C-ITS MEC, and more particularly, by providing on-road data collected by a C-ITS infrastructure device installed on a roadside to a nearby C-ITS infrastructure device. Real-time traffic information provision system for autonomous driving based on C-ITS MEC that can provide real-time traffic information by reducing information transmission latency as the C-ITS MEC server installed in the will be. A system for providing real-time traffic information for autonomous driving based on C-ITS MEC according to the present invention is connected to a C-ITS infrastructure device that collects on-road data and the C-ITS infrastructure device, and receives and receives the on-road data. processing to generate processed data including safe driving information, and capable of wireless communication with the C-ITS MEC server and the C-ITS infrastructure device for transmitting the generated processed data to the C-ITS infrastructure device, and the C-ITS infrastructure device. and a vehicle receiving the processed data from the ITS infrastructure device.

Description

Real-time traffic information provision system for autonomous driving based on C-ITS MEC {REALTIME TRAFFIC INFORMATION PROVISION SYSTEM FOR AUTONOMOUS DRIVING BASED ON C-ITS MEC}

The present invention relates to a real-time traffic information providing system for autonomous driving based on C-ITS MEC, and more particularly, by providing on-road data collected by a C-ITS infrastructure device installed on a roadside to a nearby C-ITS infrastructure device. Real-time traffic information provision system for autonomous driving based on C-ITS MEC that can provide real-time traffic information by reducing information transmission latency as the C-ITS MEC server installed in the will be.

Recently, vehicles are becoming the result of complex industrial technologies in which electrical, electronic, and communication technologies are converged in the center of mechanical engineering, and in this respect, vehicles are also called smart cars. Smart cars connect drivers, vehicles, and transportation infrastructure to provide various user-customized mobility services as well as vehicle technology in the traditional sense such as traffic safety/complex relief. Such connectivity can be implemented using V2X (Vehicle to Everything) communication technology.

On the other hand, conventionally, in order to process information necessary for cooperative autonomous driving in a V2X environment, road data collected from a C-ITS infrastructure device installed on the roadside is transmitted to a center server connected to a communication network, and the center server that receives the data transmits the data After generating the processed data, the generated processed data is transmitted to the V2X RSU (ROADSIDE UNIT, roadside base station) again through the communication network, and the V2X RSU transmits the received processed data to the vehicle in the V2X method to process the data. It is used for autonomous driving of the received vehicle. However, information delay may occur in the process of exchanging data through a communication network, which greatly affects safe driving in an autonomous driving environment requiring real-time information.

Therefore, in order to collect, process, and provide real-time information necessary for autonomous driving with V2X technology, a local MEC-based C-ITS infrastructure is required to minimize delay, that is, Latency. That is, after processing and processing data on the road by the MEC server installed near the C-ITS infrastructure device and connected to the local, C-ITS can minimize the delay by transmitting the processed data to the vehicle, etc. There is a need for a real-time traffic information providing system for MEC-based autonomous driving. Since autonomously driving vehicles require real-time information, data received from field equipment such as signal controllers, CCTVs, RADARs, pedestrian detectors, and sudden detectors is processed by the center server and then autonomously operated through V2X RSU, one of the C-ITS infrastructure devices. When transmitted to a driving vehicle, delay time may occur, which may affect safe driving.

In addition, as the need for cooperative autonomous driving technology has emerged globally, C-ITS infrastructure such as local MEC server, V2X RSU, signal controller, radar, lidar, CCTV, etc. It is required to secure the core technology necessary for rapidly changing autonomous driving technology by establishing and demonstrating a system that can be directly linked to

Republic of Korea Patent Registration No. 10-2145723 (Announced on August 19, 2020) Republic of Korea Patent Registration No. 10-2103823 (Announced on April 24, 2020)

An object of the present invention to solve the above problems is to process the role of the center server for processing and processing data in the MEC server connected to the V2X RSU locally, thereby reducing the delay time of the data provided to the vehicle. C-ITS MEC It is to provide a real-time traffic information providing system for autonomous driving.

Another object of the present invention is real-time traffic for autonomous driving based on C-ITS MEC, which can process sudden detection, traffic situation, pedestrian detection, etc. using a video surveillance device (CCTV) in one MEC server. It is to provide an information system.

Another object of the present invention is a C-ITS infrastructure device such as a video surveillance device, a radar, a lidar, a signal controller, a meteorological sensor, and a V2X RSU ( It is to provide a real-time traffic information providing system for autonomous driving based on C-ITS MEC, which has simplified C-ITS infrastructure devices and improved aesthetics by providing power and communication lines necessary for such as RoadSide Unit) in a system including an MEC server.

Another object of the present invention is that the vehicle receiving data by the V2X RSU included in the C-ITS infrastructure device includes a V2X module, a multi-hop type V2X data transmission device and a multi-hop type V2X data reception device, A C-ITS MEC-based real-time traffic information providing system for autonomous driving that uses a vehicle equipped with a module as a router and hops as much as a predetermined number of hops to transmit data to the vehicle at the final destination. is to provide

In order to achieve the above object, a real-time traffic information providing system for autonomous driving based on C-ITS MEC according to an embodiment of the present invention includes a C-ITS infrastructure device for collecting road data, the C-ITS infrastructure The C-ITS MEC server and the C-ITS MEC server that is connected to the device, receives and processes the data on the road, generates processed data including safe driving information, and transmits the generated processed data to the C-ITS infrastructure device. and a vehicle capable of wireless communication with the ITS infrastructure device and receiving the processed data from the C-ITS infrastructure device.

The C-ITS infrastructure device is installed along the road, and includes a video monitoring device, a radar, a lidar, a signal controller, and a meteorological sensor. ) and V2X RSU (RoadSide Unit), and the C-ITS MEC server is installed on the roadside, GPU (Graphics Processing Unit) unit, C-ITS infrastructure interface unit, communication unit, processing unit, storage unit and power distribution board unit includes

The safe driving information includes emergency situation information and traffic information on the road, and the processing unit includes an unexpected situation detecting unit that detects an unexpected situation on the road based on the processed data and when an unexpected situation is detected by the unexpected situation detection unit. and a driver warning unit generating alarm information to be transmitted to the vehicle when the vehicle is detected, and the emergency situation detection unit includes a pedestrian detection unit detecting a pedestrian on the road and a traffic information collection unit collecting traffic information on the road.

The communication unit transmits the generated processed data to the center server through a 5G network.

The vehicle is equipped with a V2X data transmission device and a V2X data reception device, and the transmission device includes an extended information header including hopping information in data received from the V2X RSU or data generated in the vehicle to send a V2X message. a generating unit that generates; And a transmitter for transmitting the V2X message, wherein the receiver includes a receiver for receiving the V2X message, a processor for processing the V2X message, and a retransmitter for retransmitting the V2X message according to the hopping information.

According to the present invention C-ITS MEC-based real-time traffic information providing system for autonomous driving, the role of the center server for processing and processing data is processed by the MEC server connected locally to the V2X RSU and the delay time of the data provided to the vehicle There is a possible effect of reducing .

According to the C-ITS MEC-based real-time traffic information providing system for autonomous driving according to the present invention, sudden detection using a video monitoring device (CCTV), traffic situation, pedestrian detection, etc. are processed in one MEC server and data There is a possible effect of reducing the delay time of .

According to the real-time traffic information providing system for autonomous driving based on C-ITS MEC, which is the present invention, C-ITS infrastructure devices such as a video surveillance device, a radar, a lidar, and a signal controller Autonomy based on C-ITS MEC, which simplifies C-ITS infrastructure devices and improves aesthetics by providing power and communication lines necessary for MEC servers, etc. It is possible to provide a real-time traffic information providing system for driving.

According to the real-time traffic information providing system for autonomous driving based on the C-ITS MEC of the present invention, the vehicle receiving data by the V2X RSU included in the C-ITS infrastructure device is a V2X module, a multi-hop type V2X data transmission device and Including a multi-hop type V2X data receiving device, a vehicle equipped with a V2X module is used as a router, and there is an effect of transmitting data to a vehicle of a final destination by hopping as many as a predetermined number of hops. .

1 and 2 are block diagrams briefly showing the configuration of a system for providing real-time traffic information for autonomous driving based on C-ITS MEC according to the present invention.
3 is a block diagram briefly showing the configuration of a C-ITS infrastructure device according to the present invention.
4 is a block diagram briefly showing the configuration of a C-ITS MEC server according to the present invention.
5 is a block diagram briefly showing the configuration of a processing unit according to the present invention.
6 is a block diagram briefly showing the configuration of a V2X terminal mounted on a vehicle according to the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible, even if they are displayed on different drawings.

And, in describing the embodiments of the present invention, if it is determined that a detailed description of a related known configuration or function hinders understanding of the embodiments of the present invention, the detailed description will be omitted.

Also, terms such as first, second, A, B, (a), and (b) may be used to describe components of an embodiment of the present invention. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term.

In this specification, the singular form also includes the plural form unless otherwise specified in the phrase. The terms "comprising" and/or "comprising" used in the specification do not exclude the presence or addition of one or more other components other than the recited components.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram briefly showing the configuration of a real-time traffic information providing system for autonomous driving based on C-ITS MEC according to the present invention.

Referring to FIG. 1, a C-ITS MEC-based real-time traffic information providing system for autonomous driving according to the present invention includes a C-ITS infrastructure device 100, a C-ITS MEC server 200, and a vehicle 300. include Referring to FIG. 2, the system for providing real-time traffic information for autonomous driving based on C-ITS MEC according to the present invention further includes a center server 400 communicating with the C-ITS MEC server 200 through a 5G network do.

3 is a block diagram briefly showing the configuration of a C-ITS infrastructure device according to the present invention.

Referring to FIG. 3 , the C-ITS infrastructure device 100 according to the present invention is installed on a roadside and collects data on the road. The C-ITS infrastructure device 100 transmits the collected road data to the C-ITS MEC server 200. The data on the road can be the basis for processed data including safe driving information such as image information, traffic information, accident information, signal information, pothole information, and signal information generated by the C-ITS MEC server 200 to be described later. . In the C-ITS infrastructure device 100, each component may be integrated and installed on a pole installed on a roadside.

The C-ITS infrastructure device 100 according to the present invention may include a configuration for collecting road data, for example, a video monitoring device 110 (映像監視裝置), a radar 120 (Radar ), lidar 130 (Lidar), signal controller 140 (信號制御機), weather sensor 150 (Meteorological Sensor), and for V2X (Vehicle to everything) communication with the vehicle 300 It includes a V2X RSU (160) (RoadSide Unit). The C-ITS infrastructure device 100 may further include a traffic information collection device (not shown), a smart pedestrian signal system (not shown), and a pedestrian protection driver warning system (not shown).

The video monitoring device 110 according to the present invention may be a CCTV (closed circuit television) or a camera that generates video data by photographing the road. The video monitoring device 110 may be a plurality of CCTVs having different purposes, and each CCTV may collect road data including images of different predetermined areas. That is, a plurality of CCTVs with different uses are provided, and images collected by the CCTVs can be processed through an image analysis algorithm in one MEC server, and at this time, as an example, images of up to 4 channels can be processed. On the other hand, in this specification, on the road (道路 上) includes the roadside (道路邊).

The radar 120 is an abbreviation of Radio Detecting And Ranging, and transmits electromagnetic waves of the order of microwaves (ultra-high waves, 10 cm to 100 cm wavelength) to an object, receives electromagnetic waves reflected from the object, and measures the distance to the object, As a wireless monitoring device that detects direction and altitude, it is a device that measures the direction and distance of an object by measuring the time until the time the reflected wave is received using the straightness of the radio wave. LiDAR 130 is a device that precisely draws the surroundings by emitting a laser pulse, receiving the light reflected from a surrounding target object and measuring the distance to the object, etc. has the same principle as traditional radar, but the wavelength of electromagnetic waves used is different. The LIDAR 130 may measure not only the distance to the target object but also the speed and direction of movement. The LIDAR 130 is used as a sensor that acquires information necessary to implement a 3D image. The signal controller 140 refers to a device that controls traffic signals, and can collect information on traffic signals as well as control traffic signals. The weather sensor 150 is a sensor that detects weather, and may collect and generate weather information provided to the vehicle 300 autonomously traveling on the road.

The V2X RSU 160 is a kind of base station installed on a roadside to communicate with a V2X module installed in the vehicle 300 in a V2X manner, and is generally referred to as a roadside base station. The V2X RSU 160 may communicate with the vehicle 300 equipped with the V2X module, the C-ITS MEC server 200, or the center server 400. At this time, the V2X RSU 160 may perform V2X communication with the vehicle 300, local communication with the C-ITS MEC server 200, and 5G communication with the center server 400. V2X has a WAVE method and a C-V2X method, and the V2X RSU 160 according to the present invention can support both the WAVE method and the C-V2X method. More details on the V2X RSU 160 will be described later.

The traffic information collection device refers to a device that collects traffic information on the road, and the smart pedestrian signal system prevents accidents by automatically requesting a green signal by distinguishing between ordinary pedestrians and crossers at a crosswalk and notifying the danger of entering the roadway. It refers to a system that provides information such as signal waiting time and expected signal end time. The smart pedestrian signal system can transmit and receive information with the signal controller 140. Pedestrian protection driver warning system refers to a system that sends an alarm notification to the driver when a pedestrian appears. The pedestrian protection driver warning system may transmit/receive information to and from a pedestrian detection unit to be described later.

Each component of the C-ITS infrastructure device 100 according to the present invention collects road data such as vehicles 300, pedestrians, construction, and weather on the road, and the collected road data is connected to the C-ITS through local communication. It is transmitted to the MEC server 200.

The C-ITS MEC server 200 according to the present invention is installed near the roadside of the C-ITS infrastructure device 100. Preferably, the C-ITS MEC server 200 may be installed together with the support on which the C-ITS infrastructure device 100 is installed. The C-ITS MEC server 200 may support or share power and communication lines of the C-ITS infrastructure device 100 .

The C-ITS MEC server 200 receives and processes the received road data to generate processed data including safe driving information. The C-ITS MEC server 200 may be connected to a local location near the C-ITS infrastructure device 100 to reduce delay time in transmitting and receiving data. That is, after transmitting data on the road to the center server 400 using the communication network, processing the data on the road received by the center server 400, and then passing the processed data back through the communication network, the C-ITS infrastructure device 100 ), the transmission and reception of data can be faster than transmission.

The C-ITS MEC server 200 generates processing data including the received safety driving information, and transmits the generated processing data to the locally connected C-ITS infrastructure device 100 again. As an embodiment, the C-ITS MEC server 200 includes a communication unit 230 for local communication with the V2X RSU 160. The C-ITS MEC server 200 transmits the generated processing data to the V2X RSU 160 included in the C-ITS infrastructure device 100, and the V2X RSU 160 receiving the processing data is a nearby vehicle 300 send to The vehicle 300 is capable of wireless communication with the C-ITS infrastructure device 100 . In detail, the vehicle 300 is equipped with a V2X module capable of communication of the V2X method. The V2X RSU 160 included in the C-ITS infrastructure device 100 transmits processed data to the V2X module mounted on the vehicle 300 through the V2X method. The vehicle 300 receiving the processed data may transmit the processed data to the vehicle 300 equipped with another V2X module through a V2X communication method, which will be described later.

4 is a block diagram briefly showing the configuration of a C-ITS MEC server according to the present invention.

Referring to FIG. 4, the C-ITS MEC server 200 according to the present invention includes a GPU (Graphics Processing Unit) unit, a C-ITS infrastructure interface unit 220, a communication unit 230, a processing unit 240, a storage It includes a unit 250, a firewall unit, a PoE processing unit, a UDP processing unit, an optical converter unit, and a power distribution board unit 260.

The GPU unit 210 according to the present invention includes a graphic processing device and processes an image captured by an image sensing device. As described above, the image sensing device includes a plurality of different CCTVs, and simultaneously processes images captured by each CCTV. At this time, based on the image processed by the GPU unit 210, the processing unit 240, which will be described below, performs image analysis, and performs emergency situation detection, traffic situation collection, pedestrian detection, etc. through image analysis. The C-ITS infrastructure interface unit 220 includes a communication interface for transmitting and receiving information from each component included in the C-ITS infrastructure device 100 . The C-ITS MEC server 200 may communicate with each component of the C-ITS infrastructure device 100 through PoE, communicate with the signal controller 140 through 10BT, and communicate with the radar 120 through RS- 485 standard for communication. The communication unit 230 may process communication packets transmitted and received with the C-ITS infrastructure device 100, the V2X RSU 160, the V2X module or the center server 400 mounted on the vehicle 300. The communication unit 230 may have specifications of 2.4GHz, 6core, HDD1.2TB, 1000BaseTX*4port or more. The processing unit 240 processes road data collected by each component of the C-ITS infrastructure device 100 to generate processed data. The processing unit 240 may have specifications of 2.4GHz, 6core, HDD1.2TB, 1000BaseTX*4port or more. The storage unit 250 stores road data received by the communication unit 230 or processed data processed by the processing unit 240 . The storage unit 250 may correspond to a cache. The storage unit 250 may have specifications of 2.4GHz, 6core, HDD1.2TB, 1000BaseTX*4port or more. The firewall unit can have 2.4GHz Memory 4GB, Hdd500GB, 10/100/1000BaseTX*8port, or more specifications than NIS CC certification. The power distribution board 260 may receive power from the outside. The power distribution board 260 is electrically connected to each component of the C-ITS infrastructure device 100 to supply power to each component of the C-ITS infrastructure device 100. As the power distribution board 260 supplies power to each component of the C-ITS infrastructure device 100, each component included in the C-ITS infrastructure device 100 does not have to have a separate power line, so the C-ITS infrastructure device (100) can be simplified and aesthetically improved.

The processing unit 240 according to the present invention receives and processes road data to generate processed data. The processing data includes safe driving information, and the safe driving information includes image information, traffic information, accident information, signal information, pothole information, signal information, and the like. Safe driving information includes unexpected situation information and traffic information on the road.

5 is a block diagram briefly showing the configuration of a processing unit according to the present invention.

Referring to FIG. 5, the processing unit 240 according to the present invention detects an unexpected situation on the road based on the processing data, and the vehicle when an unexpected situation is detected by the unexpected situation detection unit 241 and the unexpected situation detection unit 241 and a driver warning unit 245 generating warning information to be transmitted to (300).

The unexpected situation on the road detected by the unexpected situation detection unit 241 according to the present invention includes an unexpected situation caused by construction work on the road, an unexpected situation caused by vehicles 300 on the road, and an unexpected situation caused by a pedestrian on the road.

The unexpected situation detecting unit 241 according to the present invention includes a construction detecting unit (not shown) that detects an unexpected situation due to construction on a road or the like. The construction detection unit monitors the construction status on the road and detects the fact that construction is in progress or an increase or decrease in the scope of construction progress to detect an unexpected situation due to construction on the road or the like.

The unexpected situation detecting unit 241 according to the present invention includes a vehicle detecting unit (not shown) that detects an unexpected situation caused by the vehicles 300 on the road. The vehicle detecting unit detects an unexpected situation caused by the vehicles 300 on the road by detecting congestion of the vehicle 300 on the road, the presence or absence of the vehicle 300 stopped or in an accident, and the vehicle 300 traveling in reverse.

The unexpected situation detection unit 241 according to the present invention includes a pedestrian detection unit 242 that detects a pedestrian on the road in order to detect an unexpected situation caused by a pedestrian on the road.

The unexpected situation detection unit 241 according to the present invention further includes a traffic information collection unit 243 that collects traffic information on the road. The traffic information collection unit 243 collects traffic information, and the traffic information includes traffic volume or queue.

The driver warning unit 245 according to the present invention generates alarm information including information about the unexpected situation when the unexpected situation detecting unit 241 detects the aforementioned unexpected situation. The alert information generated by the driver alert unit 245 is included in the processed data and transmitted to the V2X RSU 160, and the V2X RSU 160 transmits the received processed data to the nearby vehicle 300. The alarm information included in the processing data may be, for example, a still image or a short video, and when the driver of the vehicle 300 wants to check the cause of congestion, a navigation device installed in the vehicle 300 or an HMI including a smartphone (human machine interface), when the emergency situation button is pressed, the V2X module mounted on the vehicle 300 in a stopped state broadcasts video information in a multi-hopping mode, and the V2X module of another vehicle 300 in the middle relays it to cause an unexpected event As the image is displayed on the HMI of the desired vehicle 300, an unexpected situation can be confirmed. Details related to multi-hopping will be described later.

On the other hand, referring to FIG. 2 again, the communication unit 230 of the C-ITS MEC server 200 transmits the processed data to the V2X RSU 160 included in the C-ITS infrastructure device 100 and generates Processing data may be transmitted to the center server 400 . The center server 400 may be used when calculating statistical data after receiving and storing processed data. Meanwhile, the communication unit 230 may transmit processed data to the center server 400 through a 5G communication network. The communication unit 230 and the center server 400 further include a 5G communication module capable of 5G communication.

6 is a block diagram briefly showing the configuration of a V2X terminal mounted on a vehicle according to the present invention.

The C-ITS infrastructure apparatus 100 according to the present invention includes a V2X RSU 160 that transmits processed data received from the C-ITS MEC server 200 to the vehicle 300 through wireless communication. Referring to FIG. 6 , the vehicle 300 according to the present invention is equipped with a V2X module that receives processed data from the V2X RSU 160. The V2X module receives V2X data from the V2X data transmission device 310 and the V2X RSU 160 for transmitting V2X data, or V2X data for receiving V2X data from the V2X data transmission device 310 of another vehicle 300. A receiving device 320 is included.

Data packets according to the present invention are generated (transmitting end) and processed (receiving end) in the application layer (layer). Therefore, when a data packet is created, productivity can be improved because it can be created without worrying about packet communication or subsequent modifications.

The V2X data transmission device 310 (hereinafter referred to as 'transmission device') includes a generator (not shown) that generates packets containing hopping information and a transmitter that transmits packets through wireless communication using V2X communication standards according to the hopping information (not shown). city), including

The generation unit according to the present invention generates a packet containing hopping information. A packet generated by the generation unit includes a header and a payload, and the header includes a first header, a second header, and a third header. The payload includes the remaining data of the packet excluding the header. In general, the address or order to which data is to be delivered is recorded in the header of the packet.

The header of the packet according to the present invention includes hopping information. In this specification, hopping may mean a state in which data is transmitted and received between nodes. The V2X data transmission device 310 and the reception device 320 according to the present invention are included in the vehicle 300 and may transmit and receive data between the vehicles 300. At this time, each vehicle 300 may serve as a router. Therefore, data received from one vehicle 300 to another vehicle 300 or a plurality of other vehicles 300 may be transmitted. In this case, data transfer from one vehicle 300 to another vehicle 300 may be referred to as hopping. In this specification, hopping information may be information required, generated, or modified during hopping. Briefly, information about a source where data is generated, information on a router receiving and retransmitting data in the middle, information on a destination containing information on a final destination, or Information on the number of remaining hopping counts may be included.

The generator according to the present invention generates packets containing hopping information, and includes the generated packets in a Basic Safety Message (BSM) message or a Radio Technical Commission for Maritime Services (RTCM) message. The BSM message is called a safety message and is a message containing basic information of the vehicle 300 related to the location, speed, acceleration, brake operation, etc. of the vehicle 300 in order to provide safety services for the vehicle 300 . BSM messages are generated in the vehicle 300 and delivered to other vehicles 300 . Vehicle 300 thus includes sensors that collect data for generating BSM messages. An RTCM message is GPS correction data used by an RTK module that encapsulates RTCM differential corrections for GPS and other radionavigation signals, as defined in the RTCM Special Committee for Radio Services Committee DSRC media, and then defines a final It can be reconstructed into the expected format. Contains information that is directly used in various positioning systems in accordance with the RTCM standard to increase the absolute and relative accuracy estimates produced. The RTCM information is information that corrects the location information of each object so that the exact location can be known. The transmitter 310 according to the present invention may further include a BSM generating module or RTCM generating module for generating a BSM or RTCM message. RTCM information is encoded and generated as RTCM, which is an international standard message protocol, in a RSU (base station) and transmitted to each vehicle 300 through V2X communication. When the RTCM message is out of the transmission range of the roadside device, smooth delivery becomes difficult. The packet according to the present invention includes hopping information so that each vehicle 300 serves as a router to deliver the BSM message or RTCM message to the final destination so that it can smoothly arrive at the final destination. For example, in the case of RTCM provided by a roadside device, the data providing area is generally 500 meters in radius of the roadside device, but when RTCM is transmitted using hopping according to the present invention, the data providing area can be extended several times. .

The generating unit may include a packet containing hopping information in a BSM message or RTCM message (hereinafter referred to as 'message'). The message includes processed data received from the V2X RSU (160) of the C-ITS infrastructure device (100). Each header of the BSM message or RTCM message may further include an extended information header. Extension information added in each message is recorded in the extension information header. The extension information header may include hopping information according to the present invention, and may further include other additional information.

In summary, the transmitter 310 according to the present invention generates a packet containing hopping information and includes a generator that includes the packet in a message and a transmitter that transmits the message according to the V2X communication standard according to the hopping information. In the V2X data transmission device 310, the message is a BSM (Basic Safety Message) message generated in the vehicle 300 or a RTCM (Radio Technical Commission for Maritime Services) message generated in the roadside base station, the message, The packet includes an extension information header included in the packet, and the packet includes a header in which the hopping information is recorded.

A packet generated by the generator according to the present invention includes a first header, a second header, and a third header.

The first header includes information on the number of remaining hops deducted according to packet transmission according to the V2X module. The remaining hopping count information is deducted by, for example, 1 each time the value included in the maximum hopping count information, which will be described later, is delivered or intermediated by the V2X transmission device 310, and when it reaches 0, the corresponding packet is forwarded. It is information (or value) that disappears from the network without becoming available.

The first header further includes next header information in which the existence of the next header is recorded, version information to identify the version of the corresponding protocol, and information on the remaining time of the packet, i.e., the time a packet can endure until it reaches its final destination. can include The time (value) included in the remaining time information can prevent old messages from remaining in the network.

The second header includes information on the maximum hopping count of the packet. The maximum number of hopping may be set when a first packet is generated. The maximum number of hops is not changed by intermediaries or routers.

The second header further includes transmission method information indicating a transmission method of the packet. Transmission method information may be unicast or topologically-scoped broadcast. In unicast, a specific destination is determined, and messages are delivered by hopping until the message is relayed to the destination. The unicast method can be used when relaying BSM messages. The topological method is a method of mediating messages until the number of remaining hopping counts is exhausted. The topological method can be used when relaying RTCM messages.

The second header further includes next header information in which whether or not the next header exists is recorded, class information in which whether or not the device receiving the relayed message has a corresponding message when there are no neighbors around the device, and length information of the packet excluding the header are further included. can include

A third header includes identification information of the packet. Identification information is information used when detecting duplicate packets. The identification information may be a serial number of the packet. The third header further includes original source information indicating information of a device that generated the packet in addition to the identification information. The initial source information may be a serial number of a device that generated the packet. The identification information and the first source information are used when determining whether the corresponding packet is the same as the previously received packet when the receiving device receives the packet, which will be described later.

The third header further includes final destination information of the packet when the transmission method of the packet included in the second header is unicast. In the unicast method, a message is delivered to a specific destination, and information on the specific destination is included in the third header.

In summary, the header includes a first header, a second header, and a third header, and the first header includes information on the remaining hopping counts deducted according to transmission of the packet and a life in which the remaining time of the packet is stored. time information, the second header includes information on the maximum hopping count of the packet and transmission method information indicating a transmission method of the packet, and the third header includes identification information of the packet and generation of the packet It includes the initial source information indicating the information of a device. If the transmission method of the packet included in the second header is unicast, the third header further includes final destination information of the packet.

The transmission unit according to the present invention transmits the packet generated by the generation unit and included in the message to another V2X receiving device 320. That is, the transmission unit transmits the message to the receiving device 320 mounted on another vehicle 300 . When transmitting a message including a packet, the transmitter may transmit the message to another vehicle 300 or object equipped with the V2X receiving device 320 .

The V2X data receiving device 320 (hereinafter referred to as 'receiving device') according to the present invention includes a Basic Safety Message (BSM) message generated by the vehicle 300 and a Radio Technical Commission for Maritime Services (RTCM) message generated by a roadside device. Included in at least one of the messages, a receiving unit (not shown) receiving a packet containing hopping information, a processing unit (not shown) processing the packet, and a retransmitting unit retransmitting the packet according to the V2X communication standard according to the hopping information (not shown).

The receiving unit according to the present invention receives a message transmitted by the transmitting unit of the transmitting device 310 . Messages received by the receiver are processed by the processing unit. The message received by the receiver includes a packet containing hopping information. A packet includes a first header, a second header, and a third header, and information included in each header is as described above.

The processing unit according to the present invention first determines whether extension information is included in the received message. Extended information, as described above, may be included in the header of the message, and when additional information is recorded in the message, it is recorded in the extended information. The extended information may include hopping information according to the present invention, or additional information other than hopping information may be recorded. Therefore, if the extension information is included in the header of the message, the processing unit processes (decrypts) the received message with the possibility that there may be hopping information according to the present invention.

The processing unit according to the present invention extracts packet hopping information by processing the header of the received message. As described above, the hopping information includes information included in the first header, the second header, and the third header. First, the processing unit checks the remaining time information of the first header and deletes the corresponding packet when the remaining time is 0. If the remaining time information is not 0, the processing unit checks the transmission method information of the second header.

If the transmission method information recorded in the second header is a unicast method, it is checked whether the corresponding packet is a duplicate packet with a previously received packet (or a message duplicated with a previously received message). At this time, the identification information of the third header and the first source information are used. It is determined whether it is a duplicate packet by comparing whether the identification information and the first source information recorded in the corresponding packet are the same as the identification information and the first source information recorded in the previously received packet. In the case of a duplicated packet, the message including the corresponding packet is deleted, and in the case of a non-redundant packet, the final destination information recorded in the third header of the corresponding packet is extracted. When the final destination information and the received information of the receiving device match, the entire corresponding message is processed and stored in the database of the receiving device 320 . If the final destination information and the received information of the receiving device do not match, the value of the remaining hopping count information recorded in the first header of the corresponding packet is subtracted by one. If the subtracted remaining hopping count information is 0, the corresponding packet and message are deleted. If the subtracted remaining hopping count information is not 0, the corresponding packet and message are retransmitted to another V2X receiving device 320 by a retransmitter, which will be described later. That is, if the remaining hopping count information is not 0, the remaining hopping count is deducted and the message is retransmitted. As described above, when the transmission method of the message is the unicast method, the corresponding message is preferably a BSM message.

If the transmission method information recorded in the second header is a topological method, it is checked whether the corresponding packet is a duplicate packet with a previously received packet (or a message duplicated with a previously received message). At this time, the identification information of the third header and the first source information are used. It is determined whether it is a duplicate packet by comparing whether the identification information and the first source information recorded in the corresponding packet are the same as the identification information and the first source information recorded in the previously received packet. In the case of duplicated packets, the message including the corresponding packet is deleted, and in the case of non-redundant packets, the entirety of the corresponding message is processed and stored in the database of the receiving device 320. In the case of the topological method, unlike the unicast method with a specific final destination, if the remaining hopping count is not 0 even after message processing, the received message is retransmitted to another V2X receiving device 320. Therefore, after processing the message, the remaining hopping count information (value) recorded in the first header is subtracted by 1, and if the remaining hopping count information becomes 0, the corresponding message is deleted. If the remaining hopping count information is not 0, another V2X It is retransmitted to the receiving device 320. That is, if the remaining hopping count information is not 0, the remaining hopping count is deducted and the message is retransmitted.

The retransmission unit according to the present invention retransmits the packet generated by the generation unit and included in the message to the receiving device 320 mounted in another vehicle 300 . That is, the retransmitter transmits the message to another receiving device 320 . When transmitting a message including a packet, the retransmitter may transmit the V2X receiving device 320 to other surrounding vehicles 300 or objects. The retransmitter deducts the aforementioned remaining number of hops each time a packet is transmitted.

The retransmitter according to the present invention retransmits the received message to another V2X receiving device 320 when the subtracted remaining hopping count information is not 0. That is, the receiving device 320 serves as the transmitting device 310 . As shown in FIG. 1, the receiving device 320 after the first transmitting device 310 serves as the transmitting device 310 serving as a router until the remaining hopping count information becomes 0. . The retransmitter transmits packets and messages including packets according to the V2X communication standard according to hopping information such as the remaining number of hoppings. That is, the packet and the message including the packet may be transmitted between the V2X data transmission devices 310 until the remaining hopping count becomes 0 and arrive at the final destination.

In summary, the receiving device 320 according to the present invention includes a receiving unit for receiving a message including a packet including hopping information, a processing unit for processing the message, and a retransmission for retransmitting the message according to the V2X communication standard according to the hopping information. In the V2X data receiving device 320 including a unit, the message is a Basic Safety Message (BSM) message generated in the vehicle 300 or a Radio Technical Commission for Maritime Services (RTCM) message generated in a roadside device, and the The message includes an extension information header included in the packet, the packet includes a header in which the hopping information is recorded, the header includes a first header, a second header, and a third header, The first header includes information on the remaining number of hops deducted according to transmission of the packet and lifetime information in which the remaining time of the packet is stored, and the second header includes information on the maximum number of hoppings of the packet and Includes transmission method information indicating a transmission method, wherein the third header includes identification information of the packet, first source information indicating information of a device that generated the packet, and final destination information of the packet, wherein the processing unit , When processing the message, it is determined whether the extended information header exists in the message, and when the extended information header does not exist in the message as a result of determining whether the extended information header exists, the message is no longer If the extended information header exists in the message as a result of determining whether the extended information header exists without performing processing, the extended information header is extracted from the message, and the packet is extracted from the extracted extended information header. and extracts the header from the extracted packet, extracts the transmission method information from a second header of the extracted header, determines the transmission method of the packet based on the extracted transmission method information, and determines the transmission method of the packet. As a result of determining the method, if the transmission method of the packet is the unicast method, the final destination information is extracted from the third header of the extracted header, and the final destination of the received message is determined based on the extracted final destination information. As a result of determining whether the receiving device 320 is the receiving device 320 and determining whether the final destination is the receiving device 320, if the final destination of the received message is the receiving device 320, the received message is stored in the database.

On the other hand, the retransmitter, when transmitting the message in the V2X communication standard according to the hopping information, as a result of determining whether the final destination is the receiving device 320, the final destination of the received message is the receiving device 320 If not, the remaining hopping count information is extracted from the first header of the extracted header, 1 is subtracted from the value of the extracted remaining hopping count information, and the value of the subtracted remaining hopping count information is not 0. In this case, the received message is retransmitted to another receiving device 320 .

The transmitting device 310 and the receiving device 320 according to the present invention may be integrated and operated as one device, which may be a multi-hop type V2X communication module or a V2X module. In addition, the V2X data transmission device 310 may be included in the vehicle 300, a base station including a roadside device (V2X RSU 160), or the like, respectively.

The V2X RSU 160 according to the present invention is installed on the roadside and may be a hybrid V2X RSU 160 for supporting both the WAVE method and the C-V2X method, which are V2X communication methods.

The V2X communication transmission device 310 and the reception device 320 according to the present invention have a V2X module, are integrated with a V2X terminal, are mounted in the vehicle 300, and transmit and receive data based on wireless communication using the V2X communication standard. This is possible and may be known. The V2X communication standard described in this specification may be a known one, and as an embodiment, it may be Wireless Access for Vehicular Environment (WAVE) made of IEEE 802.11p and IEEE 1609.x standards, but is not limited thereto, and for example, C -It could be V2X.

The hybrid V2X RSU 160 according to the present invention includes a C-V2X module and a WAVE module. The vehicle 300 according to the present invention is equipped with a C-V2X transmitter 310 and a C-V2X receiver 320 (hereinafter referred to as 'C-V2X terminal') for C-V2X communication or WAVE for WAVE communication. A transmitting device 310 and a WAVE receiving device 320 (hereinafter, 'WAVE terminal') are mounted.

The C-V2X module included in the hybrid V2X RSU 160 according to the present invention transmits and receives data by communicating with the C-V2X terminal installed in the vehicle 300, and the WAVE included in the hybrid V2X RSU 160 according to the present invention The module communicates with the WAVE terminal installed in the vehicle 300 to transmit and receive data. The hybrid V2X RSU 160 according to the present invention can simultaneously communicate with the C-V2X terminal and the WAVE terminal installed in the vehicle 300.

The C-V2X module according to the present invention includes an LTE-V2X module and/or a 5G-V2X module. The C-V2X module includes an LTE-V2X antenna and/or a 5G-V2X antenna for transmitting and receiving a C-V2X signal including an LTE-V2X signal and/or a 5G-V2X signal. The C-V2X module demodulates the LTE-V2X signal and / or 5G-V2X signal received by the LTE-V2X antenna and / or 5G-V2X antenna to obtain LTE-V2X communication information and / or 5G-V2X communication information, It includes a C-V2X modem that modulates the obtained LTE-V2X communication information and / or 5G-V2X communication information to generate an LTE-V2X signal and / or a 5G-V2X signal.

The WAVE module according to the present invention includes a WAVE antenna for transmitting and receiving a WAVE-V2X signal. The WAVE module includes a WAVE modem that demodulates the WAVE-V2X signal received by the WAVE antenna to obtain WAVE communication information and modulates the WAVE communication information to generate a WAVE signal.

As described above, WAVE and C-V2X have different protocol stacks and different data modulation methods. If the frequencies used by the C-V2X terminal and the WAVE terminal are different, there is no mutual effect even if they are used simultaneously. Therefore, the hybrid V2X RSU 160 according to the present invention includes a C-V2X module and a WAVE module, and as the C-V2X and WAVE communicate with the vehicle 300 using different frequencies at the same time, the C-V2X method and WAVE-type V2X communication are all possible.

The hybrid V2X RSU 160 according to the present invention collects the communication information obtained by the C-V2X module and the WAVE module, respectively, or the modulated signal based on the communication information and the processed data generated by the C-ITS MEC server 200 and transmits it to the center server 400. The hybrid V2X RSU 160 may transmit the collected data to the center server 400 using 5G communication. The center server 400 generates information for operation of the roadside base station, information for operation of the vehicle 300, and additional information based on the received data, and returns the generated information to the hybrid V2X RSU (160) according to the present invention. ) can be sent. The center server 400 according to the present invention may be a central control center server 400 located outside the C-ITS infrastructure device 100 .

The hybrid V2X RSU 160 according to the present invention transmits processing data to the vehicle 300. When the hybrid V2X RSU 160 transmits processed data to the vehicle 300, the information modulated by the C-V2X module is sent to the vehicle 300 equipped with the C-V2X terminal, and the information modulated by the WAVE module is WAVE. It can be controlled to transmit to the vehicle 300 equipped with the terminal.

The hybrid V2X RSU 160 according to the present invention transmits the collected processed data to the center server 400 through a 5G network, and further includes a 5G module that receives information generated by the center server 400 through 5G communication. include The 5G module includes a 5G antenna for transmitting and receiving 5G signals and a 5G modem for processing the 5G signals received through the 5G antenna. The 5G module obtains information for operation of the roadside base station, information for operation of the vehicle 300, and additional information from the center server 400 through 5G communication, and the above-mentioned C-V2X communication information, WAVE communication information, DSRC Communication information is transmitted to the center server 400 through 5G communication. The 5G module supports 5G communication. The 5G module connects the network and terminals for 5G wireless communication service.

The hybrid V2X RSU 160 may transmit roadside base station information to the vehicle 300 and the center server 400 through 5G communication. The information about the roadside base station may include information about the vicinity of the roadside base station collected by the corresponding roadside base station and/or information about the vehicle 300 within the area of the corresponding roadside base station. When the 5G module transmits C-V2X communication information and WAVE communication information to the center server 400 through 5G communication, information around the base station may also be transmitted to the center server 400.

The hybrid V2X RSU 160 according to the present invention, as an embodiment, performs basic vehicle 300 communication (V2V, Vehicle to Vehicle) using a C-V2X module, and uses a WAVE module to perform vehicle 300 communication. ) and infrastructure communication (V2I, Vehicle to Infrastructure).

The hybrid V2X RSU 160 according to the present invention further includes a GNSS module that processes GNSS signals to obtain location information of roadside base stations, and the GNSS module includes a GNSS antenna that transmits and receives GNSS signals. The GNSS module processes the GNSS signal received by the GNSS antenna to obtain location information of the hybrid V2X RSU (160).

On the other hand, the pedestrian detection unit 242 according to the present invention, as an embodiment, determines that an unexpected situation has occurred when an abnormal behavior of a pedestrian located in a safe area is detected. The pedestrian detecting unit 242 detects a pedestrian by processing data on the road captured by the video monitoring device 110, that is, image data. At this time, the video surveillance device 110 may be installed to photograph a crosswalk, a sidewalk near a crosswalk, a road near a sidewalk, or a zone where pedestrians wait to cross the road using a crosswalk (hereinafter referred to as a 'safe zone'). . The image monitoring device 110, which captures an image processed by the pedestrian detection unit 242, captures the entire crosswalk and pedestrians in a safe area located at both ends of the crosswalk as an object, and captures the image of the pedestrian Create video data.

The pedestrian detecting unit 242 recognizes the presence of a pedestrian between the crosswalk and the safety zone by analyzing and processing the image data of the zone including the safety zone and the crosswalk captured by the video monitoring device 110 . The pedestrian detection unit 242 analyzes the image data generated by the video monitoring device 110 . The pedestrian detecting unit 242 may draw a plurality of virtual lines in a safety area adjacent to a crosswalk. As an example, the virtual line may be drawn in a direction perpendicular to the direction of a crosswalk formed to connect two sidewalks. That is, the virtual line may be drawn in a direction parallel to the road.

The pedestrian detecting unit 242 may draw a plurality of virtual lines. For example, when there are two virtual lines, one may be drawn closer to the crosswalk and the other may be drawn farther from the crosswalk. If the virtual line drawn farther from the crosswalk is referred to as a first virtual line, and the virtual line drawn closer to the crosswalk is referred to as a second virtual line, the pedestrian detection unit 242 determines the location of a pedestrian waiting to cross in a safe zone. It is determined whether is waiting away from the first virtual line, waiting to cross the first virtual line and contact the second virtual line, or passing both the first virtual line and the second virtual line and waiting.

The pedestrian detection unit 242 according to the present invention may receive a signal of a pedestrian traffic light from the signal controller 140 and the communication unit 230, and may operate when the pedestrian traffic light is currently red. The pedestrian detecting unit 242 may detect when the pedestrian traffic light is currently red, when the pedestrian passes the first virtual line and waits to come into contact with the second virtual line, and when the pedestrian passes both the first virtual line and the second virtual line and waits. It is possible to detect the pedestrian's dangerous situation as an unexpected situation caused by a pedestrian on the road. When the pedestrian detection unit 242 detects a dangerous situation for a pedestrian, the driver warning unit 245 generates alarm information to be transmitted to the vehicle 300, and the generated alarm information is transmitted to the V2X RSU 160. , The V2X RSU 160 transmits alarm information to the vehicle 300. On the other hand, it is preferable that the alarm information be different when the pedestrian's position passes the first virtual line and waits to contact the second virtual line and when the pedestrian passes both the first virtual line and the second virtual line and waits. . That is, even if the detection of the danger of the pedestrian is the same, the case where the pedestrian's position passes both the first virtual line and the second virtual line and waits is higher than the case where the pedestrian crosses the first virtual line and waits to touch the second virtual line. may be considered more dangerous.

The driver warning unit 245 according to the present invention, when it is determined that the current position of the pedestrian is more dangerous, that is, when the pedestrian passes both the first virtual line and the second virtual line and is waiting, an alarm with a stronger alarm information can be generated. A stronger alarm may include, for example, a change in sound that is displayed through the HMI of the vehicle 300 or an alarm is displayed even if the driver does not press the emergency button.

The pedestrian detecting unit 242 according to the present invention may classify danger according to the position of a pedestrian waiting in the safety zone and detect abnormal behavior of the pedestrian in the safety zone. As described above, the pedestrian detecting unit 242 may operate when the traffic light for pedestrians is a red light, and may determine and recognize the presence of a pedestrian in an area including a safety area and a crosswalk.

Meanwhile, the C-ITS infrastructure device 100 according to the present invention may further include a speaker unit (not shown) capable of giving an alarm to a pedestrian when an unexpected situation occurs. When the pedestrian traffic light is red and there is a pedestrian at the crosswalk, the speaker unit generates an alarm to give an auditory attention effect to the pedestrian, and at the same time, the V2X RSU (160) directs the vehicle 300 normally running on the road. Alert information can be transmitted.

The video monitoring device 110 according to the present invention includes a motion detector camera having a function of detecting a moving object in a safety zone and a crosswalk, an infrared camera having a heat detection function, a camera equipped with a radar 120, and an automatic tracking function. At least one of network cameras capable of performing IP communication may be further included.

The pedestrian detecting unit 242 according to the present invention may further include an object detecting unit (not shown) and an abnormal behavior detecting unit (not shown). The object detecting unit may detect a change in the position of a pedestrian in an area including a safety area and a crosswalk. As an embodiment, the object detecting unit detects the pedestrian by analyzing the movement trajectory of the pedestrian. In addition, the object detecting unit patterns the image data to determine whether a pedestrian is present or not when a pedestrian is detected in the safe zone based on the safe zone. Whether a pedestrian exists or not is performed based on shape information and feature information. Pedestrian location tracking is performed by performing a block-based matching algorithm based on shape control points extracted according to the occurrence of object occlusion, or tracking a pedestrian by a center-of-gravity-based block matching algorithm or a periodic update-based block matching algorithm. can do.

Specifically, the pedestrian detecting unit 242 generates a background image based on continuously input image frames, and updates the background image according to whether or not the pedestrian detected from the image frames is moving. The step of detecting a rectangle containing all the blocks in which differences exist between input image frames as a motion area, and the step of extracting a shape control point (SCP) from the detected motion area, and if object occlusion does not occur, the extracted Based on shape control points, block matching algorithm (BMA) is performed to track pedestrians, and when object occlusion occurs, center-of-gravity-based block matching algorithm (CBMA) and tracking the pedestrian by using any one algorithm from among PBMA and Periodic-update-based Block Matching Algorithm (PBMA).

Meanwhile, in the step of updating the background image described above, if the value obtained using Equation 1 below is less than a preset threshold value, the background image is updated using Equation 2 below.

<Equation 1>

Here, (u, v) represent horizontal and vertical coordinates, (mx, my) represent motion vectors, and (p, q) represent the horizontal and vertical sizes of image blocks.

<Equation 2>

Here, B(t) represents the background at time t, I(t) represents the input image at time t, and σ represents the experimentally set mixing ratio in the range of 0 to 1. Accordingly, since tracking of the pedestrian can be continued only with the verified shape control points, it is possible to be free from potential failure factors such as temporal spatial similarity between the pedestrian and the background, object deformation, and occlusion.

The abnormal behavior detection unit according to the present invention detects irregular loitering and analyzes patterns when a specific pedestrian moves in an unspecified movement trajectory continuously for a predetermined period of time within a safety zone. In the case of loitering and abnormal behavior detection, based on the change in the position of the pedestrian detected by the difference between the video frame and the background image, abnormal situations such as loitering behavior that may occur within the safety zone are detected, thereby determining whether a specific abnormal situation has occurred. effective unmanned surveillance.

The abnormal behavior detection by the abnormal behavior detection unit according to the present invention, as an embodiment, is detected between a current video frame and a previous video frame temporally preceding the current video frame among a plurality of video frames input from the video monitoring device 110. A background image generation step of generating a background image corresponding to the current image frame based on the optical flow, a moving object detection step of detecting the movement of a pedestrian from the current image frame based on the difference between the background image and the current image frame, and the current image An abnormal situation that occurs as the pedestrian moves within the safety zone corresponding to the current video frame is detected based on the movement trajectory, which is the change in the movement position of the pedestrian, which is identically detected from video frames successive in time to the frame. The frame is divided into a plurality of blocks of a preset size, and if the number of blocks that the pedestrian's movement trajectory passes is equal to or greater than the preset reference number, the pedestrian is considered to be a pedestrian acting abnormally within the safety zone. It may include an abnormal behavior detection step of determining that it is.

In addition, a stationary pedestrian can be detected based on the difference between the background image and the current image frame and the difference between the past background image and the current image frame, which is a background image generated in response to a video frame preceding the current video frame by a preset reference time, , If the position of a stationary pedestrian is the same on the previously set reference number of video frames that are temporally continuous up to the current video frame, it can be estimated as a stationary pedestrian. Such a stationary pedestrian is estimated to correspond to a person who has fainted based on the invariant moment value. In addition, it is possible to estimate the behavior of a person who hurriedly jumps toward the crosswalk in the safety zone.

In this specification, each component of the processing unit 240 that processes image data on the road may be by the GPU unit 210 including a main processing device and a video card.

On the other hand, the processing unit 240 according to the present invention proposes a design of a real-time CCTV video security system using a frame division thread when processing images collected from the video monitoring device 110 (CCTV). The proposed method uses a frame blocking method that divides a frame into blocks and compares them in order to reduce the size of image information. At this time, it is implemented using 16 threads for efficient data processing. Confidentiality of image information is guaranteed because all processed data is encrypted and stored by ARIA encryption algorithm. Through the experimental results, it was confirmed that the image processing speed was improved by about 40% compared to the existing method using the original image, and the size of the image was reduced by about 60%.

In the early days of CCTV, it was mainly used for security and crime prevention purposes, but now it is used for traffic information collection, crime prevention, accident, evidence, security, surveillance, observation, etc. CCTV is widely used. Previously, image storage in CCTVs mainly used hard disks with limited capacity, but recently, they are evolving into cloud computing system-based intelligent CCTVs that transcend existing network-based CCTVs. Until now, CCTV, which can be called representative security equipment, is being applied to various industries while meeting various information and communication technologies such as artificial intelligence, autonomous driving, Internet of Things, and big data, which are important fields of the 4th industrial revolution. In other words, CCTV can be applied not only to various smart city services such as disaster and safety fields, environment and energy, and healthcare, but also to various fields such as smart homes and smart factories. Current CCTVs have been able to store a large amount of information by converting from the analog method of the past to the digital method. As such a large amount of information is stored, the cost of storage space has increased significantly, and countermeasures for this have been required. In addition, since it is a digital method, access to information is easy, and security measures for this are also required. CCTVs, which are generally used, take images as they are, convert them into digital images, and then store them. In other words, it is a method of storing an image as it is without processing it in an appropriate way. Compared to existing cameras, IP cameras have better image quality and provide various additional functions, and intelligent IP cameras that can mask personal information analyzed from meta data are also being used. However, since the existing methods can forge or alter images, many security-related problems such as invasion of privacy may occur. The present invention proposes the design and implementation of a real-time CCTV video security system using a frame division thread. In the proposed method, in order to reduce the size of image information, a frame blocking method is used in which frames are divided into blocks and compared, and implemented using 16 threads. Confidentiality of image information is guaranteed because all processed data is encrypted and stored by ARIA encryption algorithm.

Proposed CCTV video security system

The video processing process of the system proposed in the present invention receives video from CCTV and proceeds with video processing and encryption. The processed data is transmitted to the Viewer, and the Viewer performs decoding and image restoration to reproduce the original image. The image processing method in the present invention is a method using frame correlation, and compares a frame to be compared with a currently reproduced frame to extract and store only meaningful pixel values. This is a method using the fact that CCTVs that continuously photograph the same place at a fixed location have overlapping pixel values than other images. By using this method, since only meaningful frames are extracted excluding overlapping frames and combined with existing overlapping frames, frame capacity can be greatly reduced. Assuming that a video security system shoots with a camera and creates a standard frame every 60 seconds, the operation from shooting to transmission to the Viewer server is shown in Figure 1.

<Figure 1. Flowchart of proposed image processing>

As shown in Figure 1, the proposed image processing process receives frames starting with camera shooting, finds the difference between the frame to be compared and the current frame, and then blocks the value. At this time, the difference between frames and blocking are designed based on 16 divided threads in consideration of efficiency. After that, the blocked frames are sorted and the first coordinates (reference frame) of each block are saved in a file. Afterwards, the ARIA encryption algorithm is used to encrypt each standard frame data and send it to the Viewer, and the processed block alignment data is also sent to the Viewer.

16 split thread design

Moving picture is a technology that makes each still picture, that is, frame, pass quickly so that it appears as if it is moving. The correlation between frames refers to the difference between two frames where the background is the same but there is a moving object when there is a slight difference between successive frames. Conventionally, in order to obtain a pixel difference using frame correlation, a method of comparing entire two frames in units of pixels has been used. However, since this method takes a lot of time for comparison, the present invention proposes a method of processing an image by dividing it into 16 parts for more efficient frame comparison. A meaningful content is extracted by comparing the frame to be compared (reference frame) with the currently played frame. That is, in comparison with the two frames, a fixed (standard) frame without a moving object is not extracted, and only a pixel difference between the two frames is obtained when a moving object exists. At this time, by comparing pixel differences, redundant data is removed, and only meaningful data is composed of variable blocks. In other words, after comparing the pixel differences, the same parts are filled with white, and only the moving objects (different parts from the reference frame) are filled in the image. The reason for dividing the frame into blocks is to objectify meaningful pixel values and process them quickly and accurately. At this time, blocks without values are excluded from calculation because they are meaningless. Since both calculating the difference value between pixels (frames) and blocking frames can be performed after designating an area, the entire area can be divided into 16 parts and threads can be used at the same time. Therefore, the thread of each divided area performs the work of the next frame when the difference value between frames is calculated and frame blocking is performed at the same time, and the work for all areas is completed. By designing 16 divided threads in this way, the image processing speed can be increased to prevent speed interference between the current frame and the next frame. Figure 2 shows an example of the configuration of a frame 16 division thread. A thread is the smallest program unit executed in a CPU and is composed of information such as a thread ID, a set of registers, a stack, and a program counter. Multi-threading enables memory sharing even within the same process, enabling efficient data processing and reducing system resource waste. In the present invention, when many threads are executed, the image is processed by dividing into 16 in consideration of data processing efficiency. In Figure 2, Locked is used for synchronization, and 16 threads belonging to one process share resources such as memory to enable parallel processing, so the processing speed becomes very fast.

<Figure 2. Example of configuration of 16 frame division threads>

Specify frame of reference

Designation of a reference frame (image) creates a reference image at each designated time by using the characteristics of CCTV images photographing a designated place. When a reference image is specified, subsequent frames compare pixels with this reference image. Assuming that the background brightness of the reference image and the subsequent frame are the same and there is a moving object, there is no pixel difference between the previous and overlapping parts, so this part is filled with white, and only the moving object is extracted and filled in the image. In order to save the size of image files for comparison, reference images generated at regular intervals are used. Figure 3 (a) shows a method for generating a reference image, and (b) shows an efficient algorithm for pixel comparison.

<Figure 3(a) Base image creation>

<Figure 3(b) Pixel Comparison>

As can be seen in Figure 3(a), the reference image is created every 60 seconds, and this file is encrypted and stored.

Frame blocking and block alignment

When overlapping data is removed by comparing pixel differences, only meaningful data is composed of variable blocks. At this time, the parts with the same pixels are filled with white, and only the parts different from the standard frame (image) are filled and block with a certain size. If you want to extract motion or data more precise than this level, you can extract data by lowering the reference value of pixel comparison or reducing the size of the block. After blockization, each block is filled in order from the upper left corner. At this time, coordinate values are put in front of each block so that the original position of each block can be found. Blocking and block alignment are performed by searching the entire image for each block size. Only if the searched value is not white, the value is saved and blocked. It stores the first coordinates of the block and aligns the blocks. Figure 4 shows the blocking and block sorting processes executed simultaneously in 16 divisions.

<Figure 4. Process of image blocking and block alignment>

When an image is processed through this process, the size of an image file can be reduced up to 1/30 of the original original image file size by reducing unnecessary data. When sorting, the first coordinates of each block are saved in a file and read together when reading through the Viewer to be used when rearranging the blocks.

Image Viewer design

To reproduce an image as a video, you need to know the processing speed (FPS, Frame Per Second) when shooting each image so that you can see the same video as the actual shot. If you don't know your FPS, the video may be too fast or too slow and there may be parts missing during video playback. Therefore, it is necessary to save the FPS in the image file while shooting CCTV. In the Viewer server, processed image files are stored after taking CCTV images. At this time, since the images are saved after blockization and block alignment, the viewer must find the coordinates (coordinates for each standard frame) of each block (piece) that has already been transmitted. Then, the original image is reproduced by connecting the blocks arranged in the reference image file according to the block coordinates.

Encryption of frame of reference

The proposed CCTV security system performs encryption and decryption based on the ARIA encryption algorithm. ARIA, a symmetric key algorithm, has a block length of 128 bits and can use variable key lengths of 128/192/256 bits. In addition, encryption is performed by repeating 10/12/14 rounds, respectively, according to the size of the key. The implemented encryption engine consists of a round module that performs encryption and decryption and a scheduler that generates key values. In the present invention, an encryption engine is designed and implemented using a 128-bit key. The round function module consists of three main parts: key addition (AddRoundKey function), substitution layer (SubstLayer function), and diffusion layer (DiffLayer function). The key addition of the round function performs an XOR operation for each bit of the 128-bit round key and the 128-bit input value. The substitution layer is composed of two types, and each layer has an 8-bit input and output S-box and an inverse transform structure for them. A spreading layer that performs a spreading function has a 16×16 involution structure and performs matrix operations.

implementation result

In the present invention, the system was implemented by converting the existing ARIA C (C++) source code into C# code. The definition part, encryption, and decryption parts were reconstructed using each class, and a fixed key value was set and used. Figure 5 shows the process of converting the C source code of the existing ARIA algorithm into C# code. Figure 6 shows an encrypted example of aligned frames and reference frames. Encryption is performed only on the final frame (sorted reference frame) that comes out after block sorting. As shown in Figure 6, when the aligned frames are encrypted, the encrypted frames (reference frames) are stored at the same address. At this time, the file to be encrypted is a standard frame file (bmp file), and due to the nature of this file, if the header part is damaged, the entire file becomes unreadable. Therefore, in the present invention, a method of increasing the encryption speed by partially encrypting the bmp file, which is the header part of the frame, was used. Figure 7 shows the results of encryption (ciphertext) and decryption (Decrypted) for each of the three plaintext addresses of the frame aligned in Figure 6. As explained above, the viewer in the imaging system reads the fixed frame file, the aligned file, and the coordinate file. Then, it reads the coordinates of the block from the coordinate file and applies these coordinates to block alignment to move the block to its original position.

<Figure 5. Process of converting existing ARIA C code to C#>

<Figure 6. Examples of aligned frames and encrypted frames>

<Figure 7. Encryption and decryption results for aligned frames>

performance evaluation

Table 1 shows the experimental results of the existing original video and the proposed method while recording the video for 5 minutes.

<Table 1. Image processing results and performance comparison>

In the case of the original video, the video is recorded as it is without processing the video. In fact, it is the most commonly used method in Korea and requires a lot of storage space, and anyone can access it, so there is a problem with data security. Proposed method 1 shows the result of performing the split-thread-based method proposed here, and proposed method 2 performs the split-thread-based and encryption method in parallel. As shown in Table 1, the proposed method 1 improved the FPS by about 40% compared to the original video. In addition, the capacity has also been reduced by about 75%, but anyone can access it and is exposed to security. In the case of proposed method 2, only the header part of the BMP file is partially encrypted and transmitted, so there is no difference from method 1 in terms of speed. However, the average capacity increased to some extent through encryption, but security is excellent because unauthorized access and confidentiality are guaranteed. Figure 8 shows a part of the screen executed in Viewer. In Figure 8, the white part represents the part where there is no change (difference) from the reference frame. Among the four files, the image files stored as files on the server are block alignment (a) and reference frame (c). The reference frame and block image (d) is an image synthesized with the result of block rearrangement in the reference frame, and the actual captured image is the same as (d). Encrypted images are decrypted only when playing them in the Viewer.

<Figure 8. Viewer execution screen>

The implemented CCTV video security system was implemented using C# of Visual Studio 15, and the camera used USB WebCam ALC-M600.

Since the existing CCTV converts the original video into a digital video without processing the video, security is weak in forgery and alteration of the video. In the present invention, the implementation of a frame division thread-based CCTV security system using the ARIA algorithm has been described. The implemented system extracts only meaningful frames excluding overlapping frames and synthesizes them into existing overlapping frames to process images, thereby significantly reducing frame capacity. The processing speed of the implemented security system was improved by about 40% compared to the existing image processing method, and the capacity was reduced by about 60%. In addition, the implemented system is effective in preventing invasion of personal information or privacy by a third party because image data is encrypted and stored.

In the present specification, the C-ITS MEC server 200, the GPU unit 210, the C-ITS infrastructure interface unit 220, the communication unit 230, the processing unit 240, the unexpected situation detection unit 241, the pedestrian detection unit ( 242), traffic information collection unit 243, driver warning unit 245, storage unit 250, power distribution board 260, and/or center server 400 may be processors that execute sequential execution processes stored in memory. there is. Or, it can operate as software modules driven and controlled by a processor. Further, a processor may be a hardware device.

In this specification, a "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware.

The protection scope of the present invention is not limited to the description and expression of the embodiments explicitly described above. In addition, it is added once again that the scope of protection of the present invention cannot be limited due to obvious changes or substitutions in the technical field to which the present invention belongs.

100: C-ITS infrastructure device 110: video surveillance device
120: radar 130: lidar
140: signal controller 150: weather sensor
160: V2X RSU 200: C-ITS MEC Server
210: GPU unit 220: C-ITS infrastructure interface unit
230: communication unit 240: processing unit
241: unexpected situation detection unit 242: pedestrian detection unit
243: traffic information collection unit 245: driver warning unit
250: storage unit 260: power distribution board
300: vehicle 310: V2X data transmission device
320: V2X data receiving device 400: center server

Claims (5)

C-ITS infrastructure devices that collect on-road data;
The C-ITS MEC is connected to the C-ITS infrastructure device, receives and processes the data on the road, generates processed data including safe driving information, and transmits the generated processed data to the C-ITS infrastructure device. server; and
a vehicle capable of wireless communication with the C-ITS infrastructure device and receiving the processed data from the C-ITS infrastructure device;
including,
The C-ITS infrastructure device is installed on the roadside,
Including a video surveillance device, a radar, a lidar, a signal controller, a meteorological sensor, and a V2X RSU (RoadSide Unit),
The C-ITS MEC server is installed on the roadside,
Including a GPU (Graphics Processing Unit) unit, a C-ITS infrastructure interface unit, a communication unit, a processing unit, a storage unit, and a power distribution board,
The safe driving information includes unexpected situation information and traffic information on the road,
The processing part,
an unexpected situation detecting unit for detecting an unexpected situation on the road based on the processing data; and
And a driver warning unit for generating alarm information to be transmitted to the vehicle when an unexpected situation is detected by the unexpected situation detection unit,
The emergency situation detection unit,
A pedestrian detecting unit for detecting a pedestrian on the road; and
Including a traffic information collection unit for collecting traffic information on the road,
The vehicle is equipped with a V2X data transmission device and a V2X data reception device,
The V2X data transmission device,
A generating unit that generates a V2X message including the processing data and generates a packet containing hopping information and includes it in the V2X message; and
A transmitter for transmitting the V2X message;
including,
The V2X data receiving device,
Receiving unit for receiving the V2X message;
A processing unit for processing the V2X message; and
Including a retransmission unit for retransmitting the V2X message according to the hopping information,
The packet includes an extension information header containing the hopping information,
The extension information header includes a first header, a second header, and a third header,
The first header includes information on the number of remaining hopping counts deducted according to transmission of the packet and lifetime information in which the remaining time of the packet is stored;
The second header includes information on the maximum hopping count of the packet and transmission method information indicating a transmission method of the packet;
The third header includes identification information of the packet, first source information indicating information of a device that generated the packet, and final destination information of the packet,
When the processing unit processes the V2X message,
Determining whether the extension information header exists in the V2X message,
When the extension information header exists in the V2X message as a result of determining whether or not the extension information header exists, the extension information header is extracted from the V2X message, and the first header to the third header are extracted from the extension information header. A header is extracted, the remaining time of the packet is checked from the extracted first header, and the transmission method information is extracted from the extracted second header, and the transmission method of the packet is based on the extracted transmission method information. and, as a result of the determination of the transmission method, when the transmission method of the packet is a unicast method, the final destination information is extracted from the extracted third header, and the received V2X is based on the extracted final destination information. It is determined whether the final destination of the message is the V2X data receiving device, and as a result of determining whether the final destination is the V2X data receiving device, if the final destination of the received V2X message is the V2X data receiving device, the received V2X message is stored in the database. stored in,
When the extended information header does not exist in the V2X message as a result of determining whether or not the extended information header exists, processing of the V2X message is no longer performed, for autonomous driving based on C-ITS MEC. Real-time traffic information provision system. delete delete The method of claim 1,
The communication department,
A system for providing real-time traffic information for autonomous driving based on C-ITS MEC, characterized in that for transmitting the generated processed data to a center server through a 5G network. delete

Images