KR102378499B1 - Solution for scaling/reducing employing 5G image communication - Google Patents

Abstract

An embodiment relates to an image enlargement/reduction solution using 5G image communication. Particularly, this solution is used for a 5G-based IoT device for mounting on an accident management vehicle. In addition, an embodiment relates to a 5G device-based two-way response image system technique between a control center and a command vehicle so that the control (command) center can grasp and respond to an on-site situation in real time when a disaster occurs in a bridge or a large bridge. In this case, an SR GAN-based deep learning-based image enlargement technique for centralized control is provided. Therefore, an embodiment secures a support technique capable of preventing secondary collisions and minimizing damage through recognition of traffic accidents, rapid determination according to accident types, and responses thereto. In the event of a collision, the control center can the collision and can support images to so that the collision is identified and the control center can cooperate with related organizations (fire department, police station, etc.). In addition, situations can be accurately transmitted and collisions can be handled by using image sharing, based on 5G new technology, in an accident handling communication system such as a radio and a wired/wireless telephone between the existing accident management vehicle and the control center. In addition, IP speakers are installed in each road facility section to support selective evacuation/announcement broadcast such as distance-specific bypass guidance, front-accident guidance, and warning broadcasting guidance around accident points, thereby providing convenience and smooth measures.

Description

Image enlargement/reduction solution using 5G image communication {Solution for scaling/reducing employing 5G image communication}

The content disclosed in the present specification relates to an image processing method for an IoT device for a vehicle, and more particularly, when a vehicle is driven, an image of an accident scene is collected around the vehicle, and the image is converted to match the wall system of the control center and monitored to make it smoother.

Unless otherwise indicated herein, the material described in this section is not prior art to the claims of this application, and inclusion in this section is not an admission that it is prior art.

In general, in the case of disasters in urban infrastructure and multi-use facilities, which are increasing recently, it is difficult to rescue survivors due to risk detection, delay in propagation, and evacuation/response, leading to large-scale damage.

Therefore, we develop technologies to reduce disasters and safety accidents for efficiency in disaster prevention/preparation/response/recovery work.

And, for this purpose, as the level of demand for safety advancement for real-time response such as prevention, detection, and response from disasters/disasters increases, various technologies such as CCTV image-based crime prevention and real-time traffic accident monitoring/response system establishment have been developed. appeared.

However, the existing control system for road facilities has limitations in intensive monitoring of the accident site because it is impossible to enlarge or reduce the image in a general matrix method. And, although the image can be checked on a large monitor, there is also the fatigue of the control due to the resolution problem when the image is enlarged with a low-quality CCTV image.

In particular, when the CCTV image, which is generally used on the road, is enlarged with a wall system, the image quality deteriorates.

The prior art literature in this background is the extent to which the following patent documents appear.

(Patent Document 1) KR101923494 Y1

(Patent Document 2) KR101033237 Y1

For reference, the technology of Patent Document 1 relates to a disaster response system for a mobile vehicle using a directional speaker, and may be to the extent that a directional speaker is installed in a mobile vehicle to broadcast a disaster to enable evacuation.

And, Patent Document 2 is about an enforcement system that provides various functions through an ultra-wide image.

The disclosed content applies the concept of transmitting an external image to a control system and a wall system to enlarge/reduce an external analog/digital input image to 1:1, 1:N, N:N to one on multiple monitors. In addition, to solve the problem of resolution degradation caused by image enlargement, a deep learning image magnification algorithm of the super-resolution concept is applied, and the image is customized and converted and transmitted from the management vehicle to the control system to solve the accident. We want to provide an image enlargement/reduction solution using 5G video communication that can be performed effectively.

The video enlargement/reduction solution using 5G video communication according to the embodiment,

It is used in 5G-based IoT device devices for accident management vehicles. In the event of a disaster or disaster on a bridge or bridge, the control (command) center can understand and respond to the situation in real time. It is about the video system technology for interactive response between the control center of the control center/commanding vehicle. And, in this case, it is characterized by including an ESRGAN-based deep learning-based image magnification technology for centralized control.

According to the embodiments, it provides support technology that can prevent a secondary collision accident and minimize damage through recognition of a traffic accident and rapid judgment and response according to the accident type.

And, in the event of a collision, video support is possible so that the control center can check it and cooperate with related organizations (fire department, police department, etc.).

In addition, it is possible to accurately communicate the situation and cope with collision accidents through video sharing based on 5G new technology through the accident management communication system such as radio and wired/wireless phone between the existing accident management vehicle and the control center.

In addition, it is installed for each road facility section through an IP speaker and provides convenience and smooth measures through selective evacuation/information support such as detour road guidance for each street centering on the accident point, forward accident guidance, and warning broadcast guidance.

1 is a diagram for conceptually explaining an image enlargement/reduction method using 5G video communication according to an embodiment;
2 is a view showing the overall system to which the image enlargement/reduction method using 5G video communication according to an embodiment is applied;
3 is a block diagram showing the configuration of a 5G-based IoT device device for mounting an accident management vehicle to which an image enlargement/reduction method using 5G video communication is applied according to an embodiment;
4 is a flowchart sequentially illustrating an image enlargement/reduction method using 5G video communication according to an embodiment;
5 is a diagram for explaining an image enlargement format (algorithm) applied to an image enlargement/reduction method using 5G image communication according to an embodiment;

1 is a diagram for conceptually explaining an image enlargement/reduction method using 5G video communication according to an embodiment.

As shown in Figure 1, the image enlargement/reduction method of an embodiment is a 5G device so that when a disaster or disaster occurs on a bridge or a bridge, the control (command) center can understand the situation in real time and respond It is about the two-way response image system technology between the control center/commanding vehicle based on the And, in addition to this, the IoT device includes an SR GAN-based deep learning-based image enlargement technology for centralized control and an IP speaker control technology for establishing a response system for each section.

So, through this, through the development of coping technologies for collision accidents on bridges and bridges, the establishment of a rapid response system for disaster and disaster accident management, the reinforcement of the accident response expertise and efficiency of the control center and the command vehicle, and accident management/recovery improve crisis management skills.

Specifically, these IoT device devices include a CCTV camera for accident scene video collection and an accident alarm (directional) IP speaker installed outside the vehicle, and an IoT device for control, a 5G video/text transmission/reception module, and a dash for driver confirmation in the vehicle. Includes board (monitor).

And, in this case, the CCTV camera is installed on the exterior of the vehicle to collect images of the accident site.

In addition, the IoT device device for control includes a window-based IoT device control SW. In addition, it is equipped with 5G-based video/text transmission/reception and communication status monitoring functions. Then, the collected data storage DB is configured and video data can be streamed in real time. In addition, the IoT device for control has a user-defined data format and file format, and a user interface UI/UX. In addition, it includes a function to check the video collected by CCTV cameras. In addition, it has a control center video/text check and output function and a control center video/text check function. So, through this, the IP speaker-related accident information/evacuation broadcast transmission function is performed. And, in particular, when the CCTV image generally used on the road is enlarged with the Wall System, the image quality deteriorates. have magnification skills. And, this technology also includes converting the wall system image into a high-definition image and transmitting it to this IoT device device. use it as

The 5G video/text transmission/reception module is a vehicle ?? It is used to wirelessly transmit and receive real-time images and texts between control centers.

And, the dashboard is installed in the vehicle, and the user uses it for the purpose of checking the status of the CCTV camera's collected image and the control center's received image.

In addition, the (directional) IP speaker is installed on the exterior of the vehicle and used to notify the accident status to people around the accident site. In this case, the IP speaker provides ambient noise cancellation and accurate sound quality transmission through the purchase of a directional speaker.

FIG. 2 is a view showing the overall system to which the image enlargement/reduction method using 5G video communication according to an embodiment is applied.

As shown in FIG. 2 , the system of one embodiment includes a 5G-based IoT device device 100 for mounting an accident management vehicle, a control center 200 , and a manager terminal 300 .

Additionally, the system according to an exemplary embodiment is a place for performing services in connection with the outside in conjunction with the control center 200, and includes a troubleshooting information processing device 400-1 and a hospital information processing device 400-2; and a police station information processing device 400-3.

In addition, at this time, the respective devices connect to each other via a private network. In this case, each unit in the 5G-based IoT device device 100 for mounting the accident management vehicle is connected to either serial (RS232, RS485) or wireless (LoRA, RF, BT, BLE) through a network. . And, as a self-network between each device, Wi-Fi or LTE is used as a wireless communication method.

The 5G-based IoT device device 100 for mounting the accident management vehicle is based on a 5G device so that when a disaster or disaster occurs on a bridge or a bridge, the control (command) center can understand and respond to the situation in real time. A two-way response video system between the control center of the control center/commanding vehicle is provided. Specifically, when the IoT device 100 is operating a vehicle, it is installed in the vehicle and transmits the image captured by the CCTV camera to the registration control center, and receives the image and text from the control center and outputs each. do it In addition, SR GAN-based deep learning-based image magnification technology for centralized control and IP speaker control technology for establishing a response system for each section are provided. In other words, this image enlargement technology causes a decrease in image quality when the CCTV image used on the road is enlarged with a wall system, etc. To this end, the technology converts the CCTV image into an image of 4K or higher. (100) and the control center 200, that is, the image is converted in advance into a high-definition image between the wall system and transmitted. In other words, these technologies are used as processing in the image exchanged between the 5G-based IoT device device mounted on the accident management vehicle and the wall system.

The control center 200 (or the manager terminal 300) transmits the accident scene image in a wireless communication method (eg, a web-based service) through the 5G-based IoT device device 100 for mounting the accident management vehicle. If received, take action against the accident situation by providing monitoring images and guidance text from the wall system. For example, when the control center 200 receives such an accident scene image, it determines whether an accident has occurred and the degree from the corresponding image, and when confirming the occurrence of an accident, a 5G-based IoT device device mounted on an accident management vehicle ( 100) is notified. So, through this, the 5G-based IoT device device 100 for mounting an accident management vehicle can perform an evacuation broadcast through an IP speaker.

3 is a block diagram illustrating the configuration of a 5G-based IoT device for mounting an accident management vehicle to which an image enlargement/reduction method using 5G video communication is applied according to an embodiment.

As shown in FIG. 3, the 5G-based IoT device apparatus 100 for mounting an accident management vehicle according to an embodiment includes a CCTV camera 101, an IoT device 102 for control, a 5G video/text transmission/reception module 103, and a monitor. (Dashboard for driver confirmation) 104 and an IP speaker 105 .

Additionally, the 5G-based IoT device apparatus 100 for mounting an accident management vehicle according to an embodiment performs the voice output operation of the key signal input unit 106 and the IP speaker 105 for receiving various setting information according to the user's key operation. and a sound control unit 107 which is driven differently according to the wishes of the administrator.

When the CCTV camera 101 operates a vehicle, it is installed outside the vehicle and captures and collects surrounding images.

The IoT device for control 102 is installed in the vehicle, transmits the image captured by the CCTV camera 101 to the registration control center, and receives the image and text from the control center 200 to output each. In this case, according to an embodiment, the IoT device for control 100 transmits a) an accident scene image from the CCTV camera 101 to the control center 200 when an accident occurs around the vehicle, A wall system image is received from the control center 200 , and the accident scene image and the wall system image are output to the monitor 104 . And, b) when a guide text is received when receiving a wall system image from the control center 200 , the guide text is combined and output to the monitor 104 . c) In addition, when an accident is confirmed through the accident scene image, the wall system image, or the guide text, the operation of outputting the voice from the IP speaker 105 is controlled to output an evacuation broadcast. d) Also, when delivering the accident scene image to the control center 200, a single image is created from the accident scene image in a set single image format, and after converting to a set single Mat format image, set By changing to SI (super image) by SR GAN-based deep learning-based image enlargement format and upscaling to a single image or entire video format corresponding to a number of different CCTV image file formats, and converting it to a high-definition image, The accident scene image is transmitted to the control center 200 . Alternatively, even when such a control IoT device 102 is provided with a wall system image, by receiving the same high-definition image from this method, all IoT device devices of various methods can adapt these high-definition images to each device again. Convert it so you can use it.

The 5G video/text transmission/reception module 103 is installed in the vehicle to receive images and texts between the IoT device for control 102 and the control center 200 . In this case, the 5G video/text transmission/reception module 103 is connected to the control center 200 to transmit and receive information, for example, an image of an accident site and an image/guide text of the wall system. For example, you may use Wi-Fi or LTE.

The monitor (dashboard for driver confirmation) 104 displays a combination of the image of the CCTV camera 101 and the image of the control center 200 by the IoT device 102 for control.

The IP speaker 105 outputs a voice by the IoT device 102 for control or the control center 200 .

4 is a flowchart sequentially illustrating an image enlargement/reduction method using 5G video communication according to an embodiment (refer to FIG. 3 ).

As shown in FIG. 4 , in the image enlargement/reduction method according to an embodiment, first, when a vehicle is driven, a surrounding image is captured by a CCTV camera installed outside the vehicle and collected.

And, in this case, the IoT device for control is installed in the vehicle, transmits the image captured by the CCTV camera to the registration control center, and receives the image and text from the control center to output each.

In addition, in this case, the 5G video/text transmission/reception module is installed in the vehicle to receive images and texts between the IoT device for control and the control center.

And, on the monitor (dashboard for driver confirmation), the video of the CCTV camera and the video of the control center are combined and displayed by the IoT device for control.

In addition, in this case, the IP speaker outputs audio by the IoT device for control or the control center.

In this state, the IoT device for control performs the following operation according to an embodiment.

a) First, when an accident occurs around the vehicle (S401), the IoT device for control transmits the accident scene image from the CCTV camera to the control center (S402), and receives the wall system image from the control center (S403), the accident scene image and the wall system image are output to the monitor.

b) In addition, when a guide text is received when receiving a wall system image from the control center (S404), the guide text is combined and output to the monitor (S405).

c) Also, when an accident is confirmed through the accident scene image, the wall system image, or the guide text, the operation of outputting the voice from the IP speaker is controlled to output an evacuation broadcast (S406).

d) In particular, according to an embodiment, the processing method in the image exchanged between the 5G-based IoT device device mounted on the accident management vehicle and the wall system is as follows.

For example, when delivering an accident scene image to the control center, a single image is created from the accident scene image in a set single image format, converted into a set single Mat format image, and then set SR GAN-based deep By changing to SI (super image) by the running-based image enlargement format, upscaling to a single image or an entire video format corresponding to a number of different CCTV image file formats, and converting it to a high-definition image, the accident scene at the control center The video is transmitted (refer to FIG. 5 for more detailed SR GAN contents).

Alternatively, when a wall system image is provided to such an IoT device, the same effect is provided by converting the wall system image into a high-quality image from this method.

As described above, in one embodiment, when a disaster or disaster occurs on a bridge or a bridge, the control (commanding) center can understand and respond to the situation in real time in real time between the control center/commanding vehicle based on a 5G device. It is a corresponding video system technology. And, in addition to this, the IoT device includes an SR GAN-based deep learning-based image enlargement technology for centralized control and an IP speaker control technology for establishing a response system for each section.

Therefore, through this, through the development of coping technologies for collision accidents on bridges and bridges, the establishment of a rapid response system for disaster and disaster accident management, the reinforcement of the accident response expertise and efficiency of the control (command) center and command vehicle, and accident management/recovery improve crisis management skills.

5 is a diagram for explaining an image enlargement/reduction format (algorithm) applied to an image enlargement/reduction method using 5G video communication according to an embodiment.

As shown in Figure 5, the image enlargement/reduction format of one embodiment generally causes a degradation in image quality when the CCTV image used on the road is enlarged by the wall system, and the CCTV image is converted into a 4K or higher image, etc. SR GAN-based deep learning-based image magnification technology.

In addition, this image enlargement content, for example, on the CCTV side, has a technology for converting a CCTV image of one channel into an image of a maximum size of 32 channels (the wall system is also used in the same or similar manner).

And, also, for example, an accident scene image, that is, an input image is converted into a matte (MAT) format image, and a deep learning-based image magnification algorithm is applied (through this, the value can be changed by accessing the pixel unit).

The main content of this algorithm is to change a single image to SI (super image) and up-scaling to a single image or an entire video format. . Therefore, a low-quality image is converted into a high-definition image for image magnification of 4K or higher, and in this case, the conversion operation applies an algorithm of Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation (TecoGAN) among SR algorithms. In addition, this SRGAN algorithm is a technology that solves the problems of the existing SR (Super-Resolution) technique, and using GAN, it is possible to restore the same as a real photo (images of 4K or higher can be secured). Also, in the existing SRResNet or bicubic interpolation method, the image is blurry because it learns the average (average) of the surrounding pixels. A clearer output value can be obtained.

On the other hand, in addition to this, the image enlargement/reduction method using this 5G video communication provides a service quickly and conveniently by keeping the database consistent with each other when transmitting and receiving various types of information.

To this end, the IoT device for control performs the following operations.

a) First, the control center and the table storing the device registration information and data are equally provided with each other, and a matching relationship for the table is set and registered in advance.

b) Therefore, when information in a table is mutually changed, the tables are synchronized according to the matching relationship.

c) and, when synchronizing the table, by diversifying the information per data type per device device type and per manager device type (class or device type) with a number of different equipment types, for example, match

In addition, in this case, the IoT device apparatus diversifies data through the interface unit and transmits it in connection with the DB matching method, so that information can be exchanged efficiently and quickly.

a) To this end, first, an output port table is provided inside the interface unit, for example, in the gateway, and the output port table diversifies information for each equipment type and data type as above and inputs and outputs.

b) So, when transmitting various data in this state, by diversifying the data from the output port table and transmitting it to the control center, etc. quickly and collectively, notification information is provided efficiently and quickly.

c) And, additionally, in this diversification method for data transmission, when information is transmitted in this way, when information in the table is changed in connection with the table, information on the output port table is also changed by synchronizing, for example, IoT device information, etc. do. In this case, the matching relation also includes a format related to the output port table to operate in combination with each other. So, through this, it can be used efficiently in this diversification method.

On the other hand, additionally, when such an IoT device device provides an image for an accident site, etc., it secures a connection to a manager terminal in the vicinity in real time, so that information can be delivered to the manager quickly and easily.

To this end, the IoT device for control performs the following operations.

a) First, when the IoT device for control communicates with the registration manager terminal, for example, by using the above diversification method, it first checks whether the registration local network is connected, and as a result of the check, the local communication network is connected. In the case of connection, it is connected as a setting administrator common account corresponding to each different management work location.

b) As a result of the check, if the local communication network is not connected, it is checked whether the registered wireless communication network is connected secondarily, and when the wireless communication network is connected as a result of the check, an individual IP address is connected.

c) As a result of the check, if the wireless communication network is not connected, the connection is made with the terminal identification number of the registered mobile communication network, so the connection with the manager terminal is secured

On the other hand, in the case of real-time connection with the administrator terminal in this way, for the security of the connection, the IP table is used to monitor the IP and manage the monitoring (or log) according to the connection of the unauthorized person (in this case, the IP table can be done in the same way as the above table of diversification method). In addition, in this case, for example, the IP table uses the same method as the diversification table to diversify data for each manager terminal type or management task type to input/output.

a) Specifically, for this, an IP table in which an administrator public account of the local communication network and individual IP addresses of the wireless communication network are registered is configured in advance.

b) And, when an alarm is provided to the manager terminal in this way, the next hop switch IP address is extracted from the response result by sending a hello message of the corresponding communication network.

c) Next, check the switch IP address that is the same as the next hop switch IP address in the switch neighbor connection relationship list.

d) As a result of the check, if there is a switch IP address that is the same as the IP address of the next hop switch, it is checked whether the corresponding administrator public account or individual IP address is also in the IP table, so that unauthorized access is checked.

e) As a result of the above check, if the corresponding administrator public account or individual IP address is also in the IP table, a JOIN/PRUNE message is transmitted, so it is connected to the corresponding communication network.

100: 5G-based IoT device for accident management vehicle mounting
200: control center
101: CCTV camera 102: IoT device for control
103: 5G video/text transmission/reception module 104: monitor (dashboard for driver confirmation)
105 : IP speaker

Claims (4)

When operating a vehicle, a CCTV camera installed outside the vehicle to capture and collect surrounding images;
a control IoT device installed in the vehicle to transmit the image captured by the CCTV camera to a registration control center, and to receive and output images and text from the control center;
a 5G video/text transmission/reception module installed in the vehicle to receive images and texts between the control IoT device and the control center;
a monitor (dashboard for driver confirmation) displaying a combination of the video of the CCTV camera and the video of the control center by the IoT device for control; and
an IP speaker outputting voice by the IoT device for control or the control center; includes,

The control IoT device,
a) When an accident occurs around the vehicle, the accident scene image from the CCTV camera is transmitted to the control center, and a Wall System image is received from the control center, and the accident scene image and output the wall system image,
b) If a guide text is received when receiving a wall system image from the control center, a combination of the guide text is output on the monitor;
c) When an accident is confirmed through the accident site image, the wall system image, or the guide text, the operation of audio output from the IP speaker is controlled to output an evacuation broadcast;
d) When transmitting the accident scene image to the control center or receiving the wall system image, a single image is created from the accident scene image or the wall system image in a set single image format, and a set single mat format After converting to an image, it is changed to SI (super image) by the setting SR GAN-based deep learning-based image enlargement format, and upscaling to a single image or entire video format corresponding to a number of different CCTV image file formats. By converting it into high-definition video, we deliver accident scene video and wall system video to each other.
The IoT device for control is
a-1) The control center and the table storing the device registration information and data are equally provided with each other, and a matching relationship for the table is set and registered in advance;
b-1) When information in a table is mutually changed, each table is synchronized according to the matching relationship;
c-1) In the case of synchronizing the table, the database is matched by diversifying the information for each data type for a number of different equipment types and for each manager device type;
The IoT device for control is
a-2) In the case of communication with the registration manager terminal, it is first checked whether the registered local communication network is connected, and as a result of the check, when the local communication network is connected, as a setting administrator common account corresponding to different management work locations connect,
b-2) As a result of the above check, if the local communication network is not connected, it is checked whether the registered wireless communication network is connected secondarily, and when the wireless communication network is connected as a result of the check, an individual IP address is connected;
c-2) as a result of the check, if the wireless communication network is not connected, the connection is made with the terminal identification number of the registered mobile communication network, so that the connection with the manager terminal is secured; 5G-based IoT device device for accident management vehicle equipped with an image enlargement/reduction method using 5G video communication, characterized in that.
delete delete The method according to claim 1,
The IoT device for control is
a) preconfiguring an IP table in which an administrator public account of the local communication network and individual IP addresses of the wireless communication network are registered;
b) When an alarm is provided to the manager terminal, the HELLO message of the corresponding communication network is transmitted to extract the next hop switch IP address in the response result,
c) Check the switch IP address identical to the next hop switch IP address in the switch neighbor connection relationship list;
d) As a result of the check, if there is a switch IP address that is the same as the next hop switch IP address, it is checked whether the corresponding administrator public account or individual IP address is also in the IP table, so that unauthorized access is checked;
e) As a result of the check, when the corresponding administrator public account or individual IP address is also in the IP table, the JOIN/PRUNE message is transmitted, so connecting to the corresponding communication network; 5G-based IoT device device for accident management vehicle equipped with an image enlargement/reduction method using 5G video communication, characterized in that.

Images