KR102358773B1 - Smart guide device for guiding objects, system and method including the device - Google Patents

Abstract

According to one embodiment of the present invention, a smart guidance board device for guiding objects comprises: a filming unit installed in a location requiring object recognition and monitoring to perform image filming and performing object recognition by an object recognition algorithm for image information; a control unit receiving a filmed image and object recognition result from the filming unit and providing a warning guidance service including pedestrian and vehicle guidance according to the object recognition result; and a display unit provided to visually display the warning guidance service according to the object recognition result under control of the control unit. The present invention can transmit a state check signal by receiving the state check signal from a monitoring server connected to the smart guidance board device through a communication network.

Description

Smart guide device for guiding objects, system and method including the device

The present invention relates to a system and method including a smart notice board device and a smart sign board device, and more particularly, to a system and method capable of performing object recognition and guidance with a smart sign board device and checking the status of the smart sign board device will be.

Recently, as awareness of social safety has been raised, camera installation using object recognition and an integrated control center system are being built.

When the monitoring target area of the camera is wide, a system is used that installs a plurality of monitoring cameras at specific locations in the monitoring target area and divides and displays the area on one monitor screen. When many channels of monitoring cameras are connected, monitoring is performed while the video of the monitoring camera displayed on the screen is periodically changed or the monitoring camera displayed on the screen is selected by the controller.

However, the existing video monitoring equipment simply provides a danger notification by the light emitting unit depending on whether an object is recognized during video monitoring, or even if the object is recognized, the object recognition rate is low depending on the surrounding weather or environment, so it is not suitable for video monitoring. there was also

In addition, even when object recognition is performed, AI learning is performed through a separate server, so the communication environment is important, and there are problems such as delay in learning and object recognition results transmission due to large data transmission, providing rapid image confirmation and monitoring function There was a problem that couldn't be done.

In addition, existing video monitoring equipment only provides video monitoring function, and for separate maintenance, it does not include a device or function capable of diagnosing failure through abnormal condition check, making it difficult to quickly respond and prescribe in case of video monitoring equipment failure. There was a problem.

Therefore, in order to solve the above-mentioned problem, research on a system and method including a smart guide device and a smart guide device is required.

Republic of Korea Patent Publication No. 10-2016-0128169 (published on November 07, 2016)

It is an object of the present invention to be installed at a location where object recognition and monitoring is required, perform image capturing, perform object recognition by an object recognition algorithm on image information, receive a captured image and object recognition result, and apply the object recognition result to the object recognition result. It provides warning guidance service including pedestrian and vehicle guidance according to the result, visually displays warning guidance service according to the object recognition result, and receives a status check signal from a monitoring server connected through a smart information board device and a communication network to receive a status check signal It is to provide a system and method including a smart signboard device capable of transmitting

In the smart guide board device using object recognition in an image according to an embodiment of the present invention, it is installed at a location where object recognition and monitoring are required, performs image capturing, and performs object recognition by object recognition algorithm for image information. filming department; a control unit receiving a captured image and an object recognition result from the photographing unit, and providing a warning guidance service including pedestrian and vehicle guidance according to the object recognition result; and a display unit provided to visually display a warning guidance service according to an object recognition result under the control of the controller, wherein the object recognition algorithm recognizes a specific object of an image by learning a characteristic of a pedestrian shape, a car shape, or a two-wheeled vehicle shape By classifying into values, extracting and analyzing feature values, it is possible to recognize whether an object is an object, and the smart guide board device receives a status check signal from a monitoring server connected through a communication network with the smart guide board device, and provides a status check signal It is characterized in that it can be transmitted.

In the above, in order to be used for object recognition as an auxiliary role in an environment where object identification is difficult due to photographing by the photographing unit when recognizing an object, further comprising a sensor unit having a distance detection sensor for detecting distance information about an object do.

The control unit may perform clustering-based object detection based on the distance information, extract an object candidate group through clustering that binds similar points in a point cloud, and filter from the candidate group. Object recognition algorithm by generating an RGB-D image by detecting an object through classification, or combining at the feature level using the RGB image of the image information of the photographing unit and the distance information in the form of a point cloud of the distance sensor It is characterized in that object detection is performed by

The smart guide device includes: a storage unit for storing image information obtained from the photographing unit, sensing information of the sensor unit, object recognition algorithm, learning data for object recognition, and control information for a warning alarm service through the display unit; a transmission unit including an internal communication protocol and an external communication protocol for communicating with a monitoring server through a communication network, and transmitting a status check signal by receiving a status check signal from the monitoring server; In order to protect information from external hacking risks for data stored in the storage unit or data communicated with a monitoring server or manager terminal, a security unit that applies data encryption/decryption technology to transmission/reception of the data further includes do.

In the system including the smart guide board device according to an embodiment of the present invention, the abnormal state check function of the smart guide board device is performed, and a database is built for events related to abnormalities and failures to be utilized for future maintenance. It further includes a monitoring server to enable it.

In the above, the monitoring server is a communication unit including an external communication protocol for communicating with the transmission unit of the smart guide device; and a status monitoring unit for periodically transmitting a status check signal to the smart guide board device through a communication network, and receiving a status check signal that is a response signal of the smart guide board device to perform status monitoring.

The monitoring server generates and manages event information when an event occurs periodically for the purpose of monitoring and management according to the abnormality of the smart information board device, and transmits a warning or alarm message to a preset manager terminal when an event for an abnormal state occurs, an event management unit that provides corresponding event information upon a request from a manager terminal; and a protection unit for performing data encryption on data communicating with the smart guide device or the manager terminal.

Object guide method according to an embodiment of the present invention, the smart guide device and the monitoring server connected through a communication network, communicating with the smart guide device and periodically transmitting a status check signal for status monitoring; determining, by the monitoring server, whether the state of the device is abnormal by receiving a status check signal, which is a response signal transmitted from the smart guide device; The smart guide board device may include: acquiring image capturing from a photographing unit provided in the smart guide board device or distance information from a sensor unit in a normal state according to the determination of whether the monitoring server is abnormal; The smart guide device includes the steps of identifying and recognizing an object with a pre-learned object recognition model using image information and distance information; The smart information board device includes a step of notifying the object recognition situation around the display unit by displaying object-related information through a warning guidance service such as an object target or an object entry direction according to the recognition result.

When the monitoring server determines that it is an abnormal state, it further includes generating an abnormality, a failure event, and transmitting an alarm to the manager terminal.

The object guidance method may be performed by a computer program stored in a computer-readable recording medium.

The system including the smart sign board device of the present invention performs object recognition by itself by neural network learning for the captured image after the video is taken without the smart sign board device building a separate server, and provides related warning and guidance services based on object recognition. Since it is utilized as a standalone device to provide a service through device internal communication in an edge computing environment, it has the advantage of being able to perform rapid learning and object warning guidance service regardless of external communication environment.

In addition, the smart information board device periodically checks whether there is an abnormality by communication with the server for state monitoring, and when an abnormal state occurs, a warning or an alarm message about the abnormal state is sent to a preset manager terminal to take immediate action, and it can be used for maintenance. there are advantages to

1 is a block diagram showing the overall configuration of a system including a smart guide board device according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the internal configuration of the smart guide device of FIG. 1 in detail.
FIG. 3 is a block diagram showing the internal configuration of the monitoring server of FIG. 1 in detail.
4 is a flowchart of a method for guiding an object of a system including a smart guide device according to an embodiment of the present invention.
5 to 9 are views showing various types of smart guide board device installation examples.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the presented embodiments, and those skilled in the art who understand the spirit of the present invention may add, change, delete, etc. other elements within the scope of the same spirit, and may use other degenerative inventions or the present invention. Other embodiments included within the scope of the invention may be easily proposed, but this will also be included within the scope of the invention. In addition, components having the same function within the scope of the same idea shown in the drawings of each embodiment will be described using the same reference numerals.

1 is a block diagram showing the overall configuration of a system including a smart guide board device according to an embodiment of the present invention, FIG. 2 is a block diagram showing the internal configuration of the smart guide board device of FIG. 1 in detail, and FIG. It is a block diagram showing the internal configuration of the monitoring server in 1 in detail.

As shown in FIG. 1, the system including the smart guide device of the present invention is a smart guide device 100, a monitoring server 200 that monitors the state of the smart guide device 100, and the smart guide device 100. It may include a manager terminal 400 that receives an alarm on whether or not.

The smart information board device 100 performs object recognition by learning an artificial intelligence-based object recognition model on its own for the captured image after image shooting without building a separate server, and provides related warning and guidance services based on object recognition. It is a standalone device provided.

The monitoring server 200 connected to the smart guide device 100 performs an abnormal state check function of the smart guide device 100 by communication with the smart guide device 100, and a database ( 250) so that it can be used for future maintenance.

Specifically, the smart information board device 100 acquires an image with a photographing unit installed in an intersection, blind spot, various protection areas, or a traffic accident risk section, and performs object recognition using an object recognition algorithm by learning a neural network by itself.

The smart guide device 100 may be installed on the side of the display unit 150, but may also be installed on the holding 10, etc. if necessary, and may be installed anywhere within the range that can recognize objects with a camera. , the present invention is not limited thereto.

2 , the smart guide device 100 includes a control unit 110 , a photographing unit 120 , a sensor unit 130 , a storage unit 140 , a display unit 150 , a security unit 160 , and a transmission unit 170 . ) is further included.

The photographing unit 120 is installed at a location where object recognition and monitoring are required, performs image photographing, and performs object recognition on image information.

In addition, the photographing unit 120 may be various types of IP cameras, webcams, CCTVs, etc. with a built-in object recognition algorithm circuit, and the camera type requires a fixed camera, a rotary camera, a dome type camera, a box type camera, a PTZ camera, etc. It can be changed and used accordingly.

In addition, the object recognition algorithm can recognize whether an object exists by classifying a specific object of an image into feature values such as a pedestrian shape, a car shape, and a two-wheeled vehicle shape, for example, and extracting and analyzing the feature values.

As object recognition algorithms, there are Fast R-CNN, R-FCN, You only Look Once (YOLO), TensorFlow, and Single Shot MultiBox Detector (SSD). It can be applied, and the speed of object recognition can be improved by simultaneously recognizing multiple objects by applying the above-described algorithm.

In particular, YOLO is an algorithm that can improve training and detection speed by approaching object recognition by regression and simplifying the overall structure. .

This tensor divides the image into a grid form to express each region, and through this, the region object is recognized. In addition, YOLO has the advantage of fast processing speed and high accuracy for recognizing objects.

In particular, by applying YOLO V4 in the present invention, high accuracy and fast object detector can be trained, and although the speed is slightly slower than Yolo V3, the small object classification accuracy is higher.

In addition, if it is determined that the object is held by a person by the small object classification accuracy, it is thrown away and the object recognition accuracy can be increased so that it can be determined as a person even if it is pulled a cart, a cane, or a wheelchair, and the number of detected objects does not affect the accuracy of classification.

Also, basically, object tracking can be performed for the entire area, not a part of the image information, and the background is separated through the background difference technique (K-Nearest, KNN), and only the difference from the background is checked to proceed with object tracking. can

In addition, the photographing unit 120 may simply perform photographing, transmit the photographed image to the control unit 110 , and determine whether the object is recognized by performing the object recognition algorithm described above in the control unit 110 .

The controller 110 may receive the captured image and the object recognition result from the photographing unit 120 , and may provide a warning guidance service such as pedestrian and vehicle guidance according to the object recognition result.

Specifically, with reference to FIGS. 5 to 9 , the controller 110 may control to display an entry direction, a recognized object target, a related guide phrase, and the like through the display unit 150 in various forms.

For example, according to object recognition, warnings about an object entering from a different entry direction based on the corresponding access road are displayed in the form of text, image, or icon, but depending on the entry distance, different text, images, or icons for the object are displayed or , the display unit 150 may be controlled to display the remaining distance from the current location of the object or the speed of the object.

In addition, the caution statement may indicate danger if the entry distance is a dangerous section, and a warning if it is a warning section, and display different colors for each section. It is also possible to change the shape or display a different color.

The sensor unit 130 serves as an auxiliary role in an environment where object identification is difficult due to photographing by the photographing unit 120 during object recognition, and is provided to collect sensor-based sensing information and utilize it for object recognition.

Specifically, the sensor unit 130 may be a lidar sensor having a distance sensing function, and in addition to the lidar sensor, a distance sensing sensor such as a laser sensor, a temperature/humidity sensor, a thermal image sensor, and the like may be additionally utilized.

Since the distance information of the object detected by the distance sensor is not affected by the weather or the surrounding weather environment, the control unit ( 110), which can be used for object recognition and discrimination.

Specifically, the control unit 110 may perform clustering-based object detection based on distance information, and extract an object candidate group through clustering that binds similar points in a point cloud, and from the candidate group Objects can be detected through classification through filtering.

For clustering, K-means clustering can be used, for example, and it is a method of clustering the entire cloud point into K groups so that the Euclidean distance is minimized around a specific reference point. The procedure defines the number of K clusters, locates the centroid, and groups the data collected at each centroid into one cluster.

Also, hierarchical clustering is a method of grouping similar clusters together until they are finally unified. It is an algorithm based on the distance between clusters, and unlike K-means, it is not necessary to set the number of clusters separately.

In addition, the RGB image of the image information of the photographing unit 120 and the point cloud distance information of the lidar sensor are combined at the feature level to generate an RGB-D image and can be used in algorithms such as object detection.

The storage unit 140 includes image information acquired from the photographing unit 120 , sensing information of the sensor unit 130 , an object recognition algorithm, learning data for object recognition, and control information for a warning alarm service through the display unit 150 . etc. may be stored.

The display unit 150 is provided to visually display a warning guidance service according to the object recognition result under the control of the control unit 110, and is shown in Figs. As in the example of 9, it can be represented in various forms.

In addition, the display unit 150 is provided in the form of a front LED panel on the transverse arm of the post 10, as shown in FIG. 5, which can be implemented in various types of installation methods, and the photographing unit 120 is installed as a built-in type or in FIG. As shown, the photographing unit 120 may be independently installed above the display unit 150 , and a separate sensor unit 130 (a lidar sensor, etc.) may also be installed on the transverse arm pole 30 to facilitate sensing.

In addition, the remaining components including the control unit 110 (storage unit 140, security unit 160, and transmission unit 170) may be installed in the post 10, and the display unit by embedding a line in the post 10. It may be installed to be connected to 150 .

In addition, in FIG. 6 , a plurality of display units 150 may be provided in the form of vertical bars on both sides of the guide sign 20 , and a photographing unit 120 (camera) may be provided as a built-in type at the indicated position in the direction of the arrow.

7 is a form in which the display unit 150 and the photographing unit 120 are embedded in the child protection area guide sign 20 installed on the iron pillar, and FIG. 8 is similar to FIG. 7, but corresponds to the case where the iron pillar is a concrete post 10 do.

The security unit 160 transmits the data stored in the storage unit 140 or the data communicated with the monitoring server 200 or the manager terminal 400 to protect information from risks such as external hacking. Data encryption/decryption technology can be applied to /receive.

More specifically, by giving identification information, each of which can be identified for identification, to the smart guide device 100, the monitoring server 200 and the manager terminal 400, each monitoring server 200 or the manager terminal 400 ) to perform a lightweight encryption algorithm using the identification information as a private key. The lightweight encryption algorithm is an encryption technology designed to be implemented in a limited environment such as a smart device such as the administrator terminal 400. The symmetric key encryption algorithm HIGHT (HIGh security and light weigHT), LEA (Lightweight Encryption), and the hash function LSH (Lightweight Secure Hash), etc. can be used.

By using this lightweight encryption algorithm to encrypt/decrypt data, it is possible to prevent illegal control due to external leakage or external hacking of the data. The lightweight cryptographic algorithm preferably uses a lightweight hash function suitable for such an embedded computing environment.

A lightweight hash function is a hash function that consumes relatively low computing power designed to ensure the integrity of transmitted or received data, excluding features that require high computing power in standard cryptographic hash algorithms such as SHA-3. (one-way function).

More specifically, it is preferable to use a sponge algorithm that enables permutation of data without a key (unkeyed) among these lightweight hash functions.

More specifically, the sponge makes the original message (here, the original data of the random key) a certain size (padding), and then converts it to a specific standard size that only the creator of the key can know (for example, the original message divided into a specific bit size). After dividing into plural pieces, random data is exchanged using several update functions at the rear end of the divided data (segmented original message), and on the other side, it is implemented to decode using a known reference size.

The transmission unit 170 may include an internal communication protocol and an external communication protocol for communicating with the monitoring server 200 through the communication network 300 , and receives a status check signal from the monitoring server 200 to receive a status check signal can be sent.

Examples of communication may include RS-485 communication, an intranet, a private network (VPN) used in the internal communication network 300 , and the wired/wireless communication network 300 for external communication.

The status monitoring function will be described in detail through the status monitoring unit 220 of the monitoring server 200 to be described later.

Furthermore, the smart guide board device 100 may further include a power supply unit (not shown) for supplying internal power.

The monitoring server 200 further includes a communication unit 210 , a status monitoring unit 220 , an event management unit 230 , a protection unit 240 , and a database 250 as shown in FIG. 3 .

The communication unit 210 includes an external communication protocol for communicating with the transmission unit 170 of the smart guide device 100 .

The status monitoring unit 220 may periodically transmit a status check signal to the smart guide panel device 100 through the communication network 300 , and receive a status check signal that is a response signal of the device to perform status monitoring.

Specifically, there is a difference in the method depending on the communication environment, but in the case of a communication environment such as wireless WIFI, a communication status command such as PING can be transmitted to the smart guide panel device 100 and the status can be checked with the received status check signal, A state check signal may be generated and transmitted to the smart guide device 100, and the response signal may include information on whether the state is abnormal (alive or dead).

As another method, the state monitoring unit 220 sends a state check signal in a form similar to the heart-beat signal at every specific reference period to check the state, and in an abnormal situation (including a failure), it may take advantage of the fact that the signal does not arrive within the period. .

Specifically, if the signal does not arrive within the period, the signal is transmitted with a reference period shorter than the reference period. When the count value is higher than the set reference count value, it is estimated as a failure, so that it is possible to reduce misdiagnosis of a failure for device managers and to increase the reliability of an appropriate time for maintenance.

The event management unit 230 may generate and manage event information when an event occurs periodically for the purpose of monitoring and management depending on whether the smart guide device 100 is abnormal, and a preset manager terminal 400 when an event for an abnormal state occurs. to transmit a warning or alarm message, and to provide corresponding event information upon request of the manager terminal 400 to be utilized for maintenance and the like.

The protection unit 240 may perform data encryption on data for communication to prevent abuse such as external hacking.

The data encryption method may be, for example, HTTPS (TLS) encryption, and may additionally utilize or replace the above-described lightweight encryption algorithm.

The database 250 may store and manage event data transmitted and received or data transmitted for backup from the smart guide device 100 (object recognition data-image information, sensing information, etc.), and at this time, block for data protection A chain-based distributed communication network 300 can be built and managed.

4 is a flowchart of a method for guiding an object of a system including a smart guide device according to an embodiment of the present invention.

First, the monitoring server 200 communicates with the smart guide device 100 and periodically transmits a status check signal for status monitoring (S401).

The monitoring server 200 determines whether the state of the device is abnormal by receiving the status check signal, which is a response signal transmitted from the smart guide device 100 (S402).

When the monitoring server 200 is determined to be an abnormal state, it may generate a failure event and transmit an alarm to the manager terminal 400 (S403).

The smart guide board device 100 acquires image capture or distance information from the sensor unit 130 from the photographing unit 120 provided in the smart guide panel apparatus 100 in a normal state according to the determination of whether the monitoring server 200 is abnormal. do (S404).

The smart guide device 100 may identify and recognize an object with an object recognition model learned in advance using image information and distance information (S405).

According to the recognition result, the smart guide panel device 100 displays object-related information through the display unit 150 through a warning guide service such as an object target or an object entry direction to notify the surrounding object recognition situation (S406).

In the present specification, a 'terminal' may be a wireless communication device with guaranteed portability and mobility, for example, any type of handheld-based wireless communication device such as a smartphone, tablet PC, or notebook computer. In addition, the 'terminal' may be a wired communication device such as a PC that can be connected to another terminal or server through the communication network 300 . In addition, the communication network 300 refers to a connection structure capable of exchanging information between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), It includes the Internet (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, and the like.

Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

10 : holding
20: guide sign
30: transverse arm pole
100: smart information board device
110: control unit
120: shooting unit
130: sensor unit
140: storage
150: display
160: security department
170: transmission unit
200: monitoring server
210: communication department
220: status monitoring unit
230: event management unit
240: protection
250 : database
300: communication network
400: administrator terminal

Claims (10)

In a smart guide board device for guiding objects using object recognition in an image,
a photographing unit installed at a location requiring object recognition and monitoring to capture an image, and to perform object recognition on image information by an object recognition algorithm;
a control unit receiving a captured image and an object recognition result from the photographing unit, and providing a warning guidance service including pedestrian and vehicle guidance according to the object recognition result;
a display unit provided to visually display a warning guidance service according to an object recognition result under the control of the control unit;
a sensor unit having a distance detection sensor for detecting distance information about an object in order to use it for object recognition as an auxiliary role in an environment where object identification is difficult due to photographing by the photographing unit during object recognition;
a storage unit for storing image information obtained from the photographing unit, sensing information of the sensor unit, object recognition algorithm, learning data for object recognition, and control information for a warning alarm service through a display unit;
a transmission unit including an internal communication protocol and an external communication protocol for communicating with a monitoring server through a communication network, and transmitting a status check signal by receiving a status check signal from the monitoring server;
In order to protect information from external hacking risk for data stored in the storage unit or data communicated with the monitoring server or manager terminal, the smart information board device, monitoring server, and manager terminal respectively identify the corresponding data in transmission/reception A security unit that applies data encryption/decryption technology using a light-weight encryption algorithm that grants verification, identification information, and uses the identification information of each monitoring server or administrator terminal as a private key includes,
The object recognition algorithm classifies a specific object of an image into a feature value for a pedestrian shape, a car shape, or a two-wheeled vehicle shape by learning, and by extracting and analyzing the feature value, it is possible to recognize whether the object is an object,
The smart guide device is
By receiving the status check signal from the monitoring server connected through the smart guide device and the communication network, the status check signal can be transmitted,
The monitoring server includes a status monitoring unit that periodically transmits a status check signal to the smart guide device through a communication network, receives a status check signal that is a response signal of the device, and performs status monitoring,
The status monitoring unit sends a status check signal similar to the heart-beat signal at every specific reference cycle to check the status. there is,
If the signal does not arrive within the period, the signal is transmitted in a reference period shorter than the reference period, and if the abnormal situation persists, it is estimated as a failure or a counter is provided to count the number of retry, and the current count value If it is higher than the set reference count value, the smart guide board device, characterized in that it is estimated as a failure. delete According to claim 1,
the control unit
Clustering-based object detection can be performed based on the distance information, and object candidate groups are extracted through clustering that binds similar points in a point cloud, and classification is performed through filtering from the candidate group. (classification) to detect an object, or
Smart, characterized in that object detection is performed by an object recognition algorithm by combining at the feature level using the RGB image of the image information of the photographing unit and the distance information in the form of a point cloud of the distance sensor to generate an RGB-D image signage device. delete [Claim 4] In a system comprising the smart guide device of claim 1 or 3,
A monitoring server that performs an abnormal state check function of the smart guide device and builds a database for failure-related events so that it can be utilized for future maintenance
A system including a smart signage device comprising a. 6. The method of claim 5,
The monitoring server
a communication unit including an external communication protocol for communicating with the transmission unit of the smart guide device;
A status monitoring unit that periodically transmits a status check signal to the smart guide device through a communication network, and receives a status check signal that is a response signal of the smart guide device to monitor the status
A system including a smart signage device comprising a. 6. The method of claim 5,
The monitoring server
For the purpose of monitoring and management according to the abnormality of the smart information board device, event information is generated and managed periodically when an event occurs, and when an event for an abnormal state occurs, a warning or alarm message is transmitted to a preset manager terminal, and the request of the manager terminal an event management unit that provides relevant event information at the time;
A protection unit that performs data encryption for data communicating with the smart guide device or the manager terminal
A system including a smart signage device comprising a. The monitoring server connected to the smart guide device through a communication network communicates with the smart guide device and periodically transmits a status check signal for status monitoring;
Determining whether the state of the device is abnormal by the monitoring server receiving a status check signal that is a response signal transmitted from the smart guide device;
The smart guide board device may include: acquiring image capturing from a photographing unit provided in the smart guide board device or distance information from a sensor unit in a normal state according to the determination of whether the monitoring server is abnormal;
The smart guide device may include: identifying and recognizing an object with an object recognition model learned in advance using image information and distance information;
The smart information board device displays object-related information through a display unit through a warning guidance service such as an object target or an object entry direction according to the recognition result to notify the surrounding object recognition situation
including,
The smart guide device is
a sensor unit having a distance detection sensor for detecting distance information about an object in order to use it for object recognition as an auxiliary role in an environment where object identification is difficult due to photographing by the photographing unit during object recognition;
a storage unit for storing image information obtained from the photographing unit, sensing information of the sensor unit, object recognition algorithm, learning data for object recognition, and control information for a warning alarm service through a display unit;
a transmission unit including an internal communication protocol and an external communication protocol for communicating with a monitoring server through a communication network, and transmitting a status check signal by receiving a status check signal from the monitoring server;
In order to protect information from external hacking risk for data stored in the storage unit or data communicated with the monitoring server or manager terminal, the smart information board device, monitoring server, and manager terminal respectively identify the corresponding data in transmission/reception A security unit that applies data encryption/decryption technology using a light-weight encryption algorithm that grants verification, identification information, and uses the identification information of each monitoring server or administrator terminal as a private key includes,
The monitoring server includes a status monitoring unit that periodically transmits a status check signal to the smart guide device through a communication network, receives a status check signal that is a response signal of the device, and performs status monitoring,
The status monitoring unit sends a status check signal similar to the heart-beat signal at every specific reference cycle to check the status. there is,
If the signal does not arrive within the period, the signal is transmitted in a reference period shorter than the reference period, and if the abnormal situation persists, it is estimated as a failure or a counter is provided to count the number of retry, and the current count value Object guidance method, characterized in that if it is higher than the set reference count value, it is estimated as a failure. 9. The method of claim 8,
When the monitoring server determines that it is in an abnormal state, generating a failure event and transmitting an alarm to the manager terminal
An object guidance method further comprising a. A computer program stored in a computer-readable recording medium for performing the object guidance method of claim 8 or 9.

Images