KR20230102185A - Railway asset management server, system, and method based on artificial intelligence - Google Patents

Abstract

A railroad asset management server is started. The railway asset management server includes an output unit including at least one of a speaker and a display, an information acquisition unit including at least one camera, a radar sensor, a lidar sensor, and an ultrasonic sensor, and based on data obtained from the information acquisition unit and an asset management unit that determines at least one piece of railway asset status information and controls an output unit to output the railway asset status information, wherein the asset management unit includes: a memory that stores data obtained from the information acquisition unit; Acquisition of data of each of the at least one railroad asset based on the identified railroad asset class and an artificial intelligence model learned to identify the at least one railroad asset class based on the data and the learning data generated using the data and one or more processors configured to determine the periodicity and to monitor the at least one railroad asset based on data acquired per determined acquisition periodicity.

Description

AI-based railway asset management server, system and method {RAILWAY ASSET MANAGEMENT SERVER, SYSTEM, AND METHOD BASED ON ARTIFICIAL INTELLIGENCE}

The present disclosure relates to an AI-based railway asset management server, system, and method.

A railway is a concept that includes all facilities necessary for the operation of trains. It is essential to ensure the punctuality and stability of train operation that railroads are managed/maintained in optimal conditions. Therefore, quality monitoring and maintenance for railways must be performed periodically/efficiently.

Conventionally, it has been common to monitor railroad conditions using expensive precision sensors such as contact sensors (eg, linear variable displacement transducers). However, in the case of monitoring the condition of the railroad through such a contact sensor, secondary wear may occur due to contact with the railroad surface. In addition, there was a problem that monitoring the condition of the railroad in this way required a lot of manpower, time and cost.

Registered Patent Publication No. 10-2091165, 2020.03.13

The present disclosure has been made to solve the above-described problems, and an object of the present disclosure is to provide an artificial intelligence-based railway asset management server, system, and method.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

In one embodiment of the present disclosure, the railway asset management server includes an output unit including at least one of a speaker and a display, an information acquisition unit including at least one camera, a radar sensor, a lidar sensor, and an ultrasonic sensor, and the information acquisition unit. and an asset management unit configured to determine at least one piece of railway asset status information based on data acquired by the unit and control the output unit to output the railway asset status information, wherein the asset management unit comprises: based on a memory storing the data, an artificial intelligence model learned to identify the at least one railroad asset class based on the data and the learning data generated using the data, and the identified railroad asset class; and one or more processors configured to determine an acquisition period of the data of each of the at least one railway asset, and to monitor the at least one railway asset based on the data obtained at each determined acquisition period.

And, the one or more processors determine the similarity of the data of each of the at least one railway asset based on the data obtained for each of the determined acquisition periods, and the similarity of the data of each of the at least one railway asset is predetermined If it is less than the value, the output unit may be controlled so that the railway asset state information including a warning signal is output.

Also, the first learning data among the learning data may be generated based on an image acquired through the at least one camera and preprocessing data in which labeling is performed on at least a part of the image.

The one or more processors match weight information to the at least one railway asset class, and the weight information includes a ratio of the data corresponding to each of the radar sensor, the lidar sensor, and the ultrasonic sensor. may be information.

And, the memory includes a plurality of filters for filtering the data acquired through the radar, the LIDAR, and the ultrasonic sensor, and the second learning data among the learning data is the presence of the railway asset in the data. It may be generated based on preprocessing data labeled to identify whether or not.

The one or more processors may determine bandwidths of the plurality of filters through the artificial intelligence model learned based on the second training data.

Among the learning data, third learning data is generated based on pre-processed data in which the at least one railway asset class and the similarity correspond, and the one or more processors perform the learning data based on the third learning data. An acquisition period of the data may be determined through an artificial intelligence model.

The one or more processors determine an asset area corresponding to a railway asset included in the image acquired by the at least one camera, and a contour corresponding to the asset area based on contrast information of the asset area. vector can be determined.

The one or more processors may form data of the railway asset including the contour vector, and determine a similarity of the data of the railway asset using the contour vector.

As another embodiment of the present disclosure, a method for managing railroad assets through a railroad asset management server includes acquiring data through an information obtaining unit including at least one camera, radar, LIDAR, and ultrasonic sensor; identifying the at least one railroad asset class using the data and an artificial intelligence model learned based on the learning data generated using the data; based on the identified railroad asset class, determining an acquisition period of the data of each of the at least one railroad asset; and based on the data acquired per the acquisition period, monitoring the at least one railroad asset.

In addition to this, another method for implementing the present disclosure, another system, and a computer readable recording medium recording a computer program for executing the method may be further provided.

Other specific details of the disclosure are included in the detailed description and drawings.

Through various embodiments of the present disclosure, an artificial intelligence-based railroad asset management server, system, and method may be provided.

Through another embodiment of the present disclosure, the state of railway assets can be efficiently monitored through data and artificial intelligence models acquired through various sensors.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a schematic diagram illustrating a system in which a railway asset management method is implemented according to an embodiment of the present disclosure.
2 is a block diagram illustrating the configuration of a railway asset management server according to an embodiment of the present disclosure.
3 is a flowchart illustrating a railway asset management method according to an embodiment of the present disclosure.
4 is a flowchart illustrating a method of controlling state information and a measurement period for railroad assets according to an embodiment of the present disclosure.

Like reference numbers designate like elements throughout the specification. This specification does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the present disclosure belongs is omitted. The term 'unit, module, member, or block' used in the specification may be implemented as software or hardware, and according to embodiments, a plurality of 'units, modules, members, or blocks' may be implemented as one component, It is also possible that one 'part, module, member, block' includes a plurality of components.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case of being directly connected but also the case of being indirectly connected, and indirect connection includes being connected through a wireless communication network. do.

In addition, when a certain component is said to "include", this means that it may further include other components without excluding other components unless otherwise stated.

Throughout the specification, when a member is said to be located “on” another member, this includes not only a case where a member is in contact with another member, but also a case where another member exists between the two members.

Terms such as first and second are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

Expressions in the singular number include plural expressions unless the context clearly dictates otherwise.

In each step, the identification code is used for convenience of description, and the identification code does not explain the order of each step, and each step may be performed in a different order from the specified order unless a specific order is clearly described in context. there is.

Hereinafter, the working principle and embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a schematic diagram illustrating a system in which a railway asset management method is implemented according to an embodiment of the present disclosure.

In describing the present disclosure, railroad assets may mean various facility assets required for train operation. Railroad assets, for example, include tracks (or rails), traffic lights, tunnel linings, sleepers that directly support the rails on the roadway, pegs that secure the rails to the sleepers, roadbeds that support the tracks, and rail rails. The furnace may include a tie plate inserted between the sleepers, but is not limited thereto.

As shown in FIG. 1, a system 1000 in which a railway asset management method is implemented according to an embodiment of the present disclosure includes a railway asset management server 100, a camera 200-1, and a light detection and ranging (LiDAR) sensor 200-2, radio detection and ranging (radar) sensor 200-3, ultrasonic sensor 200-4, database 300 and railroad asset management server 100. It may include a management device 400 for management.

In FIG. 1, a camera 200-1, a lidar sensor 200-2, a radar sensor 200-3, an ultrasonic sensor 200-4, and a database 300 are included in the railway asset management server 100. Although a case provided outside is shown, this is only one embodiment, and may be provided inside the railway asset management server 100. This will be described in detail with reference to FIG. 2 .

Each device and sensor included in the system 1000 may perform communication through the network W. Here, the network W may include a wired network and a wireless network. For example, the network may include various networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN).

Also, the network W may include the well-known World Wide Web (WWW). However, the network W according to an embodiment of the present disclosure is not limited to the above-listed networks, and may include at least a part of a known wireless data network, a known telephone network, and a known wired/wireless television network.

The camera 200-1, lidar sensor 200-2, radar sensor 200-3, and ultrasonic sensor 200-4 may be geographically distributed and sense different types of data. or can be obtained.

Specifically, the camera 200-1 may acquire comprehensive data such as image data (eg, image data of an area where railroad assets are located) for a specific area. The lidar sensor 200-2 can scan the area where the railroad asset is located by measuring the return time by emitting a laser beam in all directions. The radar sensor 200-3 may emit a laser beam in one direction and acquire the shape, distance, etc. of a railroad asset based on the return time. The radar sensor 200-3 may scan a longer distance than the lidar sensor 200-2, but is not limited thereto and may vary depending on the specifications of each sensor. The ultrasonic sensor 200-3 may scan a nearby area using a wave having a frequency higher than an audible frequency.

1 shows that each sensor exists one by one type, but is not limited thereto, and the number of each sensor may be implemented in various ways according to the design of the system 1000.

The railway asset management server 100 manages railway assets based on data obtained through a camera 200-1, a lidar sensor 200-2, a radar sensor 200-3, and an ultrasonic sensor 200-4. Can be identified/classified. In addition, the railroad asset management server 100 may set/update a data acquisition cycle for each railroad asset by using an artificial intelligence model. In addition, the railroad asset management server 100 may monitor the state of railroad assets based on data obtained for each set/updated acquisition period. An embodiment related to this will be described in detail in an embodiment to be described later.

The database 300 may store data acquired through the camera 200-1, lidar sensor 200-2, radar sensor 200-3, and ultrasonic sensor 200-4. The database 300 may store learning data generated based on the acquired data. Data necessary for the artificial intelligence model to perform training/inference operations may be stored in the database 300 . The database 300 may store data on the situation of each railroad asset.

Here, the artificial intelligence model is an on-device based model provided inside the railroad asset management server 100 or a cloud-based model provided in a separate cloud server and utilized by the railroad asset management server 100. can be implemented

The management device 300 is a device capable of managing/controlling the railway asset management server 100 . 1 shows a case where the management device 300 is implemented as a desktop PC, but is not limited thereto, and may be implemented in various devices such as a tablet PC, a laptop computer, a smart phone, and a wearable device.

2 is a block diagram illustrating the configuration of a railway asset management server 100 according to an embodiment of the present disclosure.

As shown in FIG. 2 , the railway asset management server 100 may include an information acquisition unit 110 , an asset management unit 120 and an output unit 130 .

However, the configuration shown in FIG. 2 is an exemplary diagram for implementing the embodiments of the present disclosure, and appropriate hardware and software configurations obvious to those skilled in the art may be additionally included in the railroad asset management server 100.

The information acquisition unit 110 may include a camera 112 , a lidar sensor 114 , a radar sensor 116 and an ultrasonic sensor 118 . Since the description of each device has been described with reference to FIG. 1, duplicate descriptions will be omitted.

The asset management unit 120 may acquire at least one railroad asset state information based on the data acquired by the information obtaining unit 110 . The asset management unit 120 may include a memory 123 , a processor 126 and an artificial intelligence model 129 .

The memory 123 may correspond to the database 300 described with reference to FIG. 1, and redundant description will be omitted.

The memory 123 may store one or more instructions for performing various operations. The memory 123 may store the data 123-2 acquired through the information acquisition unit 110. At this time, the obtained data 123-2 may be data filtered by one or more filters 123-1.

Here, the one or more filters 123-1 may include an image filter that detects abrasion (eg, corrugated abrasion occurring on a railway line) in the railway within the image data.

As another example, one or more filters 123-1 may include a preprocessing filter for preprocessing data related to railway assets. Here, the preprocessing may include noise removal, thinning, binarization, and the like. The preprocessing filter may include a Gaussian filter, a Laplacian filter, a Difference of Gaussian (DoG), and Canny edge detection.

The processor 126 may be electrically connected to the memory 123 to control overall operations and functions of the railway asset management server 100 .

Processor 126 may consist of one or more processors. At this time, the one or more processors may include a general-purpose processor such as a CPU (Central Processing Unit), an AP (Application Processor), a DSP (Digital Signal Processor), a graphics-only processor such as a GPU (Graphic Processing Unit), a VPU (Vision Processing Unit), or It can be implemented with an artificial intelligence dedicated processor such as an NPU (Neural Processing Unit). The artificial intelligence-only processor may refer to a processor designed with a hardware structure specialized for processing a specific artificial intelligence model.

The processor 126 may determine a data acquisition period of each of the at least one railroad asset based on the railroad asset class (or type) identified through the artificial intelligence model 129 . Here, the acquisition cycle of a specific railroad asset may mean a cycle of acquiring/measuring data on a specific railroad asset.

The processor 126 may monitor at least one railroad asset based on the data obtained through the information acquisition unit 11 for each determined acquisition period.

For example, the processor 126 may determine a similarity of data of each of the at least one railway asset based on data obtained for each determined acquisition period. Here, the similarity of data of each railroad asset may refer to a similarity between previously acquired railroad asset data and recently (or at this time) acquired railroad asset data. Accordingly, the processor 126 may monitor changes that have occurred on railroad assets.

When the similarity of the data of each of the at least one railroad asset is less than a predetermined value, the processor 126 may control the output unit 129 to output railroad asset state information including a warning signal.

Here, that the similarity of each data of a specific railroad asset is less than a predetermined value may mean that a previous state of a specific railroad asset and a recent state of a specific railroad asset are changed. For example, when a wear condition on a track becomes serious, a degree of similarity between previously acquired image data of a track and recently acquired image data of a track may be less than a predetermined value. Accordingly, the processor 126 may control the output unit 129 to output a warning signal requesting inspection of a specific railroad asset and information on the state of a specific railroad asset.

In another embodiment of the present disclosure, the processor 126 may determine an asset area corresponding to a railway asset included in an image acquired by at least one camera 112 . The processor 126 may determine a contour vector corresponding to the asset area based on contrast information of the asset area.

The processor 126 may form data of the railroad asset that includes the contour vectors. In addition, the processor 126 may determine a similarity of railroad asset data using the contour vector.

The artificial intelligence model 129 refers to a model that has been trained to identify at least one railroad asset class. The artificial intelligence model 129 may be learned based on data obtained through the information acquisition unit 110 and learning data generated using the data.

In one embodiment of the present disclosure, the first learning data among the learning data is generated based on an image acquired through at least one camera 112 and preprocessing data in which labeling is performed on at least a part of the image. It can be.

The processor 126 may identify the type of object included in the image acquired using the camera 112 through the artificial intelligence model 129 learned based on the first learning data.

As another embodiment of the present disclosure, the second learning data among the learning data may be generated based on pre-processed data labeled to identify the presence or absence of a railroad asset in the data acquired through the information acquisition unit 110. .

Specifically, the processor 120 may match weight information to at least one railway asset class. The weight information may include data ratios corresponding to each of the radar sensor, lidar sensor, and ultrasonic sensor.

The processor 126 may determine the (frequency) bandwidth of the filter 123-1 through the artificial intelligence model 129 learned based on the second training data. That is, the processor 126 may determine, through the artificial intelligence model 129, a data range of each sensor required to acquire railroad asset information. For example, the processor 126 may determine the acquisition range of the LIDAR sensor to be 10 nm to 60 nm in order to obtain track information through the artificial intelligence model 129 .

As another embodiment of the present disclosure, the third learning data among the learning data may be generated based on preprocessing data corresponding to at least one railroad asset class and a similarity. Here, the degree of similarity means the degree of similarity of each railroad asset obtained by the processor 126 described above. That is, the third learning data may be generated based on preprocessed data in which a specific railway asset class and a similarity corresponding to a specific railway asset are mapped.

The processor 126 may determine a data acquisition period through the artificial intelligence model 129 learned based on the third learning data.

The output unit 130 may include at least one of a speaker and a display. The output unit 130 may output railway asset state information including a warning signal under the control of the processor 126 . For example, a speaker may output an audible alert indicating a warning signal, and a display may display railroad asset status information.

3 is a flowchart illustrating a railway asset management method according to an embodiment of the present disclosure.

The railway asset management server may obtain data through an information acquisition unit including at least one camera, radar sensor, lidar sensor, and ultrasonic sensor (S310).

For example, the railway asset management server may acquire image data for a specific area (eg, an area including railway assets) through a camera. As another example, the railway asset management server may obtain data by scanning a specific area through a radar sensor, lidar sensor, and ultrasonic sensor.

The railway asset management server may identify at least one railway asset class by using an artificial intelligence model learned based on data acquired through the information acquisition unit and learning data generated using the data (S320).

For example, an artificial intelligence model may be trained to identify the existence of a railroad asset and the type of railroad asset within image data acquired through a camera. As another example, the artificial intelligence model may be trained to identify whether a railroad asset exists and the type of railroad asset based on weight information matched to a railroad asset class. Here, the weight information may include data rates corresponding to each of the radar sensor, lidar sensor, and ultrasonic sensor.

The railway asset management server may determine an acquisition period of data of each of at least one railway asset based on the identified railway asset class (S330).

That is, the railway asset management server may determine an acquisition period in which data should be acquired for each railway asset through an artificial intelligence model. For example, the railroad asset management server may determine to acquire data for tracks every day and data for traffic lights every three days.

The railway asset management server may monitor at least one railway asset based on the data obtained for each acquisition period (S340). A detailed description of S340 will be described with reference to FIG. 4 .

4 is a flowchart illustrating a method of controlling state information and a measurement period for railroad assets according to an embodiment of the present disclosure.

The railway asset management server may monitor at least one railway asset based on the data obtained for each data acquisition period for each railway asset (S410).

Specifically, the railway asset management server may identify whether the similarity of data of each railway asset is less than a predetermined value (S420).

That is, the railway asset management server may acquire a similarity between previously acquired data of a specific railway asset and data of a specific railway asset obtained this time, and identify whether the obtained similarity is less than a predetermined value.

When the similarity of data of each railroad asset is less than a predetermined value, the railroad asset management server may output railroad asset state information including a warning signal (S430). That is, the railroad asset management server may monitor railroad assets by outputting a warning signal based on the similarity of data of each railroad asset.

And, the railroad asset management server may update a data acquisition cycle for railroad assets having a similarity less than a predetermined value (S440). For example, if the similarity of data of a specific railroad asset (eg, track, etc.) is less than a predetermined value, the railroad asset management server may reduce the data acquisition cycle for the specific railroad asset (eg, from 3 days to daily units). as).

Embodiments of the present disclosure may be implemented in the form of a recording medium storing instructions executable by a computer. Instructions may be stored in the form of program codes, and when executed by a processor, create program modules to perform operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

Computer-readable recording media include all types of recording media in which instructions that can be decoded by a computer are stored. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, and the like.

As above, the disclosed embodiments have been described with reference to the accompanying drawings. Those skilled in the art to which the present disclosure pertains will understand that the present disclosure may be implemented in a form different from the disclosed embodiments without changing the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

100: railway asset management server
200-1 : Camera
200-2: lidar sensor
200-3: Radar sensor
200-4: ultrasonic sensor

Claims (10)

an output unit including at least one of a speaker and a display;
Information obtaining unit including at least one camera, radar sensor, lidar sensor and ultrasonic sensor; and
Determine at least one railroad asset condition information based on the data obtained by the information acquisition unit;
An asset management unit controlling the output unit so that the railway asset state information is output,
The asset management department,
a memory for storing the data obtained from the information acquisition unit;
an artificial intelligence model learned to identify the at least one railroad asset class based on the data and the learning data generated using the data; and
determine an acquisition period of the data of each of the at least one railroad asset based on the identified railroad asset class;
and one or more processors configured to monitor the at least one railroad asset based on the data acquired per the determined acquisition period. According to claim 1,
The one or more processors,
Determine a degree of similarity of data of each of the at least one railroad asset based on the data acquired for each of the determined acquisition periods;
And controlling the output unit to output the railway asset state information including a warning signal when the similarity of the data of each of the at least one railway asset is less than a predetermined value. According to claim 1,
Among the learning data, the first learning data,
The railway asset management server, which is generated based on the image acquired through the at least one camera and pre-processed data in which labeling is performed on at least a portion of the image. According to claim 1,
The one or more processors,
Match weight information to the at least one railroad asset class;
The weight information,
Information including the ratio of the data corresponding to the radar sensor, the lidar sensor, and the ultrasonic sensor, respectively, is the railroad asset management server. According to claim 1,
the memory,
It includes a plurality of filters for filtering the data acquired through the radar, the lidar, and the ultrasonic sensor,
Among the learning data, the second learning data,
The railway asset management server is created based on pre-processed data labeled to identify the presence or absence of the railway asset in the data. According to claim 5,
The one or more processors,
and determining bandwidths of the plurality of filters through the artificial intelligence model learned based on the second learning data. According to claim 2,
Among the learning data, the third learning data,
The at least one railway asset class and the similarity are generated based on the corresponding pre-processed data;
The one or more processors,
The railway asset management server for determining an acquisition period of the data through the artificial intelligence model learned based on the third learning data. According to claim 1,
The one or more processors,
Determining an asset area corresponding to a railway asset included in an image acquired by the at least one camera;
and determining a contour vector corresponding to the asset area based on contrast information of the asset area. According to claim 8,
The one or more processors,
form data of the railway asset that includes the contour vector;
A railway asset management server for determining a degree of similarity of data of the railway asset using the contour vector. How to manage railroad assets through the railroad asset management server,
Acquiring data through an information acquisition unit including at least one camera, radar, LIDAR, and ultrasonic sensor;
identifying the at least one railroad asset class using the data and an artificial intelligence model learned based on the learning data generated using the data;
based on the identified railroad asset class, determining an acquisition period of the data of each of the at least one railroad asset; and
monitoring said at least one railroad asset based on said data acquired per said acquisition period.

Images