KR20230077215A - System for real-time high speed detection of rail way facility using machine vision algorithm based on artificial intelligent - Google Patents

Abstract

The present invention relates to a real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, which is installed on both sides of the upper part of a train moving on the track of the railway track facility, respectively, and is installed on both sides of the upper part of the corresponding train line by line. First and second line scan camera devices for scanning images to acquire images having two-dimensional images, and installed on both sides of the upper part of the corresponding train, respectively, in the form of a line of the same optical axis as the first and second line scan camera devices. The camera shutters of the first and second line scan camera devices are opened in synchronization with the trigger signal of the first and second line lighting devices that emit light and the preset camera shutter, and the first and second line lighting devices are illuminated at the same time. controls the operation of the first and second line scan camera devices and the first and second line lighting devices to emit light, receives images of both upper sides of the corresponding train obtained from the first and second line scan camera devices, and receives the corresponding railway In order to detect the object to be inspected on the upper part of the track facility, video frames on both sides of the upper part of the corresponding train are divided through a predetermined frame division technique, and in order to determine the position of the subject to be inspected in the upper part of the image frame on both sides of the upper part of the divided train, the corresponding railway track After extracting the edge components of the upper detailed parts of the facility, based on the edge components of the extracted upper detailed parts of the railway track facility, the railway track facility By including a machine vision control device that repeatedly learns the geometric relation of the upper detailed parts to detect the upper object to be inspected and detects anomaly of the upper object to be inspected, It is possible to effectively secure the stability of the train by quickly and accurately detecting and detecting abnormality on the lower object to be inspected.

Description

Real-time high-speed detection system of railway track facilities using artificial intelligence-based machine vision algorithm

The present invention relates to a real-time high-speed detection system for railway track facilities using an artificial intelligence (AI)-based machine vision algorithm.

In general, the difference between railroads and other means of transportation is that electric railroad vehicles (or trains) (eg, electric trains, subways, high-speed trains, etc.) run on the tracks, and electric railroad vehicles for transporting people or cargo It is moved along the railroad tracks installed on the ground or underground.

The electric railway vehicle is used by many people due to its speedy, accurate and stable advantages compared to other means of transportation. Moreover, with the recent opening of the high-speed rail, it is becoming more and more popular as a means of public transportation.

These electric railway vehicles mainly use a pantograph, which is a power collecting system, and the electric railway vehicle can stably operate at high speed only when the pantograph and the tram line are accurately contacted.

That is, a normal pantograph is equipped with a current collector in contact with a wire on a diamond-shaped foldable frame installed on the roof of an electric railway vehicle, and the wire is pushed up so that the wire is in close contact with the force of a spring or compressed air.

The pantograph is an important facility for delivering electricity required for an electric railway vehicle to a vehicle through contact with a catenary in an electric railway power supply system. As the speed of the electric railway vehicle increases, the separation of the pantograph with the catenary increases, resulting in an increasing number of damage cases of detailed parts such as current collectors and pantographs due to arcs and impacts.

In particular, since detailed parts that directly affect the operation of the electric railway vehicle are installed in the upper and/or lower part of the electric railway vehicle, the accuracy of safety inspection is low to determine whether or not the detailed parts are damaged only with the operator's eyes and experience. There was a problem.

Therefore, it is necessary to develop a system that can determine the abnormal state of detailed parts including these pantographs even during the high-speed operation of electric railway vehicles, and through rapid and accurate precision diagnosis of detailed parts of railway track facilities, electric railway There is an urgent need for inspection technology to secure vehicle stability.

Domestic Patent Registration No. 10-1058179 (Announced on August 22, 2011)

The present invention has been made to solve the above problems, and an object of the present invention is to irradiate light in a line form to both upper and / or lower sides of a train moving on a track of a railroad track facility, and the corresponding train line by line. After obtaining an image with a two-dimensional image by scanning both images of the upper and / or lower sides of the rail, artificial intelligence (AI) is obtained by using the entire image with the two-dimensional image of the upper and / or lower part of the railway track facility )-based machine vision algorithm to automatically perform detection and abnormality detection of the upper and/or lower part of the railway track facility, thereby detecting and abnormality of the upper and/or lower part of the railway track facility It is to provide a real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm that can effectively secure the stability of the train by quickly and accurately performing it.

In order to achieve the above object, one aspect of the present invention is installed on both sides of the upper part of a train moving on the track of a railway track facility, respectively, and scans images on both sides of the upper part of the train in units of lines to obtain an image having a two-dimensional image. First and second line scan camera devices for acquiring ; first and second line lighting devices respectively installed on both sides of the upper part of the corresponding train and irradiating light in the form of a line of the same optical axis as the first and second line scan camera devices; and the camera shutters of the first and second line-scan camera devices are opened in synchronization with a trigger signal of a preset camera shutter, and the first and second line-scan cameras emit light from the first and second line lighting devices at the same time. It controls the operation of the camera device and the first and second line lighting devices, receives images of both sides of the upper part of the corresponding train obtained from the first and second line scan camera devices, and detects the object to be inspected from the upper part of the corresponding railroad track facility. In order to do this, video frames on both sides of the upper part of the corresponding train are divided through a predetermined frame division technique, and in the video frames on both sides of the upper part of the divided train, the upper detailed parts of the railway track facility are used to determine the position of the upper object to be inspected. After extracting the edge component for the railway track facility, based on the edge component for the upper detail parts of the railway track facility extracted above, through a predetermined artificial intelligence (AI)-based machine learning method, the upper detail parts of the railway track facility Real-time high-speed detection of railway track facilities using an artificial intelligence-based machine vision algorithm that includes a machine vision control device that repeatedly learns the geometric relation for the upper object to be inspected and detects anomaly of the upper object to be inspected to provide the system.

Here, the machine vision control device receives the images on both sides of the upper part of the corresponding train obtained from the first and second line scan camera devices, and detects the object to be inspected in the upper part of the corresponding railway track facility through a predetermined frame segmentation technique. An image segmentation module dividing image frames on both upper sides of the train; An edge extraction module for extracting edge components of upper detailed parts of the railway track facility in order to determine the position of the upper object to be inspected in the image frames on both sides of the upper part of the train divided by the image segmentation module; Based on the edge components of the upper detailed parts of the railway track facility extracted from the edge extraction module, geometric association of the upper detailed parts of the railway track facility through a predetermined artificial intelligence (AI)-based machine learning method A first artificial intelligence learning module for constructing a first artificial intelligence learning model for detecting a corresponding upper subject by repeatedly learning the relationship; Using the first artificial intelligence learning model built from the first artificial intelligence learning module, inferring and analyzing the appearance and location characteristics of the corresponding upper target object for each normal/abnormal feature item of the detected upper target object, and the inference And after classifying the type of anomaly with respect to the upper subject subject based on the appearance and location characteristic information of the analyzed upper subject object, the upper subject subject is classified through a predetermined artificial intelligence (AI) based machine learning method. A second artificial intelligence learning module for constructing a second artificial intelligence learning model for abnormality detection of the upper subject by conducting repetitive learning according to the classified abnormality type; The second artificial intelligence learning model built from the second artificial intelligence learning module is used to infer and analyze the characteristics of the abnormality type of the upper object to be inspected for each characteristic item of the abnormality type of the upper object to be inspected, An upper test object abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the upper test object based on the inferred and analyzed characteristic information on the abnormal occurrence type of the upper test object; and the camera shutters of the first and second line-scan camera devices are opened in synchronization with a trigger signal of a preset camera shutter, and the first and second line-scan cameras emit light from the first and second line lighting devices at the same time. It is preferable to include a camera device and a machine vision control module for controlling the operation of the first and second line lighting devices.

Preferably, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention is installed on both sides of the lower part of the corresponding train, respectively, and images of both lower sides of the corresponding train are displayed in units of lines. Third and fourth line scan camera devices for acquiring images having two-dimensional images by scanning; and third and fourth line lighting devices respectively installed on both sides of the lower part of the corresponding train and irradiating light in a line form of the same optical axis as the third and fourth line scan camera devices.

Preferably, the machine vision control device is synchronized with a trigger signal of a preset camera shutter, and the camera shutters of the first and second line scan camera devices are opened and lights of the first and second line lighting devices are emitted at the same time. In addition to controlling the operation of the first and second line scan camera devices and the first and second line lighting devices, the camera shutters of the third and fourth line scan camera devices are opened simultaneously with the third and second line scan camera devices. 4 Controls the operation of the third and fourth line scan camera devices and the third and fourth line lighting devices so that the lights of the line lighting devices are emitted, and the corresponding train obtained from the first and second line scan camera devices; In order to detect the upper object to be inspected by receiving the images on both sides of the upper portion of the train, image frames on both sides of the upper portion of the corresponding train are divided through a predetermined frame division technique, and the position of the corresponding upper object to be inspected is determined in the image frames on both sides of the upper portion of the divided corresponding train. After extracting the edge components of the upper detailed parts of the railway track facility for the purpose, artificial intelligence (AI) based machine learning method based on the edge components of the upper detailed parts of the extracted railway track facility Through iterative learning of the geometric relation of the upper detailed parts of the railway track facility to detect the upper object to be inspected and to detect the abnormality of the upper object to be inspected, the third and fourth line scan camera devices Received images of both sides of the lower part of the train obtained from the machine learning method based on artificial intelligence (AI) to repeatedly learn the relative position and shape of the lower part of the railway track facility to detect the lower part of the target object In addition, it is possible to perform abnormality detection of the lower object to be inspected.

Preferably, the machine vision control device receives the images of the upper sides of the corresponding train obtained from the first and second line scan camera devices and uses a predetermined frame division technique for detecting the upper object to be inspected on both sides of the upper portion of the corresponding train. An image segmentation module dividing an image frame; An edge extraction module for extracting edge components of upper detailed parts of the railway track facility in order to determine the position of the upper object to be inspected in the image frames on both sides of the upper part of the train divided by the image segmentation module; Based on the edge components of the upper detailed parts of the railway track facility extracted from the edge extraction module, geometric association of the upper detailed parts of the railway track facility through a predetermined artificial intelligence (AI)-based machine learning method A first artificial intelligence learning module for constructing a first artificial intelligence learning model for detecting a corresponding upper subject by repeatedly learning the relationship; Using the first artificial intelligence learning model built from the first artificial intelligence learning module, inferring and analyzing the appearance and location characteristics of the corresponding upper target object for each normal/abnormal feature item of the detected upper target object, and the inference And after classifying the type of anomaly with respect to the upper subject subject based on the appearance and location characteristic information of the analyzed upper subject object, the upper subject subject is classified through a predetermined artificial intelligence (AI) based machine learning method. A second artificial intelligence learning module for constructing a second artificial intelligence learning model for abnormality detection of the upper subject by conducting repetitive learning according to the classified abnormality type; The second artificial intelligence learning model built from the second artificial intelligence learning module is used to infer and analyze the characteristics of the abnormality type of the upper object to be inspected for each characteristic item of the abnormality type of the upper object to be inspected, An upper test object abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the upper test object based on the inferred and analyzed characteristic information on the abnormal occurrence type of the upper test object; The images of both sides of the lower part of the train obtained from the third and fourth line scan camera devices are provided, and through a predetermined AI-based machine learning method, the appearance and relative position or geometric relation of the corresponding lower part to be inspected are used. A third artificial intelligence learning module for constructing a third artificial intelligence learning model for detection of a corresponding lower subject by repeatedly learning; Using the third artificial intelligence learning model built from the third artificial intelligence learning module, inference and analysis of the appearance and location characteristics of the corresponding lower subject for each normal/abnormal characteristic item of the detected lower subject, and the inference And after classifying the type of anomaly with respect to the corresponding lower subject based on the appearance and location characteristic information of the analyzed lower subject, the corresponding lower subject is classified through a machine learning method based on artificial intelligence (AI). A fourth artificial intelligence learning module for constructing a fourth artificial intelligence learning model for abnormality detection of the lower subject by performing repetitive learning according to the classified abnormality type; The fourth artificial intelligence learning model built from the fourth artificial intelligence learning module is used to infer and analyze the characteristics of the abnormal occurrence type of the lower inspected object for each characteristic item of the abnormal occurrence type for the corresponding lower inspected object, A lower subject abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the corresponding lower subject subject based on the inferred and analyzed characteristic information on the type of abnormal occurrence of the lower subject subject; and the camera shutters of the first and second line-scan camera devices are opened in synchronization with a trigger signal of a preset camera shutter, and the first and second line-scan cameras emit light from the first and second line lighting devices at the same time. Controls the operation of the camera device and the first and second line lighting devices, and simultaneously opens the camera shutters of the third and fourth line scan camera devices and emits light from the third and fourth line lighting devices. A machine vision control module for controlling operations of the third and fourth line scan camera devices and the third and fourth line lighting devices may be included.

Preferably, the artificial intelligence (AI)-based machine learning method applied to the first to fourth artificial intelligence learning modules includes a neural network, a support vector machine (SVM), a multi-layer perception (MLP), and deep learning. (Deep Learning) can be achieved by at least one artificial intelligence learning method.

Preferably, the machine vision control device detects the corresponding lower part under test by using a preset machine vision algorithm based on images of both sides of the lower part of the corresponding train obtained from the third and fourth line scan camera devices, and then In addition, through a machine learning method based on artificial intelligence (AI), it is possible to perform abnormality detection of the lower part of the subject.

Preferably, the preset machine vision algorithm is at least one of pattern matching, point/line/plane fitting, edge, color, and location algorithms. It can be done.

Preferably, the lower object to be inspected is a lower detail part of the railway track facility, as long as it is fixed side by side in a direction orthogonal to a plurality of sleepers arranged at regular intervals on the ground, and at the upper ends of both sides of the plurality of sleepers. It may be made of at least one lower detailed part of a pair of railway rails and a pan control for fixing the pair of railway rails on the plurality of sleepers.

Preferably, when the lower object to be inspected is a plurality of sleepers, the machine vision control device may perform sleeper abnormality detection of breakage of the sleeper.

Preferably, when the lower object to be inspected is a pair of railroad rails, the machine vision control device performs at least one of rail waviness wear, tongue rail wear, and rail head damage. Rail abnormality detection can be performed.

Preferably, when the lower object to be inspected is a pan control, the machine vision controller may detect an abnormality in pan control removal.

Preferably, the machine vision control device includes LEDs provided in the first and second line lighting devices during camera shutter exposure times of the first and second line scan camera devices and the third and fourth line scan camera devices. By applying the output intensity of the light emitting element and the LED light emitting element provided in the third and fourth line lighting devices to a preset overdrive constant current value in the form of a pulse signal greater than the rated continuous maximum value, high brightness can be obtained. The operations of the first and second line lighting devices and the third and fourth line lighting devices may be respectively controlled by a constant current over drive control method using pulse control.

Preferably, the over-driving constant current value is the cooling time of the LED light emitting elements provided in the first and second line lighting devices and the LED light emitting devices provided in the third and fourth line lighting devices between each pulse signal. can be set in consideration of each.

Preferably, the upper object to be inspected is a pantograph for collecting electric energy from the corresponding train line by contacting the corresponding train line, and supplying power to the corresponding train through the pantograph, and providing power to the tunnel or underground space. It may be made of at least one of T-bars, which are fixing means used in a device for fixing a catenary of a corresponding train to a ceiling surface.

Preferably, the first and second line lighting devices and the third and fourth line lighting devices may each include a plurality of LED light emitting channels including at least one LED light emitting element connected in parallel with each other.

Preferably, the machine vision control device emits light of each LED provided in the first and second line lighting devices and the third and fourth line lighting devices, respectively, in response to a user's specific input signal output according to a user's request. Channel output level target value setting, output channel selection and order setting of each LED light emitting channel for high-speed switching control, and clock generation for generating high-speed switching operation signals are set, and each LED set above is set. In order to adjust the brightness of each LED light emitting element provided in each LED light emitting channel based on the output level target value of the light emitting channel, a target output voltage of each LED light emitting channel is generated and output, and the output of each LED light emitting channel set above Provided respectively in the first and second line lighting devices and the third and fourth line lighting devices according to the output voltage target value of each LED light emitting channel outputted in synchronization with channel selection and order setting information and clock information High-speed switching of each LED light emitting channel can be controlled.

Preferably, the machine vision control device may output a specific input signal of a corresponding user according to a request of the corresponding user based on a graphical user interface (GUI).

Preferably, the machine vision controller may perform high-speed control by using a hardware programming-based Field Programmable Gate Array (FPGA) for a high-speed switching function.

Preferably, the machine vision control device includes first and second line scan camera devices and third and fourth line scan camera devices installed near the first and second line lighting devices and the third and fourth line lighting devices, respectively. An idle period of the photographing operation for the scan camera device is detected, and the photographing operation signals of the first and second line scan camera apparatuses and the third and fourth line scan camera apparatuses are not input for more than a preset photographing operation reference time. It is possible to perform an automatic reset function that determines the time as an idle state, removes a state during a photographing operation and initializes it to an initial input state so that the first state is maintained after the determined idle state.

Preferably, the machine vision control device includes a set value for a target output level of each LED light emitting channel, a set value for output channel selection and order of each LED light emitting channel, and a set value for generating a clock. At least one set value may be controlled to be stored in a separate storage module.

Preferably, the user's specific input signal output according to the user's request may be provided through an external terminal or server.

Preferably, the external terminal or server transmits the user's specific input signal for user setting of LED lighting control to the machine wired or wirelessly according to the user's request through a pre-installed user setting related application for LED lighting control. It can be transmitted to the vision control device.

Preferably, the machine vision control device emits light by continuously turning on each LED light emitting element provided in each LED light emitting channel of the first and second line lighting devices and the third and fourth line lighting devices. In order to protect the device from damage, a constant operation of the first and second line lighting devices and the third and fourth line lighting devices for each LED light emitting channel is performed at a predetermined operation standard time or less when the high-speed switching operation signal is changed. Each LED light emitting channel of the first and second line lighting devices and the third and fourth line lighting devices may be controlled to perform a high-speed switching operation only during the reference time.

Preferably, the machine vision control device detects the upper object to be inspected by using a preset machine vision algorithm based on the video frames on both sides of the upper portion of the divided corresponding train, and then detects the object to be inspected based on a preset artificial intelligence (AI). It is possible to perform abnormality detection of the upper subject through the machine learning method of.

Preferably, the preset machine vision algorithm is at least one of pattern matching, point/line/plane fitting, edge, color, and location algorithms. It can be done.

Preferably, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention includes images of both upper sides of the corresponding train obtained from the first and second line scan camera devices, A storage device for storing images of both sides of the lower portion of the train obtained from the third and fourth linescan camera devices and abnormal detection result information of the upper and lower objects to be inspected may be further included.

Preferably, the machine vision control device includes images of both upper sides of the corresponding train obtained from the first and second line scan camera devices, images of both lower sides of the corresponding train obtained from the third and fourth line scan camera devices, And the abnormal detection result information of the upper object to be inspected and the corresponding lower object to be inspected can be controlled to be stored in the storage device by making a database (DB) for each of the upper and lower sides of the train.

Preferably, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention includes images of both upper sides of the corresponding train obtained from the first and second line scan camera devices, A display device for displaying images of both sides of the lower portion of the train obtained from the third and fourth linescan camera devices and abnormal detection result information of the upper and lower objects to be inspected on a display screen may be further included.

Preferably, the machine vision control device includes images of both upper sides of the corresponding train obtained from the first and second line scan camera devices, images of both lower sides of the corresponding train obtained from the third and fourth line scan camera devices, And it is possible to control the operation of the display device to display the abnormality detection result information of the upper object to be inspected and the corresponding lower object to be inspected on the display screen.

Preferably, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention includes images of both upper sides of the corresponding train obtained from the first and second line scan camera devices, A communication device for transmitting images of both sides of the lower portion of the train obtained from the third and fourth linescan camera devices and abnormal detection result information of the upper and lower objects to be inspected in a wired or wireless communication method may be further included. there is.

Preferably, the machine vision control device includes images of both upper sides of the corresponding train obtained from the first and second line scan camera devices, images of both lower sides of the corresponding train obtained from the third and fourth line scan camera devices, And it is possible to control the operation of the communication device to transmit the abnormality detection result information of the upper object to be inspected and the corresponding lower object to be inspected to an external terminal or server in a wired or wireless communication method.

Preferably, the external terminal or server includes images of both upper sides of the corresponding train obtained from the first and second line scan camera devices transmitted from the communication device through a pre-installed railroad track facility detection related application, and the third And images of both sides of the lower part of the train obtained from the fourth linescan camera device and abnormal detection result information of the upper part and the lower part of the train can be retrieved or displayed on a display screen.

Preferably, the machine vision control device includes images of both upper sides of the train obtained from the first and second line scan camera devices, or images of both sides of the lower portion of the train obtained from the third and fourth line scan camera devices. received and converts the images on both sides of the upper or lower sides of the train into a panoramic image using a preset image stitching technique, and then displays the converted panoramic image on the display screen You can control what is displayed.

Preferably, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention is mounted on the inside/outside of the corresponding train, and is based on a preset standard on the track of the railway track facility. an RFID tag reader device reading corresponding reference position information from an RFID tag installed at each position; a mileage measuring device installed inside/outside the train and measuring mileage information of the train; and a location detection device mounted on the inside/outside of the corresponding train and continuously and automatically detecting the current location information of the corresponding train by using the reference location information of the RFID tag reader device and the mileage measurement information of the mileage measuring device. may be further included.

Preferably, the machine vision control device receives current location information of the corresponding train detected from the location detection device, and based on this, each image of the upper side of the corresponding train obtained from the first and second line scan camera devices is provided. The current location information of the corresponding train may be mapped for each frame or each frame of images of both lower sides of the corresponding train obtained from the third and fourth line scan camera devices and stored in a separate storage device.

Preferably, the machine vision control device generates a camera shutter trigger signal for capturing an image by decomposing the mileage measurement signal of the corresponding train measured by the mileage measuring device into a preset measurable cycle, and the generated camera Operations of the first and second line scan camera devices and the third and fourth line scan camera devices may be controlled based on a shutter trigger signal.

Preferably, the upper detailed parts of the railway track facility are a pantograph for contacting the catenary of the train and collecting electric energy from the catenary, a jog wire for supporting the load of the catenary and maintaining the temporary height ( Messenger wire) and dropper, a moving bracket to support and buffer the force received by the corresponding tram line from the progress of the train, and a steady to share the load of the corresponding tram line and buffer the vertical vibration and push-up shock. It may be made of at least one of a steady arm, a T-bar for fixing a corresponding catenary to a ceiling surface of a tunnel or underground space, and an insulation insulator for electrical insulation.

According to the real-time high-speed detection system of railway track facilities using the artificial intelligence-based machine vision algorithm of the present invention as described above, light in the form of lines on both sides of the upper and / or lower parts of the train moving on the track of the railway track facilities In the state of irradiation, after acquiring an image with a two-dimensional image by scanning the images on both sides of the upper and / or lower part of the corresponding train line by line, and then having a two-dimensional image of the upper and / or lower part of the inspection object of the railway track facility Using the entire image, AI-based machine vision algorithm is used to automatically detect and detect abnormalities in the upper and/or lower part of the railway track facility, thereby automatically detecting the upper and/or lower part of the railway track facility. There is an advantage in that the stability of the train can be effectively secured by quickly and accurately performing detection and abnormality detection on the lower object to be inspected.

1 is an overall block diagram illustrating a system for real-time high-speed detection of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention.
2 is a detailed block configuration diagram for explaining a machine vision control device applied to an embodiment of the present invention.
3 is a perspective view for explaining the disposition of first to fourth line scan camera devices and first to fourth line lighting devices applied to an embodiment of the present invention.
FIG. 4 is a diagram showing, as an example, images of both upper sides of a train obtained through first and second line scan camera devices applied to an embodiment of the present invention and image frames of both upper sides of a train divided through a machine vision control device. .
5 is a diagram showing edge components of upper detailed parts of a railway track facility extracted through a machine vision control device applied to an embodiment of the present invention as an example.
6 is an example of a geometric relationship between upper detailed parts of a railway track facility based on edge components of the upper detailed parts of a railway track facility extracted through a machine vision control device applied to an embodiment of the present invention. is the drawing shown.
7 is a view showing, for example, abnormality detection results for upper and lower inspection objects of a railway track facility detected through a machine vision control device applied to an embodiment of the present invention.
8 is a view showing, as an example, images of both sides of the lower part of a train obtained through third and fourth linescan camera devices applied to an embodiment of the present invention and a lower part of a railway track facility inspected through a machine vision control device. am.
9 is a diagram illustrating a detection process for an object to be inspected under a railway track facility through a machine vision control device applied to an embodiment of the present invention as an example.
10 is a detailed block configuration diagram for explaining an external terminal and/or server applied to an embodiment of the present invention.

The above objects, features and advantages will be described later in detail with reference to the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs will be able to easily implement the technical spirit of the present invention. In describing the present invention, if it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

The terms used in the present invention have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention of a person skilled in the art or precedent, the emergence of new technologies, and the like. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, not simply the name of the term.

When it is said that a certain part "includes" a certain component throughout the specification, it means that it may further include other components without excluding other components unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention exemplified below may be modified in many different forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to more completely explain the present invention to those skilled in the art.

Combinations of each block of the accompanying block diagram and each step of the flowchart may be performed by computer program instructions (execution engine), and these computer program instructions are executed by a processor of a general-purpose computer, special-purpose computer, or other programmable data processing equipment. Since it can be mounted, the instructions executed through the processor of a computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or each step of the flowchart. These computer program instructions may also be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular way, such that the computer usable or computer readable memory The instructions stored in are also capable of producing an article of manufacture containing instruction means for performing the functions described in each block of the block diagram or each step of the flow chart.

In addition, since the computer program instructions can be loaded on a computer or other programmable data processing equipment, a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to generate a computer or other programmable data processing equipment. It is also possible that instructions performing possible data processing equipment provide steps for executing the functions described in each block of the block diagram and each step of the flow chart.

In addition, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical functions, and in some alternative embodiments may refer to blocks or steps. It should be noted that it is also possible for functions to occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially concurrently, or the blocks or steps may be performed in the reverse order of their corresponding functions, if necessary.

1 is an overall block configuration diagram for explaining a real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention, and FIG. 2 is a machine applied to an embodiment of the present invention. It is a specific block configuration diagram for explaining a vision control device, and FIG. 3 is a perspective view for explaining an arrangement state of first to fourth line scan camera devices and first to fourth line lighting devices applied to an embodiment of the present invention. 4 shows images on both sides of the upper portion of the train obtained through the first and second line scan camera devices applied to an embodiment of the present invention and image frames on both sides of the upper portion of the train divided through the machine vision control device as an example. FIG. 5 is a diagram showing, as an example, edge components of upper detailed parts of a railway track facility extracted through a machine vision control device applied to an embodiment of the present invention, and FIG. 6 is an example of an embodiment of the present invention. Based on the edge component of the upper detailed parts of the railway track facility extracted through the machine vision control device applied to, it is a diagram showing the geometric relation of the upper detailed parts of the railway track facility as an example, and FIG. 7 is an example of the present invention. 8 is a view showing, as an example, the abnormality detection results for the upper and lower parts of the railway track facility detected through the machine vision control device applied to an embodiment of the present invention, and FIG. 4 is a diagram showing images of both sides of the lower part of the train acquired through the linescan camera device and the lower part of the railway track facility detected through the machine vision control device as an example, and FIG. 9 is a machine vision applied to an embodiment of the present invention. It is a diagram showing the detection process of the lower part of the railway track facility through a control device as an example, and FIG. 10 is a specific block configuration diagram for explaining an external terminal and / or server applied to an embodiment of the present invention.

1 to 10, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention largely includes first and second line scan camera devices (100a and 100b) ), first and second line lighting devices 200a and 200b, a machine vision control device 300, and a power supply device 400. In addition, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention includes third and fourth line scan camera devices 500a and 500b, third and fourth line lighting Devices 600a and 600b, storage device 700, display device 750, communication device 800, RFID tag reader device 850, mileage measurement device 900, position detection device 950, external A terminal and/or a server 20 may be further included. On the other hand, the components shown in FIGS. 1 to 10 are not essential, so the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention has more components than those shown in FIGS. or may have fewer components.

Hereinafter, components of a real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention will be described in detail.

The first and second line scan camera devices 100a and 100b are respectively installed on both sides of the upper part of the train 1 moving on the track of the railroad track facility, and are respectively installed on both sides of the upper part of the train 1 in units of lines. It performs a function of obtaining an image having a two-dimensional image by scanning an image (that is, an image of upper detailed parts of a railroad track facility).

That is, the first and second line scan camera devices 100a and 100b are connected to the upper detailed parts and/or the corresponding railway track facility in a state in which light is irradiated through the first and second line lighting devices 200a and 200b. The imaging surface of the upper object to be inspected is photographed, and the captured upper detailed parts of the railway track facility and/or the line scan image of the upper object to be inspected are transmitted to the machine vision controller 300 .

The first and second line scan camera devices 100a and 100b do not acquire images in units of areas, and a single line, that is, a one-dimensional linear CMOS (Complementary Metal Oxide Semiconductor) or CCD ( It is preferable to implement a camera of a method of obtaining a two-dimensional image by scanning a line unit through a charge coupled device (Charge Coupled Device) image sensor or the like.

On the other hand, as shown in FIGS. 4 to 7, the upper inspection object of the railway track facility is at least one upper detail part among the upper detailed parts provided on the upper side of the railway track facility, for example, the train (1) A pantograph for collecting electrical energy from the corresponding catenary in contact with the catenary of the catenary, and/or supplying power to the train (1) through the pantograph, and placing the train (1) on the ceiling surface of the tunnel or underground space It is preferably made of at least one of the T-bars, which are fixing means used in the device for fixing the tram line, but is not limited thereto, and at least one of the other upper detailed parts provided on the upper side of the railway track facility may be done

At this time, other upper detailed parts provided on the upper side of the railway track facility are not shown in the drawing, but, for example, messenger wire and dropper for supporting the load of the catenary of the train 1 and maintaining the temporary height ), a moving bracket for supporting and buffering the force received by the corresponding catenary from the progress of the train (1), a steady arm for sharing the load of the catenary and buffering the vertical vibration and push-up shock , And / or may be made of at least one of an insulation insulator for electrical insulation.

The first and second line lighting devices 200a and 200b are installed on both sides of the upper part of the train 1, and emit light in the form of lines of the same optical axis as the first and second line scan camera devices 100a and 100b. perform an investigative function.

Although not shown in the drawings, the first and second line lighting devices 200a and 200b preferably include a plurality of LED light emitting channels connected in parallel to each other including at least one LED light emitting element.

That is, the first and second line lighting devices 200a and 200b may include a plurality of LED light emitting channels connected in parallel with each other, and each of the plurality of LED light emitting channels may include at least one LED light emitting element. .

In addition, when there are two or more LED light emitting elements provided in each of the LED light emitting channels, each LED light emitting element may be connected in series and/or parallel in an array form.

The first and second line lighting devices 200a and 200b configured as described above may operate each LED light emitting element of each LED light emitting channel by receiving the operating voltage output from the machine vision control device 300 .

On the other hand, the first and second line lighting devices 200a and 200b have a plurality of LED (Light Emitting Diode) light emitting elements arrayed so as to radiate surface light to the upper detailed parts and/or the upper inspection object of the railway track facility. It is preferably made in a form, but is not limited thereto, and may include, for example, at least one of a fluorescent lamp, an incandescent lamp, a halogen lamp, a mercury lamp, a neon tube lamp, a sodium lamp, a metal halide lamp, or an EL (Electro-Luminescent) lamp.

In addition, the first and second line lighting devices 200a and 200b may include, for example, a liquid crystal display (LCD), a light emitting diode display (LED), and a thin film transistor liquid crystal display (Thin Film Transistor-Liquid Crystal Display). Display, TFT LCD), Organic Light Emitting Diode (OLED), Flexible Display, Plasma Display Panel (PDP), Surface Alternate Lighting (ALiS), Digital Light Source Processing (DLP) , silicon liquid crystal (LCoS), surface conduction electron emission display (SED), field emission display (FED), laser TV (quantum dot laser, liquid crystal laser), optoelectric liquid display (FLD), interferometric modulator display (iMoD) , a thick film dielectric electric (TDEL), a quantum dot display (QD-LED), a telescopic pixel display (TPD), an organic light emitting transistor (OLET), and / or a laser fluorescent display (LPD). It is not limited thereto, and may include anything as long as it is possible to irradiate surface light to the upper detailed parts and/or the upper part of the railway track facility.

The machine vision control device 300 is a device in charge of overall control of a real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention, and in particular, a preset camera shutter trigger signal. Synchronized with the first and second line scan camera devices 100a and 100b, the camera shutters of the first and second line scan camera devices 100a and 100b are opened and the first and second line scan cameras 200a and 200b emit light at the same time. It performs a function of controlling the operation of the devices 100a and 100b and the first and second line lighting devices 200a and 200b.

In addition, the machine vision control device 300 receives the images of both sides of the upper part of the train 1 obtained from the first and second line scan camera devices 100a and 100b to detect the object to be inspected from the upper part of the corresponding railway track facility. The image frames on both sides of the upper portion of the train 1 are divided through a predetermined frame division technique, and in order to determine the position of the upper object to be inspected in the image frames on both sides of the upper portion of the divided train 1, the detailed upper part of the corresponding railway track facility After extracting the edge component for the railway track facility, the upper detailed part of the corresponding railway track facility through a predetermined artificial intelligence (AI)-based machine learning method based on the edge component of the upper detail part of the extracted railway track facility It performs the function of detecting a corresponding upper object to be inspected by repeatedly learning the geometric relation of the upper object to be inspected and detecting abnormality of the corresponding upper object to be inspected.

In addition, the machine vision control device 300 detects the upper object to be inspected by using a preset machine vision algorithm based on the image frames on both sides of the upper portion of the divided train 1, and then detects the corresponding upper object under test. ) based machine learning method, it is possible to perform abnormality detection of the upper part of the subject.

In this case, the preset machine vision algorithm is, for example, at least one of pattern matching, point/line/plane fitting, edge, color, and/or location algorithms. Preferably it is made of an algorithm.

In addition, the machine vision control device 300 is synchronized with a trigger signal of a preset camera shutter, and the camera shutters of the first and second line scan camera devices 100a and 100b are opened simultaneously with the first and second line lighting devices ( The operation of the first and second line scan camera devices 100a and 100b and the first and second line lighting devices 200a and 200b is controlled so that the lights of 200a and 200b) are emitted, and the third and fourth lines are also controlled. When the camera shutters of the scan camera devices 500a and 500b are opened, the third and fourth line scan camera devices 500a and 500b and the third and fourth line illumination devices 500a and 500b emit light. and a function of controlling the operation of the fourth line lighting devices 600a and 600b.

In addition, the machine vision control device 300 receives images of both sides of the lower part of the train 1 obtained from the third and fourth line scan camera devices 500a and 500b, and receives a preset artificial intelligence (AI)-based machine learning method. Through repeated learning of the relative position and appearance of the lower part of the railway track facility, the lower part of the subject can be detected and the abnormality of the lower part of the subject can be detected.

In addition, the machine vision control device 300 uses a predetermined machine vision algorithm based on the images of both sides of the lower part of the train 1 obtained from the third and fourth line scan camera devices 500a and 500b, and uses a corresponding lower part to be inspected. After performing the detection of , it is possible to perform abnormality detection of the corresponding lower object to be inspected through a predetermined artificial intelligence (AI)-based machine learning method.

In this case, the preset machine vision algorithm is, for example, at least one of pattern matching, point/line/plane fitting, edge, color, and/or location algorithms. Preferably it is made of an algorithm.

On the other hand, as shown in FIGS. 8 and 9, the lower inspection object of the railway track facility is at least one lower detail part among the lower detailed parts of the railway track facility, and a plurality of lower parts disposed at regular intervals on the ground. At least one of the sleepers, a pair of railroad rails fixed and installed side by side in a direction orthogonal to the upper ends of both sides of the plurality of sleepers, and/or a pan control for fixing the pair of railroad rails on the plurality of sleepers. Preferably it consists of one lower detail.

If the lower object to be inspected is a plurality of sleepers, the machine vision control device 300 may perform sleeper abnormality detection of breakage of the sleeper. When the lower object to be inspected is a pair of railroad rails, the machine vision controller 300 performs at least one of rail waviness wear, tongue rail wear, and/or rail head damage Rail abnormality detection can be performed. When the lower object to be inspected is a pan control, the machine vision control device 300 may detect an abnormality in pan control removal.

In addition, the machine vision controller 300 may perform the first and second camera shutter exposure times of the first and second line scan camera devices 100a and 100b and the third and fourth line scan camera devices 500a and 500b. The output intensity of the LED light emitting elements provided in the line lighting devices 200a and 200b and the LED light emitting elements provided in the third and fourth line lighting devices 600a and 600b is a pulse signal type greater than the rated continuous maximum value. The first and second line lighting devices 200a and 200b and the third and second line lighting devices 200a and 200b and the third and second line lighting devices 200a and 200b are applied with a predetermined overdrive constant current value, and a constant current overdrive control method using a pulse control is applied to obtain high luminance brightness. It can perform a function of controlling the operation of each of the 4-line lighting devices 600a and 600b.

At this time, the over-driving constant current value is the LED light emitting element provided in the first and second line lighting devices 200a and 200b and the third and fourth line lighting devices 600a and 600b provided between each pulse signal. It is preferable to set in consideration of the cooling time of the LED light emitting element.

In addition, the machine vision control device 300 controls the first and second line lighting devices 200a and 200b and the third and fourth line lighting devices 600a and 600b according to a user's specific input signal output according to the user's request. ), setting the output level target value of each LED light-emitting channel provided in each, setting the output channel selection and order of each LED light-emitting channel for high-speed switching control, and / or clock for generating an operation signal for high-speed switching (Clock ) generation, and based on the output level target value of each LED light emitting channel set above, in order to adjust the brightness of each LED light emitting element provided in each LED light emitting channel, an output voltage target value of each LED light emitting channel is generated and output. The first and second line lighting devices 200a and 200b according to the output voltage target value of each LED light-emitting channel output in synchronization with the output channel selection and sequence setting information and clock information of the set LED light-emitting channels. and a function of controlling high-speed switching of each LED light emitting channel provided in the third and fourth line lighting devices 600a and 600b, respectively.

In addition, the machine vision control device 300 may perform a function of outputting a specific input signal of a corresponding user according to a user's request based on a graphical user interface (GUI).

In addition, the machine vision controller 300 may perform high-speed control by using a hardware programming-based Field Programmable Gate Array (FPGA) for a high-speed switching function.

In addition, the machine vision controller 300 includes first and second line scan cameras respectively installed near the first and second line lighting devices 200a and 200b and the third and fourth line lighting devices 600a and 600b. Devices 100a and 100b and the third and fourth line scan camera devices 500a and 500b detect an idle period of a photographing operation, and detect the first and second line scan camera devices 100a and 100b and the third and fourth line scan camera devices 500a and 500b. When the photographing operation signal of the fourth line scan camera devices 500a and 500b is not input for a predetermined photographing operation reference time or longer, it is determined as an idle state, and the photographing operation state is maintained in the first state after the determined idle state. It can perform an automatic reset function that removes and initializes to the initial input state.

In addition, the machine vision control device 300 sets the set value for the target output level of each LED light emitting channel, the set value of the output channel selection and order of each LED light emitting channel, and/or the setting for clock generation. It may perform a function of controlling to store at least one set value among the values in a separate storage module (not shown) or the storage device 700 .

Meanwhile, the user's specific input signal output according to the user's request may be provided through an external terminal and/or the server 20 .

In addition, the machine vision control device 300 includes the first and second line lighting devices 200a and 200b and the third and fourth line lighting devices 600a and 600b of each LED light emitting device provided in each LED light emitting channel. In order to protect each LED light emitting element from being damaged due to continuous turning on, for each LED light emitting channel of the first and second line lighting devices 200a and 200b and the third and fourth line lighting devices 600a and 600b When the high-speed switching operation signal is changed, each LED of the first and second line lighting devices 200a and 200b and the third and fourth line lighting devices 600a and 600b emits light only for a predetermined operation standard time less than the preset operation standard time. It can perform a function of controlling a channel to perform a high-speed switching operation.

In addition, the machine vision control device 300 includes images of both upper sides of the train 1 obtained from the first and second line scan camera devices 100a and 100b, and the third and fourth line scan camera devices 500a and 500b. At least one of the lower side images of the train 1 obtained from the image and/or the abnormal detection result information of the upper and lower inspected objects for each of the upper and/or lower sides of the train 1, respectively It can perform a function of controlling to be stored in a database (DB) and stored in the storage device 700 .

In addition, the machine vision control device 300 includes images of both upper sides of the train 1 obtained from the first and second line scan camera devices 100a and 100b, and the third and fourth line scan camera devices 500a and 500b. Controlling the operation of the display device 750 to display at least one of the images on both sides of the lower part of the train 1 obtained from and/or information on the abnormality detection result of the upper and lower inspected objects on the display screen. function can be performed.

In addition, the machine vision control device 300 includes images of both upper sides of the train 1 obtained from the first and second line scan camera devices 100a and 100b, and the third and fourth line scan camera devices 500a and 500b. At least one of the images of both sides of the lower part of the train 1 obtained from the image and/or the abnormal detection result information of the upper and lower parts of the inspected object is transmitted through a wired and/or wireless communication method to an external terminal and/or server It can perform a function of controlling the operation of the communication device 800 to be transmitted to (20).

In addition, the machine vision controller 300 may use the images on both sides of the upper portion of the train 1 obtained from the first and second line scan camera devices 100a and 100b, and/or the third and fourth line scan camera devices 500a. and 500b), the upper and/or lower side images of the train 1 are converted into a panoramic image (Panoramic Image) using a predetermined image stitching technique. Image), a function of controlling the converted panoramic image to be displayed on the display screen may be performed.

In addition, the machine vision control device 300 decomposes the mileage measurement signal of the train 1 measured by the mileage measuring device 900 into a preset measurable cycle to generate a camera shutter trigger signal for image capture, A function of controlling the operation of the first and second line scan camera devices 100a and 100b and the third and fourth line scan camera devices 500a and 500b may be performed based on the generated trigger signal of the camera shutter. there is.

In addition, the machine vision control device 300 receives information on the current location of the train 1 detected from the location detection device 950 and based on this received information obtained from the first and second line scan camera devices 100a and 100b. of the train 1 for each frame of images of both upper sides of the train 1 and/or for each frame of images of the lower sides of the train 1 obtained from the third and fourth line scan camera devices 500a and 500b. A function of mapping the current location information and controlling it to be stored in a separate storage device 700 may be performed.

As shown in FIG. 2, the machine vision control device 300 largely includes an image segmentation module 310, an edge extraction module 320, a first artificial intelligence learning module 330, and a second artificial intelligence learning module 340. ), an upper object abnormality detection module 350, and a machine vision control module 360. In addition, the machine vision control device 300 applied to an embodiment of the present invention includes a third artificial intelligence learning module 370, a fourth artificial intelligence learning module 380, and/or a lower subject abnormality detection module 390 etc. may be further included. Meanwhile, since the components shown in FIG. 2 are not essential, the machine vision control device 300 applied to an embodiment of the present invention may have more or fewer components.

Hereinafter, components of the machine vision control device 300 applied to an embodiment of the present invention will be described in detail.

The image segmentation module 310 receives the images on both sides of the upper part of the train 1 obtained from the first and second line scan camera devices 100a and 100b and uses a predetermined frame segmentation technique for detecting the upper part of the object under inspection. It performs a function of dividing image frames on both sides of the upper part of the train 1 through

At this time, the preset frame division technique is a common image processing technique, which is a process of dividing a digital image into a plurality of pixel sets, and is particularly a technique used to find objects and boundaries (eg, lines, curves, etc.) in an image. The image segmentation or frame segmentation technique is a commonly known image processing technique, and a detailed description thereof will be omitted.

The edge extraction module 320 extracts edge components for the upper detailed parts of the railway track facility in order to determine the position of the upper object to be inspected in the image frames on both sides of the upper part of the train 1 divided from the image segmentation module 310 perform the function of

The first artificial intelligence learning module 330 is based on the edge component of the upper detailed parts of the railway track facility extracted from the edge extraction module 320 through a predetermined artificial intelligence (AI)-based machine learning method. It performs a function of constructing a first artificial intelligence learning model for detecting the upper object to be inspected by repeatedly learning the geometric relation of the upper detailed parts of the railroad track facility.

The second artificial intelligence learning module 340 performs the upper part of the subject for each normal and/or abnormal characteristic item for the corresponding upper part of the subject detected using the first artificial intelligence learning model built from the first artificial intelligence learning module 330. After inferring and analyzing the appearance and positional characteristics of the corpse, classifying the abnormal occurrence type for the upper part of the subject based on the inference and analysis of the shape and positional characteristic information of the upper part of the subject, the predetermined artificial intelligence (AI) )-based machine learning method, it performs the function of constructing a second artificial intelligence learning model for detecting anomalies of the upper subject by proceeding with repetitive learning suitable for the type of anomaly classified for the upper subject.

The upper subject object abnormality detection module 350 uses the second artificial intelligence learning model built from the second artificial intelligence learning module 340 for each characteristic item of the abnormal occurrence type of the upper object to be inspected. Performs a function of inferring and analyzing the characteristics of the occurrence type, and automatically detecting whether and the type of abnormality have occurred in the upper part of the subject based on the inferred and analyzed characteristic information on the type of abnormality of the upper part of the subject. do.

In addition, the machine vision control module 360 is in charge of overall control of the machine vision control device 300 applied to an embodiment of the present invention, and performs all control functions performed by the machine vision control device 300 described above. In particular, the camera shutters of the first and second line scan camera devices 100a and 100b are opened in synchronization with the trigger signal of the preset camera shutter, and the first and second line lighting devices 200a and 200b It performs a function of controlling the operation of the first and second line scan camera devices 100a and 100b and the first and second line lighting devices 200a and 200b to emit light.

In addition, the machine vision control module 360 synchronizes with the trigger signal of the preset camera shutter to open the camera shutters of the third and fourth line scan camera devices 500a and 500b, and at the same time the third and fourth line lighting devices ( A function of controlling operations of the third and fourth line scan camera devices 500a and 500b and the third and fourth line lighting devices 600a and 600b is performed so that the lights 600a and 600b) emit light.

Various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). , processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, such embodiments may be implemented by machine vision control module 360 .

According to software implementation, embodiments such as procedures or functions may be implemented together with a separate software module to perform at least one function or operation. The software code may be implemented by a software application written in any suitable programming language. In addition, the software code may be stored in a separate storage module (not shown) or the storage device 700 and executed by the machine vision control module 360 .

On the other hand, as described above, that is, the image segmentation module 310, the edge extraction module 320, the first artificial intelligence learning module 330, the second artificial intelligence learning module 340, and the upper part of the subject Although the detection module 350 is separately separated and implemented so that each module performs the function, it is not limited thereto, and the machine vision control module 360 may be implemented so that all of the above-described functions are performed.

In addition, the third artificial intelligence learning module 370 receives images of both sides of the lower part of the train 1 obtained from the third and fourth line scan camera devices 500a and 500b, and the AI-based machine is preset. The function of constructing a third artificial intelligence learning model for detecting the corresponding lower target object is performed by repeatedly learning the appearance, relative position and/or geometric relation of the lower target object through the learning method.

For example, when the corresponding lower object to be inspected is a rail, the machine vision control module 360 and/or the third artificial intelligence learning module 370 detects an edge in the images on both sides of the lower part of the train 1 and then detects a straight line. A Hough Transform for detection is applied, and since there is a linear component outside the rail, the gradient value after applying the Hough Transform is learned using SVM (Support Vector Machine), and the final gradient value is selected and the rail area is detected. can do. Meanwhile, the rail may be detected through a relative position learning algorithm based on the detection area of the pan control and the sleeper (see FIG. 9).

On the other hand, when the corresponding lower object to be inspected is a sleeper, the detected rail area can be used for localization of the sleeper, and the detected rail area is centered on the SVM (Support Vector Machine), and the rail is perpendicular to the rail. It is possible to provide an initial value of localization of sleepers at regular intervals by learning the slope component and detecting edges having a slope perpendicular to the rail.

On the other hand, if the lower inspection object is a pantrol, an initial value of pantrol localization can be provided to the relative left/right position using the positioning result of the sleeper.

The fourth artificial intelligence learning module 380 performs a corresponding subtest for each normal and/or abnormal characteristic item for the corresponding subtest object detected using the third artificial intelligence learning model built from the third artificial intelligence learning module 370. After inferring and analyzing the appearance and positional characteristics of the corpse, classifying the type of abnormal occurrence for the lower subject based on the appearance and positional characteristic information of the lower subject subject that has been inferred and analyzed, the predetermined artificial intelligence (AI) )-based machine learning method, the function of constructing a fourth artificial intelligence learning model for abnormality detection of the corresponding lower subject is performed by repeating learning according to the type of anomaly classified for the corresponding lower subject.

In addition, the lower subject abnormality detection module 390 uses the fourth artificial intelligence learning model built from the fourth artificial intelligence learning module 380 for the corresponding lower subject for each characteristic item of the abnormal occurrence type for the corresponding lower subject. A function of inferring and analyzing the characteristics of the type of abnormal occurrence of the lower part of the test subject based on the characteristic information on the type of abnormality of the corresponding lower part of the subject that has been inferred and analyzed and automatically detecting whether or not the type of abnormality has occurred in the lower part of the subject. Do it.

At this time, the artificial intelligence (AI)-based machine learning method applied to the first and second artificial intelligence learning modules (330 and 340) and the third and fourth artificial intelligence learning modules (370 and 380) is, for example, a neural network (Neural Network). ), Support Vector Machine (SVM), Multi Layer Perception (MLP), and/or Deep Learning.

On the other hand, as described above, that is, the third artificial intelligence learning module 370, the fourth artificial intelligence learning module 380, and/or the lower subject abnormality detection module 390 are separately separated and each module Although implemented to perform the function, it is not limited thereto, and the machine vision control module 360 may be implemented to perform all of the functions of the aforementioned modules.

In addition, the power supply device 400 includes the aforementioned devices, that is, the first and second line scan camera devices 100a and 100b, the first and second line lighting devices 200a and 200b, and the machine vision control device ( 300), third and fourth line scan camera devices 500a and 500b, third and fourth line lighting devices 600a and 600b, a storage device 700, a display device 750, a communication device 800, It performs a function of supplying power necessary for the operation of the RFID tag reader device 850, the mileage measurement device 900, and/or the position detection device 950, and commercial AC (AC) for continuous power supply. ) It is preferable to implement a power source (eg, AC 220V or 380V, etc.) to be converted into direct current (DC) and/or alternating current (AC) power, but is not limited thereto, and may be implemented including a conventional portable battery. there is.

In addition, the power supply device 400 may include a power management unit (not shown) that protects components from external power shock and outputs a constant voltage. The power management unit may include an electro static damage (ESD) protector, a power detector, a rectifier, and a power circuit breaker.

Here, the ESD protector is configured to protect electric components from static electricity or sudden power shock. The power detector is configured to send a cut-off signal to the power circuit breaker when a voltage outside the allowable voltage range is introduced, and transmit a step-up or step-down signal to the rectifier according to a voltage change within the allowable voltage range. The rectifier is configured to perform a step-up or step-down rectification operation according to the signal of the power detector so that a constant voltage is supplied by minimizing fluctuations in the input voltage. The power breaker is configured to cut off power supplied from a battery according to a cutoff signal transmitted from the power detector.

Additionally, the third and fourth line scan camera devices 500a and 500b are respectively installed on both sides of the lower portion of the train 1, and images of both sides of the lower portion of the train 1 line by line (ie, lower details of the railroad track facility) It performs a function of acquiring an image with a 2D image by scanning images of parts).

In addition, since the third and fourth line scan camera devices 500a and 500b have the same structure as the above-described first and second line scan camera devices 100a and 100b except for the installation position, the detailed description is as described above. Descriptions of the first and second line scan camera devices 100a and 100b will be referred to.

The third and fourth line lighting devices 600a and 600b are installed on both sides of the lower part of the train 1, and emit light in the form of lines on the same optical axis as the third and fourth line scan camera devices 500a and 500b. perform an investigative function.

In addition, since the third and fourth line lighting devices 600a and 600b have the same structure as the first and second line lighting devices 200a and 200b, except for the installation position, a detailed description will be given to the first and second line lighting devices 200a and 200b. And descriptions of the second line lighting devices 200a and 200b will be referred to.

The storage device 700 is connected to the machine vision control device 300, and the train 1 obtained from the first and second line scan camera devices 100a and 100b under the control of the machine vision control device 300 Stores images on both sides of the upper side of the image, images on both sides of the lower side of the train 1 obtained from the third and fourth linescan camera devices 500a and 500b, and/or abnormality detection result information of the corresponding upper object and the corresponding lower object to be inspected perform the function of

Since the storage device 700 needs to acquire and store high-resolution, high-capacity images that move at high speed, it is preferable to implement a solid state drive (SSD) type large-capacity memory device in consideration of the bandwidth of image data, but is not limited to this. For example, flash memory type, hard disk type, multimedia card micro type, card type memory (eg SD or XD memory, etc.), RAM ( Random Access Memory (RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , and/or an optical disk.

The display device 750 is connected to the machine vision controller 300, and the train 1 obtained from the first and second line scan camera devices 100a and 100b under the control of the machine vision controller 300 Displays images on both sides of the upper side of the image, images on both sides of the lower side of the train 1 obtained from the third and fourth linescan camera devices 500a and 500b, and/or abnormality detection result information of the corresponding upper object under test and the corresponding lower object under test Performs the function displayed on the screen.

Such a display device 750 includes, for example, a liquid crystal display (LCD), a light emitting diode display (LED), a thin film transistor-liquid crystal display (TFT LCD), an organic light emitting diode (Organic Light Emitting Diode, OLED), Flexible Display, Plasma Display Panel (PDP), Surface Alternate Lighting (ALiS), Digital Light Source Processing (DLP), Liquid Crystal in Silicon (LCoS), Surface Conduction type electron emission display (SED), field emission display (FED), laser TV (quantum dot laser, liquid crystal laser), optoelectric liquid display (FLD), interferometric modulator display (iMoD), thick film dielectric electric (TDEL), It may include at least one of a quantum dot display (QD-LED), a telescopic pixel display (TPD), an organic light emitting transistor (OLET), a laser fluorescence display (LPD), and a 3D display, but is not limited thereto. Any data may be included as long as it can display at least one of external video, text, and/or document data.

The communication device 800 is connected to the machine vision control device 300, and the train 1 acquired from the first and second line scan camera devices 100a and 100b under the control of the machine vision control device 300 The images on both sides of the upper part, the images on both sides of the lower part of the train 1 obtained from the third and fourth linescan camera devices 500a and 500b, and/or the abnormality detection result information of the upper part and the corresponding lower part of the object to be inspected are transferred to the communication network Through (10), it performs a function of transmitting to an external terminal and/or server 20 in a wired and/or wireless communication method.

At this time, the communication network 10 is a communication network that is a high-speed backbone network of a large-scale communication network capable of providing large-capacity, long-distance voice and data services, and provides Internet or high-speed multimedia services through Wi-Fi, WiGig, It may be a next-generation wireless communication network including Wibro (Wireless Broadband Internet), World Interoperability for Microwave Access (Wimax), and the like.

The Internet is a TCP / IP protocol and various services existing in the upper layer, that is, HTTP (Hyper Text Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), It means a worldwide open computer network structure that provides SNMP (Simple Network Management Protocol), NFS (Network File Service), NIS (Network Information Service), etc., and an external terminal and/or server 20 is a communication device 800 ) provides an environment that allows access to Meanwhile, the Internet may be wired or wireless Internet, or may be a core network integrated with a wired public network, a wireless mobile communication network, or a portable Internet.

If the communication network 10 is a mobile communication network, it may be a synchronous mobile communication network or an asynchronous mobile communication network. As an example of the asynchronous mobile communication network, a wideband code division multiple access (WCDMA) communication network may be used. In this case, although not shown in the drawings, the mobile communication network may include, for example, a Radio Network Controller (RNC) and the like. Meanwhile, although the WCDMA network is taken as an example, it may be a next-generation communication network such as a cellular-based 3G network, an LTE network, a 4G network, a 5G network, and other IP-based IP networks. This communication network 10 serves to mutually transfer signals and data between an external terminal and/or server 20 and the communication device 800 .

On the other hand, the communication network 10 uses, for example, Bluetooth, ZigBee, Beacon, UWB (Ultra Wideband), RFID (Radio Frequency Identification), and/or infrared (IR) communication. A communication method may also be used.

The RFID tag reader device 850 is mounted on the inside and/or outside of the train 1 and reads the reference position information from the RFID tag installed at each preset reference position on the track of the corresponding railway track facility. perform a function

The mileage measuring device 900 is mounted inside and/or outside the train 1 and performs a function of measuring mileage information of the train 1. It is preferable that the mileage measurement device 900 includes a normal taco meter for collecting movement signals of the train 1.

The location detection device 950 is mounted inside and/or outside the train 1, and uses the reference location information of the RFID tag reader device 850 and the mileage measurement information of the mileage measuring device 900 to train the train. It performs the function of continuously and automatically detecting the current location information of (1).

In addition, the external terminal and/or server 20 receives the user's specific input for user setting of LED lighting control according to the user's request through a pre-installed specific application, that is, a user setting related application for LED lighting control. It performs a function of transmitting a signal to the machine vision control device 300 wired and/or wirelessly through the communication network 10 .

In addition, the external terminal and/or server 20 transmits the first and second line scan camera devices 100a and 100b from the communication device 800 through a pre-installed specific application, that is, a railway track facility detection related application. Images on both sides of the upper part of the train 1 obtained from, images on both sides of the lower part of the train 1 obtained from the third and fourth linescan camera devices 500a and 500b, and/or the upper object and the corresponding lower object under test It is possible to provide a service so that abnormality detection result information of is searched and/or displayed on a display screen.

On the other hand, the external terminal and / or server 20 applied to an embodiment of the present invention is a smartphone (Smart Phone), smart pad (Smart Pad) or smart note (Smart Note) communicating through the wireless Internet or portable Internet It is preferably composed of at least one mobile terminal device, and in addition, a personal PC, a notebook PC, a Palm PC, a mobile game device (Mobile play-station), a DMB (Digital Multimedia Broadcasting) phone with a communication function, a tablet PC, It can comprehensively mean all wired and wireless home appliances/communication devices having a user interface for accessing the communication network 10 and the communication device 800, such as an iPad.

As shown in FIG. 10, such an external terminal and/or server 20 includes a wireless communication module 21, an audio/video (A/V) input module 22, a user input module 23, and a sensing module. (24), an output module 25, a storage module 26, an interface module 27, a terminal control module 28, and a power module 29. Meanwhile, since the components shown in FIG. 10 are not essential, the external terminal and/or server 20 may have more or fewer components.

Hereinafter, the components of the external terminal and/or server 20 will be described in detail.

The wireless communication module 21 may include one or more modules enabling wireless communication between an external terminal and/or server 20 and the communication device 800 . For example, the wireless communication module 21 may include a broadcast reception module 21a, a mobile communication module 21b, a wireless Internet module 21c, a short-distance communication module 21d, and a location information module 21e.

The broadcasting receiving module 21a broadcasts signals (eg, TV broadcasting signals, radio broadcasting signals, data broadcasting signals, etc.) and/or broadcasting from an external broadcasting management server through various broadcasting channels (eg, satellite channels, terrestrial channels, etc.) Receive relevant information.

The mobile communication module 21b transmits and receives a radio signal with at least one of a base station, a user terminal, and/or the server 20 on a mobile communication network. The wireless signal may include a voice call signal, a video call signal, or various types of data according to text/multimedia message transmission/reception.

The wireless Internet module 21c is a module for wireless Internet access, and may be built into or external to an external terminal and/or server 20 . As the wireless Internet technology, for example, WLAN (Wi-Fi), Wibro, Wimax, HSDPA, LTE, and the like may be used.

The short-distance communication module 21d is a module for short-distance communication, such as Bluetooth communication, ZigBee communication, UWB (Ultra Wideband) communication, RFID (Radio Frequency Identification) communication, or infrared (IR) communication. can be used

The location information module 21e is a module for checking or obtaining the location of an external terminal and/or server 20, and uses a Global Positioning System (GPS) to determine the current location of the external terminal and/or server 20. information can be obtained.

Meanwhile, under the control of the terminal control module 28, the communication device 800 uses a specific application program stored in the storage module 26 through the above-described wireless communication module 21 and/or wired communication module (not shown). ) and data transmission/reception.

The A/V input module 22 is a module for inputting an audio signal or a video signal, and may basically include a camera unit 22a and a microphone unit 22b. The camera unit 22a processes an image frame such as a still image or moving image obtained by an image sensor in a video call mode or a photographing mode. The microphone unit 22b receives an external sound signal through a microphone in a call mode, a recording mode, or a voice recognition mode, and processes it into electrical voice data.

The user input module 23 is a module that generates input data for controlling the operation of an external terminal and/or the server 20, and in particular, data management information displayed through the display unit 25a of the output module 25. It performs a function of inputting a selection signal for any one of them, and is, for example, a touch panel (static pressure/static) type input by a user's touch or a separate input device (eg, key pad dome switch, jog wheel, jog switch) etc.) can be input.

The sensing module 24 includes the open/closed state of the external terminal and/or server 20, the location of the external terminal and/or server 20, whether or not there is a user contact, a user's touch operation on a specific part, an external terminal and / or the external terminal and / or server 20 by detecting the current state of the external terminal and / or server 20, such as the direction of the server 20, acceleration / deceleration of the external terminal and / or server 20, etc. ) generates a sensing signal to control the operation of This sensing signal is transmitted to the terminal control module 28, and may be a basis for the terminal control module 28 to perform a specific function.

The output module 25 is a module for generating an output related to sight, hearing or touch, and basically includes a display unit 25a, a sound output unit 25b, an alarm unit 25c, and a haptic unit 25d. can

The display unit 25a is for displaying and outputting information processed by the external terminal and/or server 20. For example, when the external terminal and/or server 20 is in a call mode, the UI related to the call (User Interface) or GUI (Graphical User Interface) is displayed, and in the case of a video call mode or shooting mode, a captured and/or received image, UI, or GUI is displayed.

The audio output unit 25b may output audio data received from the wireless communication module 21 or stored in the storage module 26 in, for example, a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. .

The alarm unit 25c may output a signal for notifying the occurrence of an event of an external terminal and/or the server 20 . Examples of events occurring in the external terminal and/or server 20 include call signal reception, message reception, key signal input, and touch input.

The haptic unit 25d generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic unit 25d is vibration. The intensity and pattern of vibration generated by the haptic unit 25d can be controlled.

The storage module 26 may store programs for operation of the terminal control module 28 and may temporarily store input/output data (eg, phonebook, message, still image, video, etc.).

In addition, the storage module 26 may store data related to vibration and sound of various patterns output upon touch input on the touch screen, and specific applications (eg, user setting related applications for LED lighting control, railway track facility detection) related applications, etc.) can be stored.

In addition, the storage module 26 may store source data for forming specific application-related information, such that specific application-related data may be formed in the form of images and sounds, and data related to specific applications. The progress and results of creation can also be stored together.

The storage module 26 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM , a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium.

The interface module 27 serves as a passage between external terminals and/or all external devices connected to the server 20 . The interface module 27 receives data from an external device or receives power and delivers it to an external terminal and/or each component inside the server 20, or transmits data from an external terminal and/or server 20 to an external device. to be sent to the device.

The terminal control module 28 generally controls the overall operation of the external terminal and/or server 20, and performs related control and processing for, for example, voice calls, data communications, video calls, and execution of various applications. do.

That is, the terminal control module 28 controls the specific application program stored in the storage module 26 to be executed, requests the generation of specific application-related data through the execution of the specific application program, and requests the creation of specific application related data. It performs a function to control so that related data can be provided.

In addition, the terminal control module 28, in the process of generating specific application-related data desired by the user through execution of a specific application program, displays auxiliary elements including at least one of video, audio, or sound to the display unit 25a and other data. It performs a function of controlling output through at least one of output devices (eg, sound output unit 25b, alarm unit 25c, haptic unit 25d, etc.).

In addition, the terminal control module 28 may constantly monitor the charging current and charging voltage of the battery unit 29a and temporarily store the monitoring values in the storage module 26 . At this time, the storage module 26 preferably stores battery specification information (product code, rating, etc.) as well as battery charging state information such as the monitored charging current and charging voltage.

The power module 29 receives external power and internal power under the control of the terminal control module 28 and supplies power necessary for the operation of each component. The power module 29 supplies power from the built-in battery unit 29a to each component to operate, and the battery can be charged using a charging terminal (not shown).

Various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). , processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, such embodiments may be implemented by the terminal control module 28.

According to software implementation, embodiments such as procedures or functions may be implemented together with a separate software module to perform at least one function or operation. The software code may be implemented by a software application written in any suitable programming language. Also, the software code may be stored in the storage module 26 and executed by the terminal control module 28 .

If the external terminal and/or server 20 is composed of a smartphone, the smartphone freely downloads various application programs desired by the user, unlike a general mobile phone (aka feature phone). It is a phone based on an open operating system that can be used and deleted. It includes not only commonly used functions such as voice/video call and Internet data communication, but also all mobile phones equipped with mobile office functions or those without voice call function but Internet data communication. It is preferable to understand it as a communication device including all accessible Internet phones or tablet PCs.

As such, since the smart phone uses an open operating system, a user can arbitrarily install and manage various application programs unlike a mobile phone having a closed operating system.

Although the preferred embodiment of the real-time high-speed detection system for railway track facilities using the artificial intelligence-based machine vision algorithm according to the present invention has been described, the present invention is not limited thereto, and the claims and detailed description and It is possible to carry out various modifications within the scope of the accompanying drawings, which also belong to the present invention.

100a and 100b: first and second line scan camera devices;
200a and 200b: first and second line lighting devices,
300: machine vision control device,
400: power supply,
500a and 500b: third and fourth line scan camera devices,
600a and 600b: third and fourth line lighting devices,
700: storage device,
750: display device,
800: communication device,
850: RFID tag reader device,
900: mileage measuring device,
950: position detection device

Claims (15)

First and second line scan camera devices installed on both sides of the upper portion of a train moving on the track of a railway track facility, and acquiring images having a two-dimensional image by scanning images of both sides of the upper portion of the corresponding train line by line;
first and second line lighting devices respectively installed on both sides of the upper part of the corresponding train and irradiating light in the form of a line of the same optical axis as the first and second line scan camera devices; and
The camera shutters of the first and second line scan camera devices are opened in synchronization with a trigger signal of a preset camera shutter, and the first and second line scan cameras emit lights simultaneously. Controlling the operation of the device and the first and second line lighting devices, receiving images of both sides of the upper part of the train obtained from the first and second line scan camera devices, and detecting the upper part of the railway track facility to be inspected In order to divide the video frames on both sides of the upper part of the train through a predetermined frame division technique, in order to determine the position of the upper object to be inspected in the video frames on both sides of the upper part of the divided train, for the upper detailed parts of the railway track facility After extracting the edge components, based on the extracted edge components of the upper detailed parts of the railway track facility, through a predetermined artificial intelligence (AI)-based machine learning method, for the upper detailed parts of the railway track facility A real-time high-speed detection system for railway track facilities using artificial intelligence-based machine vision algorithms including a machine vision control device that repeatedly learns the geometric relationship to detect the upper target object and detects anomaly of the upper target target object .
According to claim 1,
The machine vision control device,
Dividing the image frames on both sides of the upper part of the train through a predetermined frame division technique to detect the upper part of the railway track facility by receiving the images on both sides of the upper part of the train obtained from the first and second line scan camera devices image segmentation module;
An edge extraction module for extracting edge components of upper detailed parts of the railway track facility in order to determine the position of the upper object to be inspected in the image frames on both sides of the upper part of the train divided by the image segmentation module;
Based on the edge components of the upper detailed parts of the railway track facility extracted from the edge extraction module, geometric association of the upper detailed parts of the railway track facility through a predetermined artificial intelligence (AI)-based machine learning method A first artificial intelligence learning module for constructing a first artificial intelligence learning model for detecting a corresponding upper subject by repeatedly learning the relationship;
Using the first artificial intelligence learning model built from the first artificial intelligence learning module, inferring and analyzing the appearance and location characteristics of the corresponding upper target object for each normal/abnormal feature item of the detected upper target object, and the inference And after classifying the type of anomaly with respect to the upper subject subject based on the appearance and location characteristic information of the analyzed upper subject object, the upper subject subject is classified through a predetermined artificial intelligence (AI) based machine learning method. A second artificial intelligence learning module for constructing a second artificial intelligence learning model for abnormality detection of the upper subject by conducting repetitive learning according to the classified abnormality type;
The second artificial intelligence learning model built from the second artificial intelligence learning module is used to infer and analyze the characteristics of the abnormality type of the upper object to be inspected for each characteristic item of the abnormality type of the upper object to be inspected, An upper test object abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the upper test object based on the inferred and analyzed characteristic information on the abnormal occurrence type of the upper test object; and
The camera shutters of the first and second line scan camera devices are opened in synchronization with a trigger signal of a preset camera shutter, and the first and second line scan cameras emit lights simultaneously. A real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that it comprises a machine vision control module for controlling the operation of the device and the first and second line lighting devices.
According to claim 1,
third and fourth line scan camera devices installed on both sides of the lower portion of the corresponding train and acquiring images having a two-dimensional image by scanning images of both sides of the lower portion of the corresponding train line by line; and
Third and fourth line lighting devices are installed on both sides of the lower part of the train and emit light in the form of a line of the same optical axis as the third and fourth line scan camera devices,
The machine vision control device is synchronized with a trigger signal of a preset camera shutter to open the camera shutters of the first and second line scan camera devices and simultaneously emit light from the first and second line lighting devices. Controls the operation of the first and second line scan camera devices and the first and second line lighting devices, and simultaneously lights the third and fourth line lighting devices when the camera shutters of the third and fourth line scan camera devices are opened. Control the operation of the third and fourth line scan camera devices and the third and fourth line lighting devices so that the light of the device emits light, and the upper sides of the train obtained from the first and second line scan camera devices In order to receive the image and detect the upper object to be inspected, the image frames on both sides of the upper portion of the corresponding train are divided through a predetermined frame division technique, and in order to determine the position of the corresponding upper object to be inspected in the image frames on both sides of the upper portion of the divided corresponding train After extracting the edge components of the upper detailed parts of the railway track facility, based on the edge components of the extracted upper detailed parts of the railway track facility, the corresponding artificial intelligence (AI)-based machine learning method is used. Iteratively learns the geometric relation of the upper detailed parts of the railroad track facility to detect the upper object to be inspected and to detect the abnormality of the upper object to be inspected, and the third and fourth line scan camera devices obtain Received images of both sides of the lower part of the train and repeatedly learned the relative position and appearance of the lower part of the railway track facility through a predetermined artificial intelligence (AI)-based machine learning method to detect the lower part of the target object. A real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that for detecting anomalies of the lower inspected object.
According to claim 3,
The machine vision control device,
an image segmentation module receiving the images of both upper sides of the corresponding train obtained from the first and second line scan camera devices and dividing image frames of both upper sides of the corresponding train through a predetermined frame division technique for detecting an upper object to be inspected;
An edge extraction module for extracting edge components of upper detailed parts of the railway track facility in order to determine the position of the upper object to be inspected in the image frames on both sides of the upper part of the train divided by the image segmentation module;
Based on the edge components of the upper detailed parts of the railway track facility extracted from the edge extraction module, geometric association of the upper detailed parts of the railway track facility through a predetermined artificial intelligence (AI)-based machine learning method A first artificial intelligence learning module for constructing a first artificial intelligence learning model for detecting a corresponding upper subject by repeatedly learning the relationship;
Using the first artificial intelligence learning model built from the first artificial intelligence learning module, inferring and analyzing the appearance and location characteristics of the corresponding upper target object for each normal/abnormal feature item of the detected upper target object, and the inference And after classifying the type of anomaly with respect to the upper subject subject based on the appearance and location characteristic information of the analyzed upper subject object, the upper subject subject is classified through a predetermined artificial intelligence (AI) based machine learning method. A second artificial intelligence learning module for constructing a second artificial intelligence learning model for abnormality detection of the upper subject by conducting repetitive learning according to the classified abnormality type;
The second artificial intelligence learning model built from the second artificial intelligence learning module is used to infer and analyze the characteristics of the abnormality type of the upper object to be inspected for each characteristic item of the abnormality type of the upper object to be inspected, An upper test object abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the upper test object based on the inferred and analyzed characteristic information on the abnormal occurrence type of the upper test object;
The images of both sides of the lower part of the train obtained from the third and fourth line scan camera devices are provided, and through a predetermined AI-based machine learning method, the appearance and relative position or geometric relation of the corresponding lower part to be inspected are used. A third artificial intelligence learning module for constructing a third artificial intelligence learning model for detection of a corresponding lower subject by repeatedly learning;
Using the third artificial intelligence learning model built from the third artificial intelligence learning module, inference and analysis of the appearance and location characteristics of the corresponding lower subject for each normal/abnormal characteristic item of the detected lower subject, and the inference And after classifying the type of anomaly with respect to the corresponding lower subject based on the appearance and location characteristic information of the analyzed lower subject, the corresponding lower subject is classified through a machine learning method based on artificial intelligence (AI). A fourth artificial intelligence learning module for constructing a fourth artificial intelligence learning model for abnormality detection of the lower subject by performing repetitive learning according to the classified abnormality type;
The fourth artificial intelligence learning model built from the fourth artificial intelligence learning module is used to infer and analyze the characteristics of the abnormal occurrence type of the lower inspected object for each characteristic item of the abnormal occurrence type for the corresponding lower inspected object, A lower subject abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the corresponding lower subject subject based on the inferred and analyzed characteristic information on the type of abnormal occurrence of the lower subject subject; and
The camera shutters of the first and second line scan camera devices are opened in synchronization with a trigger signal of a preset camera shutter, and the first and second line scan cameras emit lights simultaneously. In addition to controlling the operation of the device and the first and second line lighting devices, the camera shutters of the third and fourth line scan camera devices are opened and the lights of the third and fourth line lighting devices are emitted at the same time. It includes a machine vision control module for controlling operations of third and fourth line scan camera devices and the third and fourth line lighting devices,
Artificial intelligence (AI)-based machine learning methods applied to the first to fourth artificial intelligence learning modules include neural networks, support vector machines (SVMs), multi-layer perceptions (MLPs), and deep learning (Deep Learning). ) Real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that consisting of at least one artificial intelligence learning method of.
According to claim 3,
The machine vision control device detects the corresponding lower part under test using a preset machine vision algorithm based on the images on both sides of the lower part of the train obtained from the third and fourth line scan camera devices, and then Through artificial intelligence (AI)-based machine learning method, the abnormality detection of the lower part of the subject is detected,
The preset machine vision algorithm is composed of at least one of pattern matching, point/line/plane fitting, edge, color, and location algorithms Characterized in that A real-time high-speed detection system for railway track facilities using artificial intelligence-based machine vision algorithms.
According to claim 3,
The lower inspection object is a lower detailed part of the railway track facility, a plurality of sleepers disposed on the ground at regular intervals, and a pair of railroads fixed side by side in a direction orthogonal to the upper ends of both sides of the plurality of sleepers. It consists of at least one lower detailed part of a rail and a pan control for fixing the pair of railroad rails on the plurality of sleepers,
When the lower object to be inspected is a plurality of sleepers, the machine vision control device performs sleeper abnormality detection of breakage of the sleeper,
When the lower object to be inspected is a pair of railway rails, the machine vision controller detects at least one rail abnormality among rail waviness wear, tongue rail wear, and rail head damage to perform,
When the lower inspected object is a pan control, the machine vision control device is a real-time high-speed detection system of railroad track facilities using an artificial intelligence-based machine vision algorithm, characterized in that for performing pan control dropout abnormality detection.
According to claim 3,
The machine vision control device may include LED light emitting elements provided in the first and second line lighting devices during camera shutter exposure times of the first and second line scan camera devices and the third and fourth line scan camera devices. The output intensity of the LED light emitting elements provided in the third and fourth line lighting devices is applied with a predetermined overdrive constant current value in the form of a pulse signal greater than the rated continuous maximum value, so as to obtain high luminance pulse. The operation of the first and second line lighting devices and the third and fourth line lighting devices are respectively controlled by a constant current over drive control method using control,
The over-driving constant current value considers the cooling time of the LED light emitting elements included in the first and second line lighting devices and the LED light emitting devices included in the third and fourth line lighting devices between each pulse signal. Real-time high-speed detection system of railway track facilities using artificial intelligence-based machine vision algorithm, characterized in that set by
According to claim 1,
The upper object to be inspected is a pantograph for collecting electric energy from the corresponding train line by contacting the train line, and supplying power to the train through the pantograph, and on the ceiling surface of the tunnel or underground space. A real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that it consists of at least one of the T-bars, which are fixing means used in a device for fixing the catenary of the corresponding train.
According to claim 3,
The first and second line lighting devices and the third and fourth line lighting devices each include a plurality of LED light emitting channels connected in parallel to each other including at least one LED light emitting element,
The machine vision control device is based on a graphical user interface (GUI) based on a user's specific input signal output according to a user's request, the first and second line lighting devices and the third and fourth line lighting devices Setting the output level target value of each LED light-emitting channel provided in the lighting device, setting the output channel selection and order of each LED light-emitting channel for high-speed switching control, and clock for generating the operation signal of high-speed switching To set the generation, and to adjust the brightness of each LED light emitting element provided in each LED light emitting channel based on the output level target value of each LED light emitting channel set above, generate and output an output voltage target value of each LED light emitting channel, , The first and second line lighting devices and the third and second line lighting devices according to the output voltage target value of each LED light-emitting channel output in synchronization with the setting information and the clock information of the output channel selection and order of the set LED light-emitting channel. High-speed switching of each LED light emitting channel provided in the fourth line lighting device is controlled, and high-speed control is performed by using a hardware programming-based FPGA (Field Programmable Gate Array) for the high-speed switching function, and the first and second lines The first and second line scan camera devices and the third and fourth line scan camera devices respectively installed near the lighting device and the third and fourth line lighting devices detect an idle period of a photographing operation, and when no photographing motion signals from the second line scan camera apparatus and the third and fourth line scan camera apparatuses are input for a predetermined photographing operation reference time or longer, the idle state is determined, and after the determined idle state, the first state is entered. It performs an automatic reset function that removes the state during the shooting operation and initializes it to the initial input state so that it is maintained as At least one of the setting value of the order and the setting value for clock generation is controlled to be stored in a separate storage module, and the first and second line lighting devices and the third and fourth line lighting devices are controlled to be stored. In order to protect each LED light emitting element from being damaged due to the continuous turning on of each LED light emitting element provided in each LED light emitting channel of the device, the first and second line lighting devices and the third and fourth line lighting devices When the high-speed switching operation signal for each LED light emitting channel is changed, each LED light emitting channel of the first and second line lighting devices and the third and fourth line lighting devices operates only for a predetermined operation standard time less than a predetermined operation standard time. A real-time high-speed detection system for railroad track facilities using an artificial intelligence-based machine vision algorithm, characterized in that it is controlled to operate at high speed.
According to claim 9,
The user's specific input signal output according to the user's request is provided through an external terminal or server,
The external terminal or server transmits the user's specific input signal for user setting of LED lighting control according to the user's request through a pre-installed user setting related application for LED lighting control through a wired or wireless machine vision control device. Real-time high-speed detection system of railway track facilities using artificial intelligence-based machine vision algorithm, characterized in that for transmission to.
According to claim 1,
The machine vision control device detects the object to be inspected in the upper part by using a preset machine vision algorithm based on the image frames on both sides of the upper part of the divided corresponding train, and then performs machine learning based on preset artificial intelligence (AI). Through the method, the abnormal detection of the upper object to be inspected is performed,
The preset machine vision algorithm is composed of at least one of pattern matching, point/line/plane fitting, edge, color, and location algorithms Characterized in that A real-time high-speed detection system for railway track facilities using artificial intelligence-based machine vision algorithms.
According to claim 3,
Images on both sides of the upper part of the train obtained from the first and second line scan camera devices, images on both sides of the lower part of the train obtained from the third and fourth line scan camera devices, and the upper object and the lower object under test a storage device for storing abnormality detection result information;
Images on both sides of the upper part of the train obtained from the first and second line scan camera devices, images on both sides of the lower part of the train obtained from the third and fourth line scan camera devices, and the upper object and the lower object under test a display device for displaying the abnormality detection result information on a display screen; and
Images on both sides of the upper part of the train obtained from the first and second line scan camera devices, images on both sides of the lower part of the train obtained from the third and fourth line scan camera devices, and the upper object and the lower object under test A communication device for transmitting the abnormal detection result information of the wired or wireless communication method is further included,
The machine vision control device includes images on both sides of the upper portion of the train obtained from the first and second line scan camera devices, images on both sides of the lower portion of the train obtained from the third and fourth line scan camera devices, and the upper portion of the train. The abnormal detection result information of the subject and the corresponding lower subject is controlled to be stored in the storage device by making a database (DB) for each upper side and lower side of the train, respectively, and from the first and second line scan camera devices. To display the acquired images on both sides of the upper part of the corresponding train, the images on both sides of the lower part of the corresponding train obtained from the third and fourth line scan camera devices, and the abnormality detection result information of the corresponding upper and lower objects to be inspected on the display screen It controls the operation of the display device, and images of both upper sides of the corresponding train obtained from the first and second line scan camera devices, images of both lower sides of the corresponding train obtained from the third and fourth line scan camera devices, and Controls the operation of the communication device to transmit abnormality detection result information of the upper object to be inspected and the corresponding lower object to be inspected to an external terminal or server in a wired or wireless communication method;
The external terminal or server transmits images of the upper sides of the corresponding train obtained from the first and second line scan camera devices transmitted from the communication device through a pre-installed railway track facility detection related application, and the third and fourth images of the corresponding train. An artificial intelligence-based machine characterized in that it provides a service so that images of both sides of the lower part of the train obtained from the linescan camera device and abnormal detection result information of the upper and lower inspected objects are searched or displayed on a display screen. Real-time high-speed detection system of railway track facilities using vision algorithm.
According to claim 3,
The machine vision control device receives images of both upper sides of the train obtained from the first and second line scan camera devices or images of both sides of the lower portion of the train obtained from the third and fourth line scan camera devices. After converting the images on both sides of the upper or lower sides of the train into a panoramic image using a preset image stitching technique, the converted panoramic image is controlled to be displayed on the display screen A real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm.
According to claim 3,
an RFID tag reader device mounted on the inside/outside of the corresponding train and reading the reference position information from the RFID tag installed at each predetermined reference position on the track of the railway track facility;
a mileage measuring device installed inside/outside the train and measuring mileage information of the train; and
A location detection device mounted on the inside/outside of the train and continuously and automatically detecting the current location information of the corresponding train by using the reference location information of the RFID tag reader device and the mileage measurement information of the mileage measuring device more included,
The machine vision control device receives current location information of the corresponding train detected from the location detection device, and based on this, for each frame of images on both sides of the upper portion of the corresponding train obtained from the first and second line scan camera devices, Alternatively, the current location information of the corresponding train is mapped for each frame of the images on both sides of the lower portion of the corresponding train obtained from the third and fourth linescan camera devices, and controlled to be stored in a separate storage device, and measured from the mileage measuring device. The mileage measurement signal of the corresponding train is decomposed into a preset measurable cycle to generate a camera shutter trigger signal for image capture, and the first and second line scan camera devices are based on the generated camera shutter trigger signal. And a real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that for controlling the operation of the third and fourth line scan camera devices.
According to claim 1,
The upper detailed parts of the railway track facility are a pantograph to collect electric energy from the corresponding tram line in contact with the catenary of the train, and a messenger wire to support the load of the catenary and maintain the temporary height. And a dropper, a moving bracket for supporting and buffering the force received by the corresponding tram line from the progress of the train, and a steady arm for sharing the load of the corresponding tram line and buffering the vertical vibration and push-up shock. arm), a T-bar for fixing the corresponding catenary to the ceiling surface of a tunnel or underground space, and an artificial intelligence-based machine vision algorithm characterized in that it consists of at least one of insulation insulators for electrical insulation. Real-time high-speed detection system of railroad track facilities using

Images