KR102636488B1 - System for real-time high speed detection of rail way facility using machine vision algorithm based on artificial intelligent - Google Patents

Abstract

The present invention relates to a real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm. It is installed on both sides of the lower part of a train moving on the track of a railway track facility, and is installed on both sides of the lower part of the train on a line-by-line basis. First and second line scan camera devices that scan images to acquire two-dimensional images, and are installed on both sides of the lower part of the train, respectively, in the form of a line with the same optical axis as the first and second line scan camera devices. The first and second line lighting devices that irradiate light are synchronized with the trigger signal of the preset camera shutter, and the camera shutters of the first and second line scan camera devices are opened and the first and second line lighting devices are illuminated at the same time. The operation of the first and second line scan camera devices and the first and second line lighting devices is controlled to emit light, and images from both lower sides of the train obtained from the first and second line scan camera devices are provided to preset Machine vision control that detects the lower object by repeatedly learning the appearance and relative position or geometric relationship of the lower object through artificial intelligence (AI)-based machine learning methods and detects abnormalities in the lower object. By including the device, the safety of the train can be effectively secured by quickly and accurately detecting and detecting abnormalities in the upper and/or lower inspection objects of the railway track facility.

Description

Real-time high-speed detection system for railway track facilities using artificial intelligence-based machine vision algorithm {SYSTEM FOR REAL-TIME HIGH SPEED DETECTION OF RAIL WAY FACILITY USING MACHINE VISION ALGORITHM BASED ON ARTIFICIAL INTELLIGENT}

The present invention relates to a real-time, high-speed detection system for railway track facilities using an artificial intelligence (AI)-based machine vision algorithm.

In general, what makes railways different from other means of transportation is that electric railway vehicles (or trains) (e.g., subways, subways, high-speed trains, etc.) run on tracks, and electric railway vehicles for transporting people or cargo are It is moved along railway tracks installed above or underground.

These electric railway vehicles are used by many people due to their speed, accuracy, and stability compared to other means of transportation. Moreover, with the recent opening of the high-speed rail, it is gaining more attention as a means of public transportation.

Meanwhile, a railway track installed above or underground consists of a continuously formed rail, a number of sleepers supporting the lower part of the rail, and a gravel layer or concrete layer supporting these sleepers. In this case, conventionally, Sleepers used to be made of expensive oak or low-priced pine, but because their service life is relatively short compared to the price, sleepers made of concrete are mainly used in recent years.

Conventionally, inspection of carriers and tracks such as trains and subways, inspection of damage to rail fixing members, and evaluation of suitability of railroad tracks depend on the inspection of inspectors with long experience. As such, the reality is that humans have been directly inspecting track and rail fixing members for defects and damage, resulting in not only a waste of time and manpower, but also an economic waste. Above all, because it is done by people, complete inspection is impossible. There was a problem.

In addition, electric railway vehicles mainly use a pantograph, which is a current collection system, and only when the pantograph and the tram line are in precise contact can the electric railway vehicle operate stably and at high speed.

In other words, a typical pantograph is equipped with a current collector that contacts a wire on a diamond-shaped foldable frame installed on the roof of an electric railway vehicle, and is pushed up so that the wire is in close contact with the force of a spring or compressed air.

The pantograph is an important facility in the electric railway power supply system that transmits the electricity needed for electric railway vehicles to the vehicles through contact with the tram line. As the speed of electric railway vehicles increases, the separation of these pantographs from the tram line increases, leading to increasing cases of damage to detailed parts, including current collectors and pantographs, due to arcs and impacts.

In particular, detailed parts that directly affect the operation of the electric railway vehicle are installed on the upper and/or lower part of the electric railway vehicle, so it is difficult to determine the presence or absence of damage to the detailed parts solely with the operator's visual observation and experience level, making safety inspection less accurate. There was a problem.

Therefore, there is a need to develop a system that can determine the abnormal status of detailed parts, including pantographs, even during high-speed operation of electric railway vehicles. Detection technology to ensure the safety of vehicles is urgently needed.

Domestic registered patent No. 10-1058179 (announced on August 22, 2011)

The present invention was created to solve the above-mentioned problems, and the purpose of the present invention is to irradiate light in the form of lines on both sides of the upper and/or lower sides of a train moving on the tracks of a railway track facility, and to identify the train on a line-by-line basis. After acquiring an image with a two-dimensional image by scanning the images on both the upper and/or lower sides of the railway track facility, the entire image with a two-dimensional image of the upper and/or lower inspection object of the relevant railway track facility is used to detect artificial intelligence (AI) )-based machine vision algorithm is used to automatically detect and detect abnormalities in the upper and/or lower parts of the railway track facilities, thereby detecting and detecting abnormalities in the upper and/or lower parts of the railway track facilities. The goal is to provide a real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm that can effectively secure the stability of the train by quickly and accurately performing.

In order to achieve the above-described object, one aspect of the present invention is to provide an image that is installed on both sides of the lower part of a train moving on the tracks of a railway track facility, and has a two-dimensional image by scanning the images on both sides of the lower part of the train on a line-by-line basis. First and second line scan camera devices that acquire; first and second line lighting devices installed on both sides of the lower part of the train and emitting light in the form of a line with the same optical axis as the first and second line scan camera devices; And the first and second line scans are synchronized with the trigger signal of the preset camera shutter so that the camera shutters of the first and second line scan camera devices open and the lights of the first and second line lighting devices emit light. An artificial intelligence (AI)-based machine that controls the operation of the camera device and the first and second line lighting devices, receives images from both lower sides of the train obtained from the first and second line scan camera devices, and is preset. An artificial intelligence-based system that includes a machine vision control device that detects the lower object by repeatedly learning the appearance and relative position or geometric relationship of the lower object through a learning method and detects abnormalities in the lower object. It provides a real-time, high-speed detection system for railway track facilities using machine vision algorithms.

Here, the machine vision control device receives images from both sides of the lower part of the train obtained from the first and second line scan camera devices and determines the lower part of the inspection object through a preset artificial intelligence (AI)-based machine learning method. A first artificial intelligence learning module that constructs a first artificial intelligence learning model for detecting the corresponding lower inspection object by repeatedly learning the appearance and relative position or geometric relationship; Inferring and analyzing the appearance and location characteristics of the lower test object for each normal/abnormal characteristic item for the lower test object detected using the first artificial intelligence learning model constructed from the first artificial intelligence learning module, and the inference And after classifying the type of abnormality for the lower test object based on the appearance and location characteristic information of the analyzed lower test object, the corresponding lower test object is classified through a preset artificial intelligence (AI)-based machine learning method. A second artificial intelligence learning module that constructs a second artificial intelligence learning model for detecting abnormalities in the corresponding lower inspection object by performing repeated learning appropriate for the classified abnormality occurrence type; Using the second artificial intelligence learning model built from the second artificial intelligence learning module, infer and analyze the characteristics of the type of abnormality in the lower test object for each characteristic item of the type of abnormality in the lower test object, and A lower test object abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the lower test object based on the inferred and analyzed characteristic information about the type of abnormality in the lower test object; And the first and second line scans are synchronized with the trigger signal of the preset camera shutter so that the camera shutters of the first and second line scan camera devices open and the lights of the first and second line lighting devices emit light. It is preferable to include a camera device and a machine vision control module that controls the operation of the first and second line lighting devices.

Preferably, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention is installed on both sides of the upper part of the train, and records images of both upper sides of the train on a line-by-line basis. Third and fourth line scan camera devices that acquire two-dimensional images by scanning; And third and fourth line lighting devices installed on both sides of the upper part of the train, respectively, and emitting light in the form of a line with the same optical axis as the third and fourth line scan camera devices may be further included.

Preferably, the machine vision control device is synchronized to a trigger signal of a preset camera shutter so that the camera shutters of the first and second line scan camera devices open and the lights of the first and second line lighting devices emit light. The operations of the first and second line scan camera devices and the first and second line lighting devices are controlled so that the camera shutters of the third and fourth line scan camera devices are opened simultaneously with the third and fourth line scan camera devices. The operation of the third and fourth line scan camera devices and the third and fourth line lighting devices can be controlled so that the lights of the four line lighting devices emit light.

Preferably, the machine vision control device receives images from both sides of the lower part of the train obtained from the first and second line scan camera devices and determines the lower object to be inspected through a preset artificial intelligence (AI)-based machine learning method. Detects the lower object to be inspected by repeatedly learning its appearance and relative position or geometric relationship, and detects abnormalities in the lower object to be inspected, and both sides of the upper part of the train obtained from the third and fourth line scan camera devices In order to receive an image and detect the upper object, the image frames on both upper sides of the train are divided using a preset frame division technique, and the image frames on both upper sides of the divided train are used to determine the position of the upper object. After extracting the edge components of the upper detailed parts of the railway track facility, the corresponding By repeatedly learning the geometric relationship between the upper detailed parts of a railroad track facility, the upper inspection object can be detected and abnormality detection of the upper inspection object can be performed.

Preferably, the machine vision control device receives images from both sides of the lower part of the train obtained from the first and second line scan camera devices and uses a preset artificial intelligence (AI)-based machine learning method to determine the lower part of the object to be inspected. A first artificial intelligence learning module that constructs a first artificial intelligence learning model for detecting the corresponding lower inspection object by repeatedly learning the appearance and relative position or geometric relationship; Inferring and analyzing the appearance and location characteristics of the lower test object for each normal/abnormal characteristic item for the lower test object detected using the first artificial intelligence learning model constructed from the first artificial intelligence learning module, and the inference And after classifying the type of abnormality for the lower test object based on the appearance and location characteristic information of the analyzed lower test object, the corresponding lower test object is classified through a preset artificial intelligence (AI)-based machine learning method. A second artificial intelligence learning module that constructs a second artificial intelligence learning model for detecting abnormalities in the corresponding lower inspection object by performing repeated learning appropriate for the classified abnormality occurrence type; Using the second artificial intelligence learning model built from the second artificial intelligence learning module, infer and analyze the characteristics of the type of abnormality in the lower test object for each characteristic item of the type of abnormality in the lower test object, and A lower test object abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the lower test object based on the inferred and analyzed characteristic information about the type of abnormality in the lower test object; An image segmentation module that receives images from both upper sides of the train obtained from the third and fourth line scan camera devices and divides the upper side image frames of the train through a preset frame division technique to detect the upper inspection object; An edge extraction module for extracting edge components for upper detailed parts of the railway track facility in order to determine the position of the upper object to be inspected from the image frames on both upper sides of the train divided from the image segmentation module; Geometric relationship to the upper detailed parts of the relevant railway track facility through a preset artificial intelligence (AI)-based machine learning method based on the edge components of the upper detailed parts of the relevant railway track facility extracted from the edge extraction module. A third artificial intelligence learning module that constructs a third artificial intelligence learning model for detecting the corresponding upper inspection object by repeatedly learning the relationship; Infer and analyze the appearance and location characteristics of the upper test object for each normal/abnormal feature item detected using the third artificial intelligence learning model built from the third artificial intelligence learning module, and infer the above. And after classifying the type of abnormality for the upper test object based on the analyzed appearance and location characteristic information of the upper test object, the upper test object is classified through a preset artificial intelligence (AI)-based machine learning method. A fourth artificial intelligence learning module that constructs a fourth artificial intelligence learning model for detecting abnormalities in the upper inspection object by performing repeated learning appropriate for the classified type of abnormality occurrence; Using the fourth artificial intelligence learning model built from the fourth artificial intelligence learning module, infer and analyze the characteristics of the type of abnormality of the upper inspection object for each characteristic item of the abnormality type for the upper inspection object, and An upper inspection object abnormality detection module that automatically detects whether and what type of abnormality occurs for the upper inspection object based on the inferred and analyzed characteristic information about the type of abnormality of the upper inspection object; And the first and second line scans are synchronized with the trigger signal of the preset camera shutter so that the camera shutters of the first and second line scan camera devices open and the lights of the first and second line lighting devices emit light. In addition to controlling the operation of the camera device and the first and second line lighting devices, the camera shutters of the third and fourth line scan camera devices are opened and the lights of the third and fourth line lighting devices emit light at the same time. It may include a machine vision control module that controls operations of the third and fourth line scan camera devices and the third and fourth line lighting devices.

Preferably, the artificial intelligence (AI)-based machine learning method applied to the first to fourth artificial intelligence learning modules includes neural network, support vector machine (SVM), multi layer perception (MLP), and deep learning. It can be achieved with at least one artificial intelligence learning method (Deep Learning).

Preferably, the machine vision control device detects the upper object to be inspected using a preset machine vision algorithm based on the divided image frames on both sides of the upper part of the train, and then detects the upper object to be inspected using a preset artificial intelligence (AI)-based It is possible to detect abnormalities in the upper inspection object through the machine learning method.

Preferably, the preset machine vision algorithm is at least one of pattern matching, point/line/surface fitting, edge, color, and location algorithms. It can be done.

Preferably, the upper detailed parts of the railway track facility include a pantograph for contacting the tram line of the train and collecting electrical energy from the tram line, and a clamshell line for supporting the load of the tram line and maintaining the erection height ( Messenger wire and dropper, a moving bracket to support and cushion the force received by the tram line from the progress of the train, and a steady to share the load of the tram line and cushion the up-and-down vibration and pushing shock. It may be composed of at least one of a steady arm, a T-bar for fixing the tram line to the ceiling of a tunnel or underground space, and an insulating insulator for electrical insulation.

Preferably, the machine vision control device illuminates the LEDs provided in the first and second line lighting devices during the camera shutter exposure time of the first and second line scan camera devices and the third and fourth line scan camera devices. By applying the output intensity of the light emitting element and the LED light emitting element provided in the third and fourth line lighting devices to a preset overdriving constant current value in the form of a pulse signal greater than the rated continuous maximum value, high brightness can be obtained. The operations of the first and second line lighting devices and the third and fourth line lighting devices can be controlled respectively by a constant current overdrive control method using pulse control.

Preferably, the over-driving constant current value is the cooling time of the LED light-emitting devices provided in the first and second line lighting devices and the LED light-emitting devices provided in the third and fourth line lighting devices between each pulse signal. It can be set considering each.

Preferably, the upper object to be inspected is a pantograph that contacts the tram line of the train and collects electrical energy from the tram line, and supplies power to the train through the pantograph, and supplies power to the train in the tunnel or underground space. It may be composed of at least one of T-Bars, which are fixing means used in a device for fixing the train's tram line to the ceiling surface.

Preferably, the first and second line lighting devices and the third and fourth line lighting devices may each include a plurality of LED light-emitting channels connected in parallel and each having at least one LED light-emitting device.

Preferably, the machine vision control device emits light from each LED provided in the first and second line lighting devices and the third and fourth line lighting devices according to a user's specific input signal output according to the user's request. Setting the target value of the output level of the channel, setting the output channel selection and order of each LED emitting channel for high-speed switching control, and setting the clock generation to generate the operation signal for high-speed switching, and setting each LED set above. Based on the output level target value of the light emitting channel, the output voltage target value of each LED light emitting channel is generated and output to adjust the brightness of each LED light emitting element provided in each LED light emitting channel, and the output of each LED light emitting channel set above is generated and output. Provided in the first and second line lighting devices and the third and fourth line lighting devices, respectively, according to the output voltage target value of each LED light emitting channel output in synchronization with the channel selection and sequence setting information and clock information. High-speed switching of each LED emission channel can be controlled.

Preferably, the machine vision control device can output a specific input signal of the user according to the user's request based on a graphical user interface (GUI).

Preferably, the machine vision control device can perform high-speed control using a hardware programming-based FPGA (Field Programmable Gate Array) for high-speed switching function.

Preferably, the machine vision control device includes first and second line scan camera devices and third and fourth line lighting devices installed near the first and second line lighting devices and the third and fourth line lighting devices, respectively. By detecting an idle period of the photographing operation for the scan camera device, when the photographing operation signal of the first and second line scan camera devices and the third and fourth line scan camera devices is not input for more than a preset photographing operation reference time, When the state is determined to be in an idle state, an automatic reset function may be performed to remove the state during the shooting operation and initialize it to the initial input state so that the state is maintained in the initial state after the determined idle state.

Preferably, the machine vision control device includes among the set values for the output level target of each LED light emitting channel, the set values for output channel selection and order of each LED light emitting channel, and the set values for clock generation. At least one setting value can be controlled to be stored in a separate storage module.

Preferably, the user's specific input signal output according to the user's request may be provided through an external terminal or server.

Preferably, the external terminal or server receives the user's specific input signal for user setting of LED lighting control according to the user's request through a pre-installed user setting related application for LED lighting control, by wired or wirelessly to the machine. It can be transmitted to the vision control device.

Preferably, the machine vision control device emits light from each LED due to continuous turning on of each LED light-emitting element provided in each LED light-emitting channel of the first and second line lighting devices and the third and fourth line lighting devices. In order to protect the device from damage, constant operation below a preset operation reference time occurs when the high-speed switching operation signal for each LED light emitting channel of the first and second line lighting devices and the third and fourth line lighting devices is changed. Each LED light emitting channel of the first and second line lighting devices and the third and fourth line lighting devices can be controlled to perform high-speed switching operations only at the reference time.

Preferably, the real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention includes images of both lower sides of the train obtained from the first and second line scan camera devices, A storage device that stores images of both sides of the upper part of the train obtained from the third and fourth line scan camera devices, and abnormality detection result information of the upper and lower objects to be inspected may be further included.

Preferably, the machine vision control device includes images of both lower sides of the train obtained from the first and second line scan camera devices, images of both upper sides of the train obtained from the third and fourth line scan camera devices, And it is possible to control the abnormality detection result information of the upper and lower inspection objects to be stored in the storage device by converting them into a database (DB) for each upper and lower sides of the train.

Preferably, the real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention includes images of both lower sides of the train obtained from the first and second line scan camera devices, A display device that displays images on both sides of the upper part of the train obtained from the third and fourth line scan camera devices, and abnormality detection result information of the upper and lower test objects on a display screen may be further included.

Preferably, the machine vision control device includes images of both lower sides of the train obtained from the first and second line scan camera devices, images of both upper sides of the train obtained from the third and fourth line scan camera devices, And the operation of the display device can be controlled so that abnormality detection result information of the upper and lower inspection objects is displayed on the display screen.

Preferably, the real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention includes images of both lower sides of the train obtained from the first and second line scan camera devices, A communication device that transmits images of both upper sides of the train obtained from the third and fourth line scan camera devices, and abnormality detection result information of the upper and lower inspection objects through a wired or wireless communication method may be further included. there is.

Preferably, the machine vision control device includes images of both lower sides of the train obtained from the first and second line scan camera devices, images of both upper sides of the train obtained from the third and fourth line scan camera devices, And the operation of the communication device can be controlled so that the abnormality detection result information of the upper inspection object and the lower inspection object is transmitted to an external terminal or server through wired or wireless communication.

Preferably, the external terminal or server includes images on both sides of the lower part of the train acquired from the first and second line scan camera devices transmitted from the communication device through a pre-installed railway track facility detection-related application, and the third And a service can be provided to search for images on both sides of the upper part of the train obtained from the fourth line scan camera device, and abnormality detection result information for the upper and lower objects to be inspected, or display them on a display screen.

Preferably, the machine vision control device includes images on both lower sides of the train obtained from the first and second line scan camera devices, or images on both upper sides of the train obtained from the third and fourth line scan camera devices. is provided, and the images on both lower and upper sides of the train are converted into panoramic images using a preset image stitching technique, and then the converted panoramic images are displayed on the display screen. You can control its display.

Preferably, the real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention is mounted inside/outside the train, and is based on a preset standard on the track of the relevant railway track facility. RFID tag reader device that reads the reference location information from the RFID tag installed at each location; A mileage measuring device that is mounted inside/outside the train and measures the mileage information of the train; and a position detection device that is mounted inside/outside the train and continuously and automatically detects the current location information of the train using the reference position information of the RFID tag reader device and the mileage measurement information of the mileage measurement device. More may be included.

Preferably, the machine vision control device receives the current location information of the train detected from the position detection device, and based on this, the angles of the images on both sides of the lower part of the train obtained from the first and second line scan camera devices The current location information of the train can be mapped for each frame, or for each frame of the images on both upper sides of the train obtained from the third and fourth line scan camera devices, and stored in a separate storage device.

Preferably, the machine vision control device decomposes the mileage measurement signal of the train measured from the mileage measurement device into a preset measurable cycle to generate a trigger signal for a camera shutter for image capture, and the generated camera The operations of the first and second line scan camera devices and the third and fourth line scan camera devices can be controlled based on the trigger signal of the shutter.

Preferably, the machine vision control device detects the lower object to be inspected using a preset machine vision algorithm based on the images of both lower sides of the train obtained from the first and second line scan camera devices. , it is possible to detect abnormalities in the lower test object through a preset artificial intelligence (AI)-based machine learning method.

Preferably, the preset machine vision algorithm is at least one of pattern matching, point/line/surface fitting, edge, color, and location algorithms. It can be done.

Preferably, the lower object to be inspected is a lower detailed part of the railway track facility, and is comprised of a plurality of sleepers arranged at regular intervals on the ground, as long as they are fixed and installed side by side in a direction perpendicular to both upper ends of the plurality of sleepers. It may be comprised of a pair of railway rails, and a lower detail part of at least one of a pan trolley for fixing the pair of railway rails on the plurality of sleepers.

Preferably, when the lower object to be inspected is a plurality of sleepers, the machine vision control device can detect sleeper abnormalities due to damage to the sleepers.

Preferably, when the lower object to be inspected is a pair of railway rails, the machine vision control device detects at least one of rail waviness wear, tongue rail wear, and rail head damage. Rail abnormality detection can be performed.

Preferably, when the lower object to be inspected is a pan control, the machine vision control device can detect an abnormality in the pan control being dropped.

According to the real-time high-speed detection system for railway track facilities using the artificial intelligence-based machine vision algorithm of the present invention as described above, light is formed in the form of lines on both sides of the upper and/or lower part of the train moving on the track of the railway track facility. After examining the image on both sides of the upper and/or lower part of the train in line units to obtain an image with a two-dimensional image, an image with a two-dimensional image of the upper and/or lower object of the railway track facility is obtained. Using the entire image, an artificial intelligence (AI)-based machine vision algorithm is used to automatically detect and detect abnormalities in the upper and/or lower part of the relevant railway track facility. It has the advantage of effectively securing the stability of the train by quickly and accurately detecting and detecting abnormalities in the lower inspection object.

Figure 1 is an overall block diagram illustrating a real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention.
Figure 2 is a detailed block diagram for explaining a machine vision control device applied to an embodiment of the present invention.
Figure 3 is a perspective view for explaining the arrangement of first to fourth line scan camera devices and first to fourth line lighting devices applied to an embodiment of the present invention.
Figure 4 is a diagram illustrating, as an example, images on both sides of the lower part of the train acquired through the first and second line scan camera devices applied to an embodiment of the present invention and the lower part of the railroad track facility detected through the machine vision control device. am.
Figure 5 is a diagram showing an example of a detection process for a lower inspection object of a railroad track facility through a machine vision control device applied to an embodiment of the present invention.
Figure 6 is a diagram showing an example of images on both upper sides of a train acquired through the third and fourth line scan camera devices applied to an embodiment of the present invention and image frames on both upper sides of the train divided through a machine vision control device. .
Figure 7 is a diagram showing an example of edge components for upper detailed parts of a railroad track facility extracted through a machine vision control device applied to an embodiment of the present invention.
Figure 8 is an example of a geometric relationship for the upper detailed parts of the railway track facility based on the edge components of the upper detailed parts of the railway track facility extracted through the machine vision control device applied to an embodiment of the present invention. This is the drawing shown.
Figure 9 is a diagram showing an example of abnormality detection results for the upper and lower inspection objects of a railroad track facility detected through a machine vision control device applied to an embodiment of the present invention.
Figure 10 is a detailed block diagram for explaining an external terminal and/or server applied to an embodiment of the present invention.

The above-mentioned objects, features, and advantages will be described in detail later with reference to the attached drawings, so that those skilled in the art will be able to easily implement the technical idea of the present invention. In describing the present invention, if it is determined that a detailed description of known technologies related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

The terms used in the present invention are general terms that are currently widely used as much as possible while considering the function in the present invention, but this may vary depending on the intention or precedent of a person working in the art, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the relevant invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than simply the name of the term.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements. In addition, terms such as "... unit" and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention illustrated below can be modified into various other forms, and the scope of the present invention is not limited to the embodiments detailed below. Examples of the present invention are provided to more completely explain the present invention to those skilled in the art.

The combination of each block in the attached block diagram and each step in the flow chart may be performed by computer program instructions (execution engine), and these computer program instructions can be installed on a processor of a general-purpose computer, special-purpose computer, or other programmable data processing equipment. Since it can be mounted, the instructions executed through a processor of a computer or other programmable data processing equipment create a means of performing the functions described in each block of the block diagram or each step of the flow diagram. These computer program instructions may also be stored in computer-usable or computer-readable memory that can be directed to a computer or other programmable data processing equipment to implement a function in a particular manner, so that the computer-usable or computer-readable memory The instructions stored in can also produce manufactured items containing instruction means that perform the functions described in each block of the block diagram or each step of the flow diagram.

In addition, computer program instructions can be mounted on a computer or other programmable data processing equipment, so a series of operation steps are performed on the computer or other programmable data processing equipment to create a process that is executed by the computer and run on the computer or other program. Instructions that perform possible data processing equipment may also provide steps for executing functions described in each block of the block diagram and each step of the flow diagram.

Additionally, each block or each step may represent a module, segment, or portion of code containing one or more executable instructions for executing specified logical functions, and in some alternative embodiments, the blocks or steps referred to in the blocks or steps may represent a portion of code. It should be noted that it is possible for functions to occur out of order. For example, it is possible for two blocks or steps shown in succession to be performed substantially at the same time, and it is also possible for the blocks or steps to be performed in the reverse order of the corresponding functions, if necessary.

Figure 1 is an overall block diagram illustrating a real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention, and Figure 2 is a machine applied to an embodiment of the present invention. It is a specific block diagram for explaining the vision control device, and Figure 3 is a perspective view for explaining the arrangement of the first to fourth line scan camera devices and the first to fourth line lighting devices applied to an embodiment of the present invention. 4 shows, as an example, images of both sides of the lower part of the train acquired through the first and second line scan camera devices applied in an embodiment of the present invention and the lower part of the railroad track facility detected through the machine vision control device. Figure 5 is a diagram showing an example of a detection process for a lower inspection object of a railroad track facility through a machine vision control device applied to an embodiment of the present invention, and Figure 6 is a diagram showing an example of a detection process for an object to be inspected at the bottom of a railroad track facility applied to an embodiment of the present invention. It is a diagram illustrating, as an example, images on both upper sides of the train acquired through the third and fourth line scan camera devices and image frames on both upper sides of the train divided through the machine vision control device, and FIG. 7 is an example of an embodiment of the present invention. It is a diagram showing an example of the edge components of the upper detailed parts of a railway track facility extracted through an applied machine vision control device, and Figure 8 shows a railroad track facility extracted through a machine vision control device applied to an embodiment of the present invention. It is a diagram showing an example of the geometric relationship between the upper detailed parts of a railroad track facility based on the edge components of the upper detailed parts, and Figure 9 is a diagram showing the geometric relationship between the upper detailed parts of the railway track facility, and Figure 9 is a diagram showing the geometric relationship between the upper detailed parts of the railway track facility, and Figure 9 is a diagram showing the geometric relationship between the upper detailed parts of the railway track facility and the edge components of the upper detailed parts. This is a diagram showing an abnormality detection result for the upper and lower inspection objects of a railway track facility as an example, and Figure 10 is a specific block diagram for explaining an external terminal and/or server applied to an embodiment of the present invention.

1 to 10, the real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention largely includes first and second line scan camera devices 100a and 100b. ), first and second line lighting devices (200a and 200b), a machine vision control device (300), and a power supply device (400). In addition, the real-time high-speed detection system of railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention includes third and fourth line scan camera devices 500a and 500b, and third and fourth line lighting. Devices 600a and 600b, storage device 700, display device 750, communication device 800, RFID tag reader device 850, mileage measurement device 900, location detection device 950, external It may further include a terminal and/or server 20, etc. Meanwhile, since the components shown in FIGS. 1 to 10 are not essential, the real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention has more components than those. It may have or fewer components.

Hereinafter, we will look in detail at the components of a real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention.

The first and second line scan camera devices (100a and 100b) are respectively installed on both sides of the lower part of the train (1) moving on the track of the railway track facility, and are installed on both sides of the lower part of the train (1) on a line basis. It performs the function of acquiring an image with a two-dimensional image by scanning the image (i.e., image of the detailed lower parts of the railroad track facility).

That is, the first and second line scan camera devices 100a and 100b are used to detect lower detailed parts and/or details of the railway track facility in a state in which light is irradiated through the first and second line lighting devices 200a and 200b. The imaging surface of the lower object to be inspected is photographed, and the captured detailed lower parts of the railway track facility and/or the line scan image of the lower object to be inspected are transmitted to the machine vision control device 300.

These first and second line scan camera devices 100a and 100b do not acquire images on an area basis, but rather use one line, that is, a one-dimensional linear structure CMOS (Complementary Metal Oxide Semiconductor) or CCD (CCD). It is desirable to implement it with a camera that acquires a two-dimensional image by scanning in line units through a charge coupled device (Charge Coupled Device) image sensor.

Meanwhile, as shown in FIG. 4, the lower part of the railway track facility is at least one lower part of the railway track facility, and includes a plurality of sleepers arranged at regular intervals on the ground, A lower part of at least one of a pair of railway rails fixed and installed side by side in a direction perpendicular to both upper ends of the plurality of sleepers, and/or a pan troll for fixing the pair of railway rails on the plurality of sleepers It is desirable that it be made of detailed parts.

The first and second line lighting devices (200a and 200b) are installed on both sides of the lower part of the train (1), and emit light in the form of a line with the same optical axis as the first and second line scan camera devices (100a and 100b). Performs an investigative function.

Although not shown in the drawings, the first and second line lighting devices 200a and 200b preferably include a plurality of LED light-emitting channels connected in parallel and each having at least one LED light-emitting element.

That is, the first and second line lighting devices 200a and 200b include a plurality of LED light-emitting channels connected in parallel, and each of the plurality of LED light-emitting channels may include at least one LED light-emitting element. .

Additionally, when there are two or more LED light-emitting devices provided in each LED light-emitting channel, each LED light-emitting device may be connected in series and/or parallel in an array.

The first and second line lighting devices 200a and 200b configured as described above can receive the operating voltage output from the machine vision control device 300 and operate each LED light-emitting device of each LED light-emitting channel.

Meanwhile, the first and second line lighting devices 200a and 200b have a plurality of LED (Light Emitting Diode) light-emitting elements arrayed to irradiate surface light to the lower detailed parts and/or the lower inspection object of the relevant railroad track facility. It is preferably in a form, but is not limited to this, and may include at least one of, for example, a fluorescent lamp, an incandescent lamp, a halogen lamp, a mercury lamp, a neon tube lamp, a sodium lamp, a metal halide lamp, or an EL (Electro-Luminescent) lamp.

In addition, the first and second line lighting devices 200a and 200b include, for example, a liquid crystal display (LCD), a light emitting diode (LED), and a thin film transistor liquid crystal display (Thin Film Transistor-Liquid Crystal). Display, TFT LCD), Organic Light Emitting Diode (OLED), Flexible Display, Plasma Display Panel (PDP), Alternate Surface Lighting (ALiS), Digital Light Source Processing (DLP) , liquid crystal silicon (LCoS), surface conduction electron-emitting device display (SED), field emission display (FED), laser TV (quantum dot laser, liquid crystal laser), optoelectric liquid display (FLD), interferometric modulator display (iMoD) , thick film dielectric electroplating (TDEL), quantum dot display (QD-LED), telescopic pixel display (TPD), organic light emitting transistor (OLET), and/or laser fluorescence display (LPD), It is not limited to this, and may include anything that can irradiate surface light to the lower detailed parts and/or the lower inspection object of the relevant railroad track facility.

The machine vision control device 300 is a device responsible for overall control of the real-time high-speed detection system of railroad track facilities using an artificial intelligence-based machine vision algorithm according to an embodiment of the present invention, and in particular, the trigger signal of a preset camera shutter. The first and second line scan cameras are synchronized so that the camera shutters of the first and second line scan camera devices (100a and 100b) open and the lights of the first and second line lighting devices (200a and 200b) emit light. It performs a function of controlling the operation of the devices 100a and 100b and the first and second line lighting devices 200a and 200b.

In addition, the machine vision control device 300 is synchronized to the trigger signal of the preset camera shutter, and at the same time the camera shutters of the first and second line scan camera devices 100a and 100b are opened, the first and second line lighting devices ( Controls the operation of the first and second line scan camera devices (100a and 100b) and the first and second line lighting devices (200a and 200b) so that the lights of (200a and 200b) emit light, and also controls the operation of the third and fourth lines The third and fourth line scan camera devices (500a and 500b) and the third line scan camera devices (500a and 500b) emit light at the same time as the camera shutters of the scan camera devices (500a and 500b) are opened. and can perform a function of controlling the operation of the fourth line lighting devices 600a and 600b.

In addition, the machine vision control device 300 receives images from both sides of the lower part of the train 1 obtained from the first and second line scan camera devices 100a and 100b and uses a preset artificial intelligence (AI)-based machine learning method. Through iterative learning of the appearance, relative position, and/or geometric relationship of the lower object to be inspected, the corresponding lower object to be inspected can be detected and abnormality detection of the lower object to be inspected can be performed.

At this time, when the lower object to be inspected is a plurality of sleepers, the machine vision control device 300 can detect sleeper abnormalities due to damage to the sleepers. When the lower object to be inspected is a pair of railway rails, the machine vision control device 300 controls at least one of rail waviness wear, tongue rail wear, and/or rail head damage. Rail abnormality detection can be performed. If the lower object to be inspected is a pan control, the machine vision control device 300 may detect an abnormality in the pan control being dropped.

In addition, the machine vision control device 300 uses a preset machine vision algorithm based on the images on both sides of the lower part of the train 1 acquired from the first and second line scan camera devices 100a and 100b to detect the corresponding lower inspection object. After performing the detection, abnormality detection of the corresponding lower inspection object can be performed through a preset artificial intelligence (AI)-based machine learning method.

At this time, the preset machine vision algorithm is, for example, at least one of pattern matching, point/line/plane fitting, edge, color, and/or location algorithm. It is desirable that it be done through an algorithm.

In addition, the machine vision control device 300 receives images from both upper sides of the train 1 obtained from the third and fourth line scan camera devices 500a and 500b to detect the upper inspection object of the corresponding railroad track facility. The image frames on both upper sides of the train (1) are divided through a preset frame division technique, and the upper detailed parts of the railway track facility are used to determine the position of the upper object in the divided image frames on both upper sides of the train (1). After extracting the edge components for the above-extracted upper detailed parts of the relevant railway track facilities, the upper detailed parts of the relevant railway track facilities are obtained through a preset artificial intelligence (AI)-based machine learning method based on the edge components of the upper detailed parts of the relevant railway track facilities. By repeatedly learning the geometric relationship between the objects, the upper object can be detected and abnormalities in the upper object can be detected.

In addition, the machine vision control device 300 detects the upper object to be inspected using a preset machine vision algorithm based on the image frames on both sides of the upper part of the divided train 1, and then detects the upper object to be inspected using a preset artificial intelligence (AI). )-based machine learning method can be used to detect abnormalities in the upper test object.

At this time, the preset machine vision algorithm is, for example, at least one of pattern matching, point/line/plane fitting, edge, color, and/or location algorithm. It is desirable that it be done through an algorithm.

Meanwhile, the upper inspection object of the railway track facility is at least one upper detail part among the upper detailed parts provided on the upper side of the railway track facility, as shown in FIGS. 6 to 9, for example, the train (1) A pantograph for contacting a tram line and collecting electrical energy from the tram line, and/or supplying power to the train (1) through the pantograph, and attaching the train (1) to the ceiling of a tunnel or underground space. It is preferable that it consists of at least one of the T-Bar, which is a fixing means used in the device for fixing the tram line, but is not limited to this, and is comprised of at least one of the other upper detailed parts provided on the upper side of the relevant railway track facility. It may come true.

At this time, other upper detailed parts provided on the upper side of the relevant railway track facility are not shown in the drawing, but for example, a messenger wire and a dropper to support the load of the tram line of the train (1) and maintain the temporary height. ), a moving bracket to support and buffer the force received by the tram line from the progress of the train (1), and a steady arm to share the load of the tram line and cushion the up-and-down vibration and pushing shock. , and/or an insulating insulator for electrical insulation.

In addition, the machine vision control device 300 controls the first and second line scan camera devices 100a and 100b and the third and fourth line scan camera devices 500a and 500b during the camera shutter exposure times. A pulse signal in which the output intensity of the LED light-emitting devices provided in the line lighting devices (200a and 200b) and the LED light-emitting devices provided in the third and fourth line lighting devices (600a and 600b) is greater than the rated continuous maximum value. The first and second line lighting devices (200a and 200b) and the third and second line lighting devices (200a and 200b) are controlled by a constant current overdrive control method using pulse control to obtain high brightness by applying a preset overdrive constant current value. It can perform the function of controlling the operation of the four line lighting devices (600a and 600b), respectively.

At this time, the over-driving constant current value is the LED light emitting element provided in the first and second line lighting devices (200a and 200b) and the third and fourth line lighting devices (600a and 600b) between each pulse signal. It is desirable to set it considering the cooling time of each LED light-emitting device.

In addition, the machine vision control device 300 controls the first and second line lighting devices (200a and 200b) and the third and fourth line lighting devices (600a and 600b) by the user's specific input signal output according to the user's request. ), setting the output level target value of each LED emitting channel provided in each LED, selecting and ordering the output channel of each LED emitting channel for high-speed switching control, and/or clocking to generate an operation signal for high-speed switching. ), and generate and output the output voltage target value of each LED light-emitting channel to adjust the brightness of each LED light-emitting element provided in each LED light-emitting channel based on the output level target value of each LED light-emitting channel set above. And, the first and second line lighting devices (200a and 200b) according to the output voltage target value of each LED light-emitting channel output in synchronization with the output channel selection and order setting information and clock information of each LED light-emitting channel set. It is possible to perform a function of controlling high-speed switching of each LED light-emitting channel provided in the third and fourth line lighting devices 600a and 600b, respectively.

In addition, the machine vision control device 300 can perform a function of outputting a specific input signal of the user according to the user's request based on a graphical user interface (GUI).

In addition, the machine vision control device 300 can perform high-speed control using a hardware programming-based FPGA (Field Programmable Gate Array) for high-speed switching function.

In addition, the machine vision control device 300 includes first and second line scan cameras installed near the first and second line lighting devices 200a and 200b and the third and fourth line lighting devices 600a and 600b, respectively. By detecting the idle period of the shooting operation for the devices 100a and 100b and the third and fourth line scan camera devices 500a and 500b, the first and second line scan camera devices 100a and 100b and the third and fourth line scan camera devices 500a and 500b are detected. When the shooting operation signal of the fourth line scan camera device 500a and 500b is not input for more than a preset shooting operation reference time, it is determined to be in an idle state, and the shooting operation state is maintained in the initial state after the determined idle state. An automatic reset function can be performed to remove and initialize to the initial input state.

In addition, the machine vision control device 300 sets a setting value for the output level target of each LED emission channel, a setting value for output channel selection and order of each LED emission channel, and/or a setting for clock generation. A function of controlling at least one set value among the values to be stored in a separate storage module (not shown) or storage device 700 can be performed.

Meanwhile, the user's specific input signal output according to the user's request may be provided through an external terminal and/or server 20.

In addition, the machine vision control device 300 controls each LED light-emitting element provided in each LED light-emitting channel of the first and second line lighting devices 200a and 200b and the third and fourth line lighting devices 600a and 600b. In order to protect each LED light-emitting element from damage due to continuous turning on, each LED light-emitting channel of the first and second line lighting devices (200a and 200b) and the third and fourth line lighting devices (600a and 600b) is When the high-speed switching operation signal is changed, each LED of the first and second line lighting devices (200a and 200b) and the third and fourth line lighting devices (600a and 600b) emits light only at a certain operation standard time less than the preset operation standard time. It can perform the function of controlling the channel to perform high-speed switching operation.

In addition, the machine vision control device 300 displays images on both sides of the lower part of the train 1 obtained from the first and second line scan camera devices 100a and 100b, and the third and fourth line scan camera devices 500a and 500b. At least one of the images on both upper sides of the train (1) acquired from the image and/or abnormality detection result information on the lower test object and the upper test object for each lower side and/or upper side of the train (1), respectively. It is possible to perform the function of controlling the data to be converted into a database (DB) and stored in the storage device 700.

In addition, the machine vision control device 300 displays images on both sides of the lower part of the train 1 obtained from the first and second line scan camera devices 100a and 100b, and the third and fourth line scan camera devices 500a and 500b. Controlling the operation of the display device 750 to display on the display screen at least one of the images on both upper sides of the train 1 obtained from and/or abnormality detection result information of the lower inspection object and the upper inspection object. It can perform its function.

In addition, the machine vision control device 300 displays images on both sides of the lower part of the train 1 obtained from the first and second line scan camera devices 100a and 100b, and the third and fourth line scan camera devices 500a and 500b. At least one of the images on both sides of the upper part of the train (1) acquired from the train (1), and/or the abnormality detection result information of the lower test object and the upper test object, is transmitted to an external terminal and/or server through wired and/or wireless communication. It can perform the function of controlling the operation of the communication device 800 so that it is transmitted to (20).

In addition, the machine vision control device 300 uses images on both sides of the lower part of the train 1 obtained from the first and second line scan camera devices 100a and 100b, and/or the third and fourth line scan camera devices 500a. And 500b), the images on both upper sides of the train 1 are provided, and the images on both lower sides and/or the upper sides of the train 1 are converted into panoramic images using a preset image stitching technique. Image), a function can be performed to control the converted panoramic image to be displayed on the display screen.

In addition, the machine vision control device 300 decomposes the mileage measurement signal of the train 1 measured from the mileage measurement device 900 into a preset measurable period to generate a trigger signal for a camera shutter for video shooting, Based on the generated trigger signal of the camera shutter, the function of controlling the operation of the first and second line scan camera devices (100a and 100b) and the third and fourth line scan camera devices (500a and 500b) can be performed. there is.

In addition, the machine vision control device 300 receives the current location information of the train 1 detected from the position detection device 950, and based on this, the information obtained from the first and second line scan camera devices 100a and 100b is provided. of the train 1 for each frame of the image on both lower sides of the train 1 and/or for each frame of the image on both upper sides of the train 1 acquired from the third and fourth line scan camera devices 500a and 500b. The function of mapping the current location information and controlling it to be stored in a separate storage device 700 can be performed.

As shown in FIG. 2, this machine vision control device 300 largely includes a first artificial intelligence learning module 310, a second artificial intelligence learning module 320, a lower inspection object abnormality detection module 330, and a machine It includes a vision control module 340, etc. In addition, the machine image control device 300 applied to one embodiment of the present invention includes an image segmentation module 350, an edge extraction module 360, a third artificial intelligence learning module 370, and a fourth artificial intelligence learning module 380. ), and/or an upper inspection object abnormality detection module 390, etc. may be further included. Meanwhile, since the components shown in FIG. 2 are not essential, the machine vision control device 300 applied to an embodiment of the present invention may have more or fewer components.

Below, we will look in detail at the components of the machine vision control device 300 applied to an embodiment of the present invention.

The first artificial intelligence learning module 310 receives images from both sides of the lower part of the train 1 obtained from the first and second line scan camera devices 100a and 100b and uses a preset artificial intelligence (AI)-based machine learning method. It performs the function of constructing a first artificial intelligence learning model for detection of the lower object to be inspected by repeatedly learning the appearance, relative position, and/or geometric relationship of the lower object to be inspected.

For example, if the lower object to be inspected is a rail, the machine vision control module 340 and/or the first artificial intelligence learning module 310 detects an edge in the images on both sides of the lower part of the train 1 and then draws a straight line. Hough Transform is applied for detection, and since there are straight line components other than the rail, the slope value after applying the Hough Transform is learned using SVM (Support Vector Machine), and the final slope value is selected and the rail area is detected. can do. Meanwhile, the rail can also be detected through a relative position learning algorithm based on the detection area of the pan troll and sleeper (see Figure 5).

On the other hand, if the lower object to be inspected is a sleeper, the detected rail area can be used for localization of the sleeper, and the detected rail area is perpendicular to the rail through SVM (Support Vector Machine). By learning the slope component and detecting edges with a slope perpendicular to the rail, it is possible to provide an initial value for localization of sleepers at regular intervals.

On the other hand, if the lower test object is a pan troll, the initial value of the pan troll localization can be provided to the relative left/right positions using the sleeper position determination result.

The second artificial intelligence learning module 320 is used to test the lower subject for each normal and/or abnormal feature item detected using the first artificial intelligence learning model built from the first artificial intelligence learning module 310. Infer and analyze the appearance and location characteristics of the body, classify the type of abnormality for the lower subject based on the inferred and analyzed appearance and location characteristic information of the lower subject, and then use preset artificial intelligence (AI) )-based machine learning method, it performs repetitive learning appropriate for the type of abnormality classified for the lower test object to build a second artificial intelligence learning model for detecting abnormalities in the lower test object.

The lower test object abnormality detection module 330 uses the second artificial intelligence learning model built from the second artificial intelligence learning module 320 to determine the abnormality of the lower test object according to the characteristic items of the type of abnormality. Performs the function of inferring and analyzing the characteristics of the type of occurrence and automatically detecting whether and what type of abnormality occurred for the lower test object based on the inferred and analyzed characteristic information about the type of abnormality of the lower test object. do.

In addition, the machine vision control module 340 is responsible for overall control of the machine vision control device 300 applied to an embodiment of the present invention, and performs all control functions performed by the machine vision control device 300 described above. In particular, the camera shutters of the first and second line scan camera devices (100a and 100b) are opened in synchronization with the trigger signal of the preset camera shutter, and at the same time, the camera shutters of the first and second line lighting devices (200a and 200b) are opened. It performs a function of controlling the operation of the first and second line scan camera devices (100a and 100b) and the first and second line lighting devices (200a and 200b) so that the lights emit.

In addition, the machine vision control module 340 is synchronized with the trigger signal of the preset camera shutter so that the camera shutters of the third and fourth line scan camera devices 500a and 500b are opened and the third and fourth line lighting devices (500a and 500b) are opened simultaneously. It performs a function of controlling the operation of the third and fourth line scan camera devices (500a and 500b) and the third and fourth line lighting devices (600a and 600b) so that the lights (600a and 600b) emit light.

Various embodiments described herein may be implemented, for example, in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). , may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases such embodiments may be implemented by machine vision control module 340.

According to software implementation, embodiments such as procedures or functions may be implemented with a separate software module that performs at least one function or operation. Software code can be implemented by a software application written in an appropriate programming language. Additionally, the software code may be stored in a separate storage module (not shown) or storage device 700 and executed by the machine vision control module 340.

Meanwhile, the modules described above, that is, the first artificial intelligence learning module 310, the second artificial intelligence learning module 320, and/or the lower object abnormality detection module 330, are separately separated and each module Although it is implemented so that the functions are performed in, it is not limited to this, and the machine vision control module 340 may be implemented to perform all the functions of the modules described above.

Additionally, the image segmentation module 350 receives images from both sides of the upper part of the train 1 obtained from the third and fourth line scan camera devices 500a and 500b and performs preset frame segmentation for detecting the upper object. ) The function of dividing the image frames on both upper sides of the train (1) is performed through the technique.

At this time, the preset frame division technique is a common image processing technique, which is a process of dividing a digital image into several sets of pixels. In particular, it is a technique used to find objects and boundaries (e.g., lines, curves, etc.) in the image. The image segmentation or frame segmentation technique is a widely known image processing technique, and a detailed description thereof will be omitted.

The edge extraction module 360 extracts edge components for the upper detailed parts of the railway track facility in order to determine the position of the upper inspection object in the image frames on both upper sides of the train 1 divided by the image segmentation module 350. performs the function of

The third artificial intelligence learning module 370 uses a preset artificial intelligence (AI)-based machine learning method based on the edge components of the upper detailed parts of the railway track facility extracted from the edge extraction module 360. It performs the function of constructing a third artificial intelligence learning model for detection of the upper inspection object by repeatedly learning the geometric relationships of the upper detailed parts of the railroad track facility.

The fourth artificial intelligence learning module 380 performs the upper test object for each normal and/or abnormal feature item detected using the third artificial intelligence learning model built from the third artificial intelligence learning module 370. After inferring and analyzing the appearance and location characteristics of the body, classifying the type of abnormality for the upper inspection object based on the inferred and analyzed appearance and location characteristic information of the upper inspection object, and then using preset artificial intelligence (AI) )-based machine learning method, it performs repetitive learning appropriate for the type of abnormality classified for the upper inspection object to build a fourth artificial intelligence learning model for detecting abnormalities in the upper inspection object.

In addition, the upper inspection object abnormality detection module 390 uses the fourth artificial intelligence learning model built from the fourth artificial intelligence learning module 380 to classify the upper inspection object according to the characteristic items of the type of abnormality for the upper inspection object. A function to infer and analyze the characteristics of the type of abnormality in the upper test object and automatically detect whether and what type of abnormality has occurred in the upper test object based on the inferred and analyzed characteristic information about the type of abnormality in the upper test object. Perform.

At this time, the artificial intelligence (AI)-based machine learning method applied to the first and second artificial intelligence learning modules (330 and 340) and the third and fourth artificial intelligence learning modules (370 and 380) is, for example, a neural network (Neural Network). ), SVM (Support Vector Machine), MLP (Multi Layer Perception), and/or Deep Learning. It is desirable to use at least one artificial intelligence learning method.

Meanwhile, the modules described above, that is, the image segmentation module 350, the edge extraction module 360, the third artificial intelligence learning module 370, the fourth artificial intelligence learning module 380, and/or the upper subject Although the body abnormality detection module 390 is implemented separately and implemented to perform functions in each module, the present invention is not limited to this, and the machine vision control module 340 may be implemented to perform all the functions of the modules described above.

In addition, the power supply device 400 includes each of the above-mentioned devices, that is, the first and second line scan camera devices (100a and 100b), the first and second line lighting devices (200a and 200b), and the machine vision control device ( 300), third and fourth line scan camera devices (500a and 500b), third and fourth line lighting devices (600a and 600b), storage device (700), display device (750), communication device (800), It performs the function of supplying power necessary for the operation of the RFID tag reader device 850, the mileage measurement device 900, and/or the position detection device 950, and uses commercial alternating current (AC) for continuous power supply. ) It is desirable to implement a power source (e.g., AC 220V or 380V, etc.) converted to direct current (DC) and/or alternating current (AC) power, but is not limited to this and can also be implemented using a typical portable battery. there is.

Additionally, the power supply device 400 may include a power management unit (not shown) that protects components from external power shock and outputs a constant voltage. The power management unit may include an ESD (Electro Static Damage) protector, a power detector, a rectifier, and a power breaker.

Here, the ESD protector is configured to protect electrical components from static electricity or sudden power shock. The power detector is configured to send a cut-off signal to the power breaker when a voltage outside the allowable voltage range is introduced, and to transmit a step-up or step-down signal to the rectifier according to voltage changes within the allowable voltage range. The rectifier is configured to perform a step-up or step-down rectification operation according to the signal from the power detector to minimize fluctuations in input voltage and supply a constant voltage. The power breaker is configured to cut off power supplied from the battery according to a cutoff signal transmitted from the power detector.

Additionally, the third and fourth line scan camera devices 500a and 500b are installed on both sides of the upper part of the train 1, respectively, and provide images on both sides of the upper part of the train 1 on a line-by-line basis (i.e., details of the upper part of the railroad track facility). It performs the function of acquiring an image with a two-dimensional image by scanning images of parts.

In addition, the third and fourth line scan camera devices 500a and 500b have the same structure as the above-described first and second line scan camera devices 100a and 100b except for the installation location, so the detailed description is as described above. Please refer to the description of the first and second line scan camera devices 100a and 100b.

The third and fourth line lighting devices (600a and 600b) are installed on both sides of the upper part of the train (1), and emit light in the form of a line with the same optical axis as the third and fourth line scan camera devices (500a and 500b). Performs an investigative function.

In addition, the third and fourth line lighting devices (600a and 600b) have the same structure as the above-described first and second line lighting devices (200a and 200b) except for the installation location, so the detailed description is provided in the above-mentioned first and second line lighting devices (200a and 200b). and the description of the second line lighting devices 200a and 200b.

The storage device 700 is connected to the machine vision control device 300, and the train 1 obtained from the first and second line scan camera devices 100a and 100b under the control of the machine vision control device 300. Store the images on both sides of the lower part of the train, the images on both sides of the upper part of the train 1 acquired from the third and fourth line scan camera devices 500a and 500b, and/or the abnormality detection result information of the lower object and the upper object. performs the function of

Since this storage device 700 must acquire and store high-resolution, large-capacity images that move at high speed, it is preferably implemented as a SSD (Solid State Drive) type large-capacity memory device in consideration of the bandwidth of image data, but is not limited to this. For example, Flash Memory type, Hard Disk type, Multimedia Card Micro type, card type memory (e.g. SD or XD memory, etc.), RAM ( Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), magnetic memory, magnetic disk , and/or may include at least one type of storage medium among optical disks.

The display device 750 is connected to the machine vision control device 300, and the train 1 obtained from the first and second line scan camera devices 100a and 100b under the control of the machine vision control device 300. Displays images on both sides of the lower part of the train, images on both sides of the upper part of the train 1 acquired from the third and fourth line scan camera devices 500a and 500b, and/or abnormality detection result information on the lower object and the upper object. Performs the function of displaying information on the screen.

These display devices 750 include, for example, a liquid crystal display (LCD), a light emitting diode (LED), a thin film transistor-liquid crystal display (TFT LCD), and an organic light emitting diode. (Organic Light Emitting Diode, OLED), Flexible Display, Plasma Display Panel (PDP), Alternate Surface Lighting (ALiS), Digital Light Source Processing (DLP), Liquid Crystal of Silicon (LCoS), Surface Conductivity type electron-emitting device display (SED), field emission display (FED), laser TV (quantum dot laser, liquid crystal laser), optoelectric liquid display (FLD), interferometric modulator display (iMoD), thick film dielectric display (TDEL), It may include, but is not limited to, at least one of quantum dot display (QD-LED), telescopic pixel display (TPD), organic light emitting transistor (OLET), laser fluorescent display (LPD), and three-dimensional display (3D display). Any data that can display at least one of external image, text, and/or document data may be included.

The communication device 800 is connected to the machine vision control device 300, and the train 1 obtained from the first and second line scan camera devices 100a and 100b under the control of the machine vision control device 300. Images on both sides of the lower part of the train, images on both sides of the upper part of the train 1 acquired from the third and fourth line scan camera devices 500a and 500b, and/or information on abnormality detection results of the lower object and the upper object through a communication network. It performs the function of transmitting to an external terminal and/or server 20 through wired and/or wireless communication through (10).

At this time, the communication network 10 is a communication network that is a high-speed backbone network of a large communication network capable of high-capacity, long-distance voice and data services, and includes Wi-Fi, WiGig, and It may be a next-generation wireless communication network including Wireless Broadband Internet (Wibro) and Wimax (World Interoperability for Microwave Access, Wimax).

The Internet includes the TCP/IP protocol and various services existing at its upper layer, such as HTTP (Hyper Text Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), It refers to a global open computer network structure that provides SNMP (Simple Network Management Protocol), NFS (Network File Service), NIS (Network Information Service), etc., and external terminals and/or servers (20) communicate with communication devices (800). ) provides an environment that allows access to. Meanwhile, the Internet may be wired or wireless Internet, or it may also be a core network integrated with a wired public network, wireless mobile communication network, or portable Internet.

If the communication network 10 is a mobile communication network, it may be a synchronous mobile communication network or an asynchronous mobile communication network. An example of the asynchronous mobile communication network may be a WCDMA (Wideband Code Division Multiple Access) communication network. In this case, although not shown in the drawing, the mobile communication network may include, for example, a Radio Network Controller (RNC). Meanwhile, although the WCDMA network was used as an example, it may be a next-generation communication network such as a cellular-based 3G network, an LTE network, a 4G network, and a 5G network, and other IP-based IP networks. This communication network 10 serves to mutually transfer signals and data between an external terminal and/or server 20 and the communication device 800.

Meanwhile, the communication network 10 may use at least one short-range communication method among, for example, Bluetooth, ZigBee, Beacon, Ultra Wideband (UWB), Radio Frequency Identification (RFID), and/or infrared (IR) communication. Communication methods can also be used.

The RFID tag reader device 850 is mounted inside and/or outside the train (1) and reads the reference position information from the RFID tag installed at each preset reference position on the track of the relevant railroad track facility. performs its function.

The mileage measuring device 900 is mounted inside and/or outside the train 1 and performs the function of measuring mileage information of the train 1. This traveling distance measuring device 900 preferably includes a typical taco meter for collecting movement signals of the train 1.

The position detection device 950 is mounted inside and/or outside the train 1, and uses the reference position information of the RFID tag reader device 850 and the mileage measurement information of the mileage measurement device 900 to detect the train. (1) It performs the function of automatically detecting the current location information continuously.

In addition, the external terminal and/or server 20 provides the user's specific input for user setting of LED lighting control according to the user's request through a pre-installed specific application, that is, a user setting-related application for LED lighting control. It performs the function of transmitting a signal to the machine vision control device 300 by wire and/or wirelessly through the communication network 10.

In addition, the external terminal and/or server 20 displays the first and second line scan camera devices 100a and 100b transmitted from the communication device 800 through a specific pre-installed application, that is, an application related to railway track facility detection. Images on both sides of the lower part of the train (1) acquired from, images on both sides of the upper part of the train (1) acquired from the third and fourth line scan camera devices 500a and 500b, and/or the lower object and the upper object. A service may be provided to search for abnormality detection result information and/or display it on a display screen.

Meanwhile, the external terminal and/or server 20 applied in one embodiment of the present invention is a smart phone, smart pad, or smart note that communicates through wireless Internet or mobile Internet. It is preferable that it consists of at least one mobile terminal device, and in addition, a personal PC, a laptop PC, a Palm PC, a mobile play-station, a DMB (Digital Multimedia Broadcasting) phone with a communication function, a tablet PC, It can comprehensively refer to all wired and wireless home appliances/communication devices having a user interface for accessing the communication network 10 and the communication device 800, such as an iPad.

As shown in FIG. 10, these external terminals and/or servers 20 include a wireless communication module 21, an audio/video (A/V) input module 22, a user input module 23, and a sensing module. (24), an output module (25), a storage module (26), an interface module (27), a terminal control module (28), and a power module (29). Meanwhile, since the components shown in FIG. 10 are not essential, the external terminal and/or server 20 may have more or fewer components.

Hereinafter, a detailed look at the components of the external terminal and/or server 20 is as follows.

The wireless communication module 21 may include one or more modules that enable wireless communication between an external terminal and/or server 20 and the communication device 800. For example, the wireless communication module 21 may include a broadcast reception module 21a, a mobile communication module 21b, a wireless Internet module 21c, a short-range communication module 21d, and a location information module 21e.

The broadcast reception module 21a receives broadcast signals (e.g., TV broadcast signals, radio broadcast signals, data broadcast signals, etc.) and/or broadcasts from an external broadcast management server through various broadcast channels (e.g., satellite channels, terrestrial channels, etc.). Receive relevant information.

The mobile communication module 21b transmits and receives wireless signals with at least one of a base station, a user terminal, and/or a server 20 on a mobile communication network. The wireless signal may include various types of data resulting from voice call signals, video call signals, or text/multimedia message transmission and reception.

The wireless Internet module 21c is a module for wireless Internet access and may be built into or external to the external terminal and/or server 20. For example, WLAN (Wi-Fi), Wibro, Wimax, HSDPA, LTE, etc. may be used as the wireless Internet technology.

The short-range communication module 21d is a module for short-range communication, for example, Bluetooth communication, ZigBee communication, UWB (Ultra Wideband) communication, RFID (Radio Frequency Identification) communication, or infrared (IR) communication. It can be used.

The location information module 21e is a module for confirming or obtaining the location of the external terminal and/or server 20, and uses GPS (Global Position System), etc. to determine the current location of the external terminal and/or server 20. Information can be obtained.

Meanwhile, under the control of the terminal control module 28, the communication device 800 uses a specific application program stored in the storage module 26 through the wireless communication module 21 and/or the wired communication module (not shown). ) and data transmission and reception can be performed.

The A/V input module 22 is a module for inputting audio signals or video signals, and may basically include a camera unit 22a and a microphone unit 22b. The camera unit 22a processes image frames such as still images or moving images obtained by an image sensor in video call mode or shooting mode. The microphone unit 22b receives an external sound signal through a microphone in a call mode, recording mode, voice recognition mode, etc., and processes it into electrical voice data.

The user input module 23 is a module that generates input data for controlling the operation of an external terminal and/or server 20, and in particular, data management information displayed through the display unit 25a of the output module 25. It performs the function of inputting a selection signal for one of the following, for example, in the form of a touch panel (static pressure/electrostatic) input by the user's touch or a separate input device (e.g., key pad dome switch, jog wheel, jog switch). etc.) can be entered using.

The sensing module 24 monitors the open/closed status of the external terminal and/or server 20, the location of the external terminal and/or server 20, the presence or absence of user contact, the user's touch action on a specific part, the external terminal and/or / Or detect the current state of the external terminal and / or server 20, such as the direction of the server 20, acceleration / deceleration of the external terminal and / or server 20, etc., and detect the current state of the external terminal and / or server 20 ) generates a sensing signal to control the operation. This sensing signal is transmitted to the terminal control module 28 and can become the basis for the terminal control module 28 to perform a specific function.

The output module 25 is a module for generating output related to vision, hearing, or tactile sensation, and basically includes a display unit 25a, an audio output unit 25b, an alarm unit 25c, and a haptic unit 25d. You can.

The display unit 25a is for displaying and outputting information processed by the external terminal and/or server 20. For example, when the external terminal and/or server 20 is in a call mode, the call-related UI (User Interface) or GUI (Graphical User Interface), and in video call mode or shooting mode, the captured and/or received video, UI, or GUI is displayed.

The audio output unit 25b may output audio data received from the wireless communication module 21 or stored in the storage module 26, for example, in call signal reception, communication mode or recording mode, voice recognition mode, broadcast reception mode, etc. .

The alarm unit 25c may output a signal to notify the occurrence of an event in an external terminal and/or the server 20. Examples of events occurring in the external terminal and/or server 20 include call signal reception, message reception, key signal input, and touch input.

The haptic unit 25d generates various tactile effects that the user can feel. A representative example of a tactile effect generated by the haptic unit 25d is vibration. The intensity and pattern of vibration generated by the haptic portion 25d can be controlled.

The storage module 26 can store programs for operating the terminal control module 28, and can also temporarily store input/output data (eg, phone book, messages, still images, videos, etc.).

In addition, the storage module 26 can store data on various patterns of vibration and sound output upon touch input on the touch screen, and can be used for specific applications (e.g., applications related to user settings for LED lighting control, detection of railway track facilities). (related applications, etc.) programs can be saved.

In addition, the storage module 26 can store source data for forming specific application-related information. The specific application-related data may be in the form of images and sounds, and the specific application-related data may be stored in the form of video and sound. The creation process and results can also be saved.

These storage modules 26 include flash memory type, hard disk type, multimedia card micro type, card type memory (e.g. SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, and PROM. , may include at least one type of storage medium among magnetic memory, magnetic disk, and optical disk.

The interface module 27 serves as a passageway for all external devices connected to the external terminal and/or server 20. The interface module 27 transmits data or receives power from an external device and transmits it to each component inside the external terminal and/or server 20, or transmits data inside the external terminal and/or server 20 to the external device. Let it be transmitted to your device.

The terminal control module 28 typically controls the overall operation of the external terminal and/or server 20, and performs related control and processing for, for example, voice calls, data communications, video calls, and execution of various applications. do.

That is, the terminal control module 28 controls the execution of a specific application program stored in the storage module 26, requests the creation of specific application-related data through execution of the specific application program, and generates a specific application for this. It performs a control function to ensure that relevant data is provided.

In addition, the terminal control module 28 displays auxiliary elements including at least one of video, voice, or sound in the process of generating data related to a specific application desired by the user through execution of a specific application program. It performs a function of controlling output through at least one of the output devices (for example, the audio output unit 25b, the alarm unit 25c, the haptic unit 25d, etc.).

Additionally, the terminal control module 28 can constantly monitor the charging current and charging voltage of the battery unit 29a and temporarily store the monitoring values in the storage module 26. At this time, the storage module 26 preferably stores not only battery charging status information such as monitored charging current and charging voltage, but also battery specification information (product code, rating, etc.).

The power module 29 receives external power and internal power under the control of the terminal control module 28 and supplies power necessary for the operation of each component. The power module 29 operates by supplying power from the built-in battery unit 29a to each component, and the battery can be charged using a charging terminal (not shown).

Various embodiments described herein may be implemented, for example, in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). , may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases such embodiments may be implemented by the terminal control module 28.

According to software implementation, embodiments such as procedures or functions may be implemented with a separate software module that performs at least one function or operation. Software code can be implemented by a software application written in an appropriate programming language. Additionally, the software code can be stored in the storage module 26 and executed by the terminal control module 28.

If the external terminal and/or server 20 is comprised of a smartphone, the smartphone, unlike a general cell phone (so-called feature phone), can freely download various application programs desired by the user. It is a phone based on an open operating system that can be used and deleted. In addition to commonly used functions such as voice/video calls and internet data communication, all mobile phones with mobile office functions or Internet without voice call functions are available. It is desirable to understand it as a communication device that includes all accessible Internet phones or tablet PCs.

As such, smartphones use an open operating system, so unlike mobile phones with a closed operating system, users can install and manage various application programs at will.

Although a preferred embodiment of the real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm according to the present invention has been described above, the present invention is not limited thereto, and includes the scope of the patent claims, detailed description of the invention, and It is possible to implement various modifications within the scope of the attached drawings, and these also belong to the present invention.

100a and 100b: first and second line scan camera devices,
200a and 200b: first and second line lighting devices,
300: machine vision control device,
400: power supply,
500a and 500b: third and fourth line scan camera devices,
600a and 600b: 3rd and 4th line lighting devices,
700: storage device,
750: display device,
800: communication device,
850: RFID tag reader device,
900: Odometer measuring device,
950: Position detection device

Claims (15)

First and second line scan camera devices are installed on both sides of the lower part of a train moving on the tracks of a railway track facility and acquire two-dimensional images by scanning images on both sides of the lower part of the train on a line-by-line basis;
Third and fourth line scan camera devices are installed on both sides of the upper part of the train and acquire two-dimensional images by scanning images on both sides of the upper part of the train on a line-by-line basis;
first and second line lighting devices installed on both sides of the lower part of the train and emitting light in the form of a line with the same optical axis as the first and second line scan camera devices;
third and fourth line lighting devices installed on both sides of the upper part of the train and emitting light in the form of a line with the same optical axis as the third and fourth line scan camera devices; and
The first and second line scan cameras are synchronized with the trigger signal of the preset camera shutter so that the camera shutters of the first and second line scan camera devices open and the lights of the first and second line lighting devices emit light. In addition to controlling the operation of the device and the first and second line lighting devices, the camera shutters of the third and fourth line scan camera devices are opened and the lights of the third and fourth line lighting devices emit light. Controls the operations of the third and fourth line scan camera devices and the third and fourth line lighting devices, receives images from both lower sides of the train obtained from the first and second line scan camera devices, and sets preset artificial Through an intelligence (AI)-based machine learning method, the appearance and relative position or geometric relationship of the lower test object are repeatedly learned to detect the lower test object, and anomaly detection of the lower test object is performed, and the third And receiving the images on both upper sides of the train obtained from the fourth line scan camera device, splitting the image frames on both upper sides of the train through a preset frame splitting technique to detect the upper inspection object, and dividing the image frames on both sides of the upper part of the divided train In order to determine the position of the upper object to be inspected in the upper two image frames, edge components of the upper detailed parts of the relevant railway track facility are extracted, and then based on the edge components of the extracted upper detailed parts of the relevant railway track facility Through a pre-established artificial intelligence (AI)-based machine learning method, the geometric relationship between the upper detailed parts of the railway track facility is repeatedly learned to detect the upper inspection object and to detect abnormalities in the upper inspection object. Including a machine vision control device that
The first and second line lighting devices and the third and fourth line lighting devices each include a plurality of LED light-emitting channels connected in parallel and each having at least one LED light-emitting element,
The machine vision control device outputs each LED light emission channel provided in the first and second line lighting devices and the third and fourth line lighting devices according to a user's specific input signal output according to the user's request. Setting the level target value, setting the output channel selection and order of each LED emitting channel for high-speed switching control, and setting the clock generation to generate the operation signal for high-speed switching, and setting the output channel selection and sequence for each LED emitting channel set above. In order to control the brightness of each LED light-emitting element provided in each LED light-emitting channel based on the output level target value, the output voltage target value of each LED light-emitting channel is generated and output, and the output channel of each LED light-emitting channel set above is selected and Each LED provided in the first and second line lighting devices and the third and fourth line lighting devices emits light according to the output voltage target value of each LED light emitting channel in synchronization with the sequence setting information and clock information. Controls high-speed switching of channels, outputs specific input signals for the user according to the user's needs based on a graphical user interface (GUI), and uses FPGA (Field Programmable Gate Array) based hardware programming for the high-speed switching function. High-speed control is performed using, and first and second line scan camera devices and third and fourth line scan cameras installed near the first and second line lighting devices and the third and fourth line lighting devices, respectively. When the idle period of the shooting operation of the camera device is detected and the shooting operation signal of the first and second line scan camera devices and the third and fourth line scan camera devices are not input for more than a preset shooting operation reference time determines the state to be in an idle state, performs an automatic reset function to remove the state during the shooting operation and initialize it to the initial input state so that it remains in the initial state after the determined idle state, and performs an automatic reset function to initialize the state to the initial input state, and output of each LED light emitting channel set above. Controls to store at least one setting value among the level target setting value, the setting value for output channel selection and order of each LED emitting channel, and the setting value for clock generation in a separate storage module, In order to protect each LED light-emitting element from damage caused by continuous turning on of each LED light-emitting element provided in each LED light-emitting channel of the first and second line lighting devices and the third and fourth line lighting devices, the first and fourth line lighting devices When the high-speed switching operation signal for each LED light-emitting channel of the second line lighting device and the third and fourth line lighting devices is changed, the first and second line lighting devices are illuminated only at a certain operation reference time less than or equal to the preset operation reference time. A real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that each LED light-emitting channel of the third and fourth line lighting devices is controlled to perform high-speed switching operations.
According to claim 1,
The machine vision control device,
Images on both sides of the lower part of the train obtained from the first and second line scan camera devices are provided, and the appearance and relative position or geometric relationship of the lower object to be inspected are determined through a preset artificial intelligence (AI)-based machine learning method. A first artificial intelligence learning module that constructs a first artificial intelligence learning model for detecting the corresponding lower inspection object through repeated learning;
Inferring and analyzing the appearance and location characteristics of the lower test object for each normal/abnormal characteristic item for the lower test object detected using the first artificial intelligence learning model constructed from the first artificial intelligence learning module, and the inference And after classifying the type of abnormality for the lower test object based on the appearance and location characteristic information of the analyzed lower test object, the corresponding lower test object is classified through a preset artificial intelligence (AI)-based machine learning method. A second artificial intelligence learning module that constructs a second artificial intelligence learning model for detecting abnormalities in the corresponding lower inspection object by performing repeated learning appropriate for the classified abnormality occurrence type;
Using the second artificial intelligence learning model built from the second artificial intelligence learning module, infer and analyze the characteristics of the type of abnormality in the lower test object for each characteristic item of the type of abnormality in the lower test object, and A lower test object abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the lower test object based on the inferred and analyzed characteristic information about the type of abnormality in the lower test object; and
The first and second line scan cameras are synchronized with the trigger signal of the preset camera shutter so that the camera shutters of the first and second line scan camera devices open and the lights of the first and second line lighting devices emit light. A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, comprising a device and a machine vision control module that controls the operation of the first and second line lighting devices.
delete According to claim 1,
The machine vision control device,
Images on both sides of the lower part of the train obtained from the first and second line scan camera devices are provided, and the appearance and relative position or geometric relationship of the lower object to be inspected are determined through a preset artificial intelligence (AI)-based machine learning method. A first artificial intelligence learning module that constructs a first artificial intelligence learning model for detecting the corresponding lower inspection object through repeated learning;
Inferring and analyzing the appearance and location characteristics of the lower test object for each normal/abnormal characteristic item for the lower test object detected using the first artificial intelligence learning model constructed from the first artificial intelligence learning module, and the inference And after classifying the type of abnormality for the lower test object based on the appearance and location characteristic information of the analyzed lower test object, the corresponding lower test object is classified through a preset artificial intelligence (AI)-based machine learning method. A second artificial intelligence learning module that constructs a second artificial intelligence learning model for detecting abnormalities in the corresponding lower inspection object by performing repeated learning appropriate for the classified abnormality occurrence type;
Using the second artificial intelligence learning model built from the second artificial intelligence learning module, infer and analyze the characteristics of the type of abnormality in the lower test object for each characteristic item of the type of abnormality in the lower test object, and A lower test object abnormality detection module that automatically detects whether or not an abnormality has occurred and the type of the lower test object based on the inferred and analyzed characteristic information about the type of abnormality in the lower test object;
An image segmentation module that receives images from both upper sides of the train obtained from the third and fourth line scan camera devices and divides the upper side image frames of the train through a preset frame division technique to detect the upper inspection object;
An edge extraction module for extracting edge components for upper detailed parts of the railway track facility in order to determine the position of the upper object to be inspected from the image frames on both upper sides of the train divided from the image segmentation module;
Geometric relationship to the upper detailed parts of the relevant railway track facility through a preset artificial intelligence (AI)-based machine learning method based on the edge components of the upper detailed parts of the relevant railway track facility extracted from the edge extraction module. A third artificial intelligence learning module that constructs a third artificial intelligence learning model for detecting the corresponding upper inspection object by repeatedly learning the relationship;
Infer and analyze the appearance and location characteristics of the upper test object for each normal/abnormal feature item detected using the third artificial intelligence learning model built from the third artificial intelligence learning module, and infer the above. And after classifying the type of abnormality for the upper test object based on the analyzed appearance and location characteristic information of the upper test object, the upper test object is classified through a preset artificial intelligence (AI)-based machine learning method. A fourth artificial intelligence learning module that constructs a fourth artificial intelligence learning model for detecting abnormalities in the upper inspection object by performing repeated learning appropriate for the classified type of abnormality occurrence;
Using the fourth artificial intelligence learning model built from the fourth artificial intelligence learning module, infer and analyze the characteristics of the abnormality type of the upper inspection object for each characteristic item of the abnormality occurrence type for the upper inspection object, and An upper inspection object abnormality detection module that automatically detects whether or not an abnormality occurs and the type of abnormality in the upper inspection object based on the inferred and analyzed characteristic information about the type of abnormality in the upper inspection object; and
The first and second line scan cameras are synchronized with the trigger signal of the preset camera shutter so that the camera shutters of the first and second line scan camera devices open and the lights of the first and second line lighting devices emit light. In addition to controlling the operation of the device and the first and second line lighting devices, the camera shutters of the third and fourth line scan camera devices are opened and the lights of the third and fourth line lighting devices emit light. It includes third and fourth line scan camera devices and a machine vision control module that controls the operation of the third and fourth line lighting devices,
The artificial intelligence (AI)-based machine learning method applied to the first to fourth artificial intelligence learning modules includes neural network, SVM (Support Vector Machine), MLP (Multi Layer Perception), and deep learning. ) A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that it consists of at least one artificial intelligence learning method.
According to claim 1,
The machine vision control device detects the upper inspection object using a preset machine vision algorithm based on the image frames on both sides of the upper part of the divided train, and then performs machine learning based on preset artificial intelligence (AI). Detect abnormalities in the upper test object through the method,
The preset machine vision algorithm consists of at least one algorithm among pattern matching, point/line/plane fitting, edge, color, and location algorithms. A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm.
According to claim 1,
The upper detailed parts of the relevant railway track facilities include a pantograph to contact the tram line of the relevant train and collect electrical energy from the relevant tram line, and a messenger wire to support the load of the relevant tram line and maintain the erection height. and a dropper, a moving bracket to support and cushion the force received by the tram line from the progress of the train, and a steady arm to share the load of the tram line and cushion the up-and-down vibration and pushing shock. An artificial intelligence-based machine vision algorithm characterized by consisting of at least one of an arm), a T-bar for fixing the tram line to the ceiling of a tunnel or underground space, and an insulating insulator for electrical insulation. A real-time, high-speed detection system for railway track facilities.
According to claim 1,
The machine vision control device includes LED light emitting elements provided in the first and second line lighting devices during the camera shutter exposure times of the first and second line scan camera devices and the third and fourth line scan camera devices. The output intensity of the LED light emitting elements provided in the third and fourth line lighting devices is applied at a preset over-driving constant current value in the form of a pulse signal greater than the rated continuous maximum value to obtain high brightness. Controlling the operations of the first and second line lighting devices and the third and fourth line lighting devices, respectively, using a constant current overdrive control method using control,
The over-driving constant current value considers the cooling time of the LED light-emitting elements provided in the first and second line lighting devices and the LED light-emitting elements provided in the third and fourth line lighting devices between each pulse signal, respectively. A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that it is set up.
According to claim 1,
The upper object to be inspected is a pantograph that contacts the tram line of the train and collects electrical energy from the tram line, and supplies power to the train through the pantograph, and is placed on the ceiling of the tunnel or underground space. A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that it consists of at least one of the T-Bars, which are fixing means used in the device for fixing the tram line of the train.
delete According to claim 1,
The user's specific input signal output according to the user's request is provided through an external terminal or server,
The external terminal or server is a machine vision control device that receives the user's specific input signal for user setting of LED lighting control according to the user's request through a pre-installed user setting related application for LED lighting control by wired or wirelessly. A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, characterized by transmitting to.
According to claim 1,
Images on both sides of the lower part of the train obtained from the first and second line scan camera devices, images on both sides of the upper part of the train obtained from the third and fourth line scan camera devices, and the upper and lower objects to be inspected. A storage device for storing abnormality detection result information;
Images on both sides of the lower part of the train obtained from the first and second line scan camera devices, images on both sides of the upper part of the train obtained from the third and fourth line scan camera devices, and the upper and lower objects to be inspected. A display device that displays abnormality detection result information on a display screen; and
Images on both sides of the lower part of the train obtained from the first and second line scan camera devices, images on both sides of the upper part of the train obtained from the third and fourth line scan camera devices, and the upper and lower objects to be inspected. A communication device that transmits abnormality detection result information by wired or wireless communication method is further included,
The machine vision control device includes images on both lower sides of the train obtained from the first and second line scan camera devices, images on both upper sides of the train obtained from the third and fourth line scan camera devices, and images on both sides of the upper side of the train obtained from the first and second line scan camera devices. Control the abnormality detection result information of the inspected object and the corresponding lower inspected object into a database (DB) for each upper and lower side of the train and store it in the storage device, and retrieve the abnormality detection result information from the first and second line scan camera devices. The acquired images on both sides of the lower part of the train, the images on both upper sides of the train acquired from the third and fourth line scan camera devices, and the abnormality detection result information of the upper and lower objects to be inspected are displayed on the display screen. Controls the operation of the display device, and includes images on both lower sides of the train obtained from the first and second line scan camera devices, images on both upper sides of the train obtained from the third and fourth line scan camera devices, and Controlling the operation of the communication device so that the abnormality detection result information of the upper inspection object and the lower inspection object is transmitted to an external terminal or server through wired or wireless communication,
The external terminal or server displays images on both sides of the lower part of the train acquired from the first and second line scan camera devices transmitted from the communication device through a pre-installed railway track facility detection-related application, and the third and fourth An artificial intelligence-based machine that provides a service to search or display on a display screen images of both sides of the upper part of the train obtained from a line scan camera device, and abnormality detection result information of the upper and lower test objects. A real-time, high-speed detection system for railway track facilities using a vision algorithm.
According to claim 1,
The machine vision control device receives images of both lower sides of the train obtained from the first and second line scan camera devices, or images of both upper sides of the train obtained from the third and fourth line scan camera devices. After converting the lower or upper two sides of the train into a panoramic image using a preset image stitching technique, control the converted panoramic image to be displayed on the display screen. A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm.
According to claim 1,
An RFID tag reader device that is mounted inside/outside the train and reads the reference position information from the RFID tag installed at each preset reference position on the track of the relevant railroad track facility;
A mileage measuring device that is mounted inside/outside the train and measures the mileage information of the train; and
A position detection device is mounted inside/outside the train and continuously and automatically detects the current location information of the train using the reference position information of the RFID tag reader device and the mileage measurement information of the mileage measurement device. Includes more,
The machine vision control device receives the current location information of the train detected from the position detection device, and based on this, for each frame of the image on both lower sides of the train obtained from the first and second line scan camera devices, Alternatively, the current location information of the train is mapped for each frame of the image on both sides of the upper part of the train obtained from the third and fourth line scan camera devices, and the current location information of the train is controlled to be stored in a separate storage device, and measured from the mileage measuring device. The driving distance measurement signal of the corresponding train is decomposed into a preset measurable cycle to generate a trigger signal for a camera shutter for video shooting, and the first and second line scan camera devices are based on the generated trigger signal for the camera shutter. A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that it controls the operation of the third and fourth line scan camera devices.
According to claim 1,
The machine vision control device detects the lower object to be inspected using a preset machine vision algorithm based on the images on both sides of the lower part of the train obtained from the first and second line scan camera devices, and then detects the lower object to be inspected using a preset machine vision algorithm. Detect abnormalities in the relevant lower test object using an artificial intelligence (AI)-based machine learning method,
The preset machine vision algorithm consists of at least one algorithm among pattern matching, point/line/plane fitting, edge, color, and location algorithms. A real-time, high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm.
According to claim 1,
The lower object to be inspected is a lower detailed part of the relevant railway track facility, and includes a plurality of sleepers arranged on the ground at regular intervals, and a pair of railroad ties fixed and installed side by side in a direction perpendicular to both upper sides of the plurality of sleepers. It consists of a lower detail part of at least one of a rail and a pan trolley for fixing the pair of railroad rails on the plurality of sleepers,
When the lower object to be inspected is a plurality of sleepers, the machine vision control device performs sleeper abnormality detection for damage to the sleeper,
When the lower object to be inspected is a pair of railway rails, the machine vision control device detects at least one rail abnormality among rail waviness wear, tongue rail wear, and rail head damage. performs,
When the lower object to be inspected is a pan troll, the machine vision control device is a real-time high-speed detection system for railway track facilities using an artificial intelligence-based machine vision algorithm, characterized in that it detects an abnormality in the pan troll being dropped.

Images