DEEP LEARNING AND LANGUAGE MODEL ENHANCED SYSTEM FOR WIND TURBINE MONITORING USING DISTRIBUTED FIBER OPTIC SENSING (DL-LM-DFOS) FIELD OF THE INVENTION [0001] This application relates generally to distributed fiber optic sensing (DFOS) systems, methods, structures, and related technologies. More particularly, it pertains to a deep learning and language model enhanced system for wind turbine monitoring using DFOS. BACKGROUND OF THE INVENTION [0002] Distributed fiber optic sensing (DFOS) systems, methods, and structures have found widespread utility in contemporary industry and society. Of particular importance, DFOS techniques have been used to usher in a new era of monitoring including perimeter security, traffic monitoring, and civil infrastructure monitoring. They can provide continuous, real-time measurements over long distances with high sensitivity, making them valuable tools for infrastructure monitoring and maintenance. [0003] Traditional wind turbine monitoring methods often rely on a limited set of sensors, such as vibration sensors, temperature sensors, and anemometers. These sensors provide valuable data, but their coverage is limited to specific points on the turbine. As a result, they fail to sense critical signs of potential issues in areas they don't cover. Additionally, these sensors often operate in isolation, meaning they don't share data with each other. This can lead to a lack of context in the data, making it harder – or impossible - to detect complex issues that affect multiple parts of the turbine. SUMMARY OF THE INVENTION [0004] The above problems are solved and an advance in the art is made according to aspects of the present disclosure directed to a deep learning and language model enhanced system and method for wind turbine monitoring using distributed fiber optic sensing (DL-LM-DFOS) which combines advantages of distributed fiber optic sensing with the power of deep learning and large language models.
[0005] In sharp contrast to the prior art, systems and methods according to aspects of the present disclosure automatically learn and extract useful features from raw sensor data, detect complex patterns indicating potential issues, and incorporate and learn from a wide range of data, including textual data such as maintenance logs, operational notes, or alarm messages. As a result, our inventive systems and methods provide comprehensive, efficient, and predictive monitoring of wind turbines. BRIEF DESCRIPTION OF THE DRAWING [0006] FIG. 1(A) and FIG. 1(B) are schematic diagrams showing an illustrative prior art uncoded and coded DFOS systems. [0007] FIG. 2 is a schematic diagrams showing illustrative distributed learning and language model enhanced system for wind turbine monitoring using distributed fiber optic sensing (DL-LM-DFOS) according to aspects of the present disclosure. [0008] FIG.3 is a schematic flow diagram showing illustrative DL-LM-DFOS component flow according to aspects of the present disclosure. [0009] FIG. 4 is a schematic flow diagram showing illustrative convolutional neural network and long short-term memory (CNN-LSTM) architecture according to aspects of the present disclosure. [0010] FIG. 5 is a schematic flow diagram showing illustrative large language model architecture according to aspects of the present disclosure. [0011] FIG. 6 is a schematic flow diagram showing illustrative predictive monitoring flow according to aspects of the present disclosure. [0012] FIG. 7 is a schematic diagram showing illustrative feature hierarchy of systems and methods according to aspects of the present disclosure.
[0013] FIG.8 is a schematic block diagram of an illustrative computing system that may be programmed with instructions that when executed produce the methods/algorithms according to aspects of the present invention. DETAILED DESCRIPTION OF THE INVENTION [0014] The following merely illustrates the principles of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its spirit and scope. [0015] Furthermore, all examples and conditional language recited herein are intended to be only for pedagogical purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor(s) to furthering the art and are to be construed as being without limitation to such specifically recited examples and conditions. [0016] Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure. [0017] Thus, for example, it will be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. [0018] Unless otherwise explicitly specified herein, the FIGs comprising the drawing are not drawn to scale. [0019] By way of some additional background, we note that distributed fiber optic sensing systems convert the fiber to an array of sensors distributed along the length of the fiber. In effect, the fiber becomes a sensor, while the interrogator generates/injects laser light energy into the fiber and senses/detects events along the fiber length.
[0020] As those skilled in the art will understand and appreciate, DFOS technology can be deployed to continuously monitor vehicle movement, human traffic, excavating activity, seismic activity, temperatures, structural integrity, liquid and gas leaks, and many other conditions and activities. It is used around the world to monitor power stations, telecom networks, railways, roads, bridges, international borders, critical infrastructure, terrestrial and subsea power and pipelines, and downhole applications in oil, gas, and enhanced geothermal electricity generation. Advantageously, distributed fiber optic sensing is not constrained by line of sight or remote power access and - depending on system configuration - can be deployed in continuous lengths exceeding 30 miles with sensing/detection at every point along its length. As such, cost per sensing point over great distances typically cannot be matched by competing technologies. [0021] Distributed fiber optic sensing measures changes in “backscattering” of light occurring in an optical sensing fiber when the sensing fiber encounters environmental changes including vibration, strain, or temperature change events. As noted, the sensing fiber serves as sensor over its entire length, delivering real time information on physical/environmental surroundings, and fiber integrity/security. Furthermore, distributed fiber optic sensing data pinpoints a precise location of events and conditions occurring at or near the sensing fiber. [0022] A schematic diagram illustrating the generalized arrangement and operation of a distributed fiber optic sensing system that may advantageously include artificial intelligence / machine learning (AI/ML) analysis is shown illustratively in FIG.1(A). With reference to FIG.1(A), one may observe an optical sensing fiber that in turn is connected to an interrogator. While not shown in detail, the interrogator may include a coded DFOS system that may employ a coherent receiver arrangement known in the art such as that illustrated in FIG.1(B). [0023] As is known, contemporary interrogators are systems that generate an input signal to the optical sensing fiber and detects / analyzes reflected / backscattered and subsequently received signal(s). The received signals are analyzed, and an output is generated which is indicative of the environmental conditions encountered along the length of the fiber. The backscattered signal(s) so received may result from reflections in the fiber, such as Raman backscattering, Rayleigh backscattering, and Brillion backscattering.
23028 [0024] As will be appreciated, a contemporary DFOS system includes the interrogator that periodically generates optical pulses (or any coded signal) and injects them into an optical sensing fiber. The injected optical pulse signal is conveyed along the length optical fiber. [0025] At locations along the length of the fiber, a small portion of signal is backscattered/reflected and conveyed back to the interrogator wherein it is received. The backscattered/reflected signal carries information the interrogator uses to detect, such as a power level change that indicates – for example - a mechanical vibration. [0026] The received backscattered signal is converted to electrical domain and processed inside the interrogator. Based on the pulse injection time and the time the received signal is detected, the interrogator determines at which location along the length of the optical sensing fiber the received signal is returning from, thus able to sense the activity of each location along the length of the optical sensing fiber. Classification methods may be further used to detect and locate events or other environmental conditions including acoustic and/or vibrational and/or thermal along the length of the optical sensing fiber. [0027] Of particular interest, distributed acoustic sensing (DAS) is a technology that uses fiber optic cables as linear acoustic sensors. Unlike traditional point sensors, which measure acoustic vibrations at discrete locations, DAS can provide a continuous acoustic / vibration profile along the entire length of the cable. This makes it ideal for applications where it's important to monitor acoustic / vibration changes over a large area or distance. [0028] Distributed acoustic sensing / distributed vibration sensing (DAS/DVS), also sometimes known as just distributed acoustic sensing (DAS), is a technology that uses optical fibers as widespread vibration and acoustic wave detectors. Like distributed temperature sensing (DTS), DAS/DVS allows for continuous monitoring over long distances, but instead of measuring temperature, it measures vibrations and sounds along the fiber. [0029] DAS/DVS operates as follows.
23028 [0030] Light pulses are sent through the fiber optic sensor cable. [0031] As the light travels through the cable, vibrations and sounds cause the fiber to stretch and contract slightly. [0032] These tiny changes in the fiber's length affect how the light interacts with the material, causing a shift in the backscattered light's frequency. [0033] By analyzing the frequency shift of the backscattered light, the DAS/DVS system can determine the location and intensity of the vibrations or sounds along the fiber optic cable. [0034] Similar to DTS, DAS/DVS offers several advantages over traditional point-based vibration sensors: High spatial resolution: It can measure vibrations with high granularity, pinpointing the exact location of the source along the cable; Long distances: It can monitor vibrations over large areas, covering several kilometers with a single fiber optic sensor cable; Continuous monitoring: It provides a continuous picture of vibration activity, allowing for better detection of anomalies and trends; Immune to electromagnetic interference (EMI): Fiber optic cables are not affected by electrical noise, making them suitable for use in environments with strong electromagnetic fields. [0035] DAS/DVS technology has a wide range of applications, including: Structural health monitoring: Monitoring bridges, buildings, and other structures for damage or safety concerns; Pipeline monitoring: Detecting leaks, blockages, and other anomalies in pipelines for oil, gas, and other fluids; Perimeter security: Detecting intrusions and other activities along fences, pipelines, or other borders; Geophysics: Studying seismic activity, landslides, and other geological phenomena; and Machine health monitoring: Monitoring the health of machinery by detecting abnormal vibrations indicative of potential problems. [0036] With the above in mind, we note once more that traditional wind turbine monitoring methods often rely on a limited set of sensors, such as vibration sensors, temperature sensors, and anemometers. These sensors provide valuable data, but their coverage is limited to specific points on
23028 the turbine. They can miss critical signs of potential issues in areas they don't cover. Additionally, these sensors often operate in isolation, meaning they don't share data with each other. This can lead to a lack of context in the data, making it harder to detect complex issues that affect multiple parts of the turbine. [0037] Moreover, traditional methods often lack the capabilities to effectively analyze the large amounts of data generated by these sensors. Important signs of potential issues, such as subtle changes in vibration patterns or temperature gradients, might be overlooked. These methods are also typically reactive, detecting issues after they occur, rather than being predictive. [0038] On the other hand, distributed fiber optic sensing offers several advantages that make it a superior solution for wind turbine monitoring. [0039] Comprehensive Coverage: Distributed fiber optic sensing can measure parameters like strain, temperature, and vibration at thousands of points along the fiber. This means they can provide comprehensive coverage of the entire turbine, including the blades, tower, nacelle, and even the internal components like the gearbox and generator. [0040] High-Dimensional Data: Distributed fiber optic sensors generate high-dimensional data, capturing detailed information about the turbine's condition in both space and time. This rich data can reveal complex patterns and subtle changes that might be missed by traditional sensors. [0041] Existing Infrastructure: Optical fiber cables are already present in many wind turbines, running up to the nacelle box. This existing infrastructure can be leveraged for distributed fiber optic sensing, making it a cost-effective solution. [0042] Immunity to Electromagnetic Interference: Unlike traditional electronic sensors, fiber optic sensors are immune to electromagnetic interference. This makes them more reliable in the electrically noisy environment of a wind turbine. [0043] Accordingly, those skilled in the art will readily understand and appreciate that our systems and methods according to the present disclosure namely, our “Deep Learning and Language Model
Enhanced System for Wind Turbine Monitoring Using Distributed Fiber Optic Sensing (DL-LM-DFOS)", combines advantages of distributed fiber optic sensing with the power of deep learning and large language models. Our systems and methods automatically learn and extract useful features from the raw sensor data, detect complex patterns indicating potential issues, and incorporate and learn from a wide range of data, including textual data such as maintenance logs, operational notes, or alarm messages. This makes our systems and methods a comprehensive, efficient, and predictive monitoring solution for wind turbines. [0044] Our inventive systems and methods therefore significantly advance the state of the art. They leverage comprehensive coverage and high-dimensional data provided by distributed fiber optic sensing, which utilizes existing optical fiber infrastructure in wind turbines, to capture detailed information about the turbine's condition in both space and time. [0045] Our inventive system and mehtod introduces a novel combination of deep learning and large language models to automatically learn and extract useful features from the raw sensor data, detect complex patterns indicating potential issues, and incorporate and learn from a wide range of data, including textual data such as maintenance logs, operational notes, or alarm messages. [0046] As will become apparent to those skilled in the art, our inventive system and method according to aspects of the present disclosure advantageously overcomes limitations of traditional monitoring methods, providing early and accurate detection of a wide range of potential issues, from cable strain and blade icing to generator faults and structural fatigue. By doing so, our system and method prevent equipment failure and downtime, improving the efficiency and reliability of wind turbines - thereby representing the significant advancement in wind farm monitoring technology. [0047] As we shall show and describe further, particularly inventive features of our system and methods include the following. [0048] FIG. 2 is a schematic diagrams showing illustrative distributed learning and language model enhanced system for wind turbine monitoring using distributed fiber optic sensing (DL-LM-DFOS) according to aspects of the present disclosure.
[0049] Distributed Fiber Optic Sensing: This feature provides comprehensive coverage of the entire wind turbine, capturing high-dimensional data about strain, temperature, vibration, and more at thousands of points along the fiber. This allows for the detection of complex patterns and subtle changes that might be missed by traditional sensors. [0050] Deep Learning: The system uses a combination of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks to process the spatial and temporal data from the sensors. This allows the system to automatically learn and extract useful features from the raw sensor data, eliminating the need for manual feature engineering. [0051] Large Language Model: The system incorporates a large language model to process and interpret any textual data associated with the sensor data, such as maintenance logs, operational notes, or alarm messages. This provides additional context that can help in diagnosing potential issues. [0052] Predictive Monitoring: By learning from the sensor data and associated textual data, the system can detect potential issues before they lead to failures. This predictive capability is a significant advancement over traditional reactive monitoring methods. [0053] Utilization of Existing Infrastructure: The system leverages the existing optical fiber cables in wind turbines for distributed fiber optic sensing. This makes it a cost-effective solution that can be easily implemented in existing wind farms. [0054] These and other inventive features work together to provide a comprehensive, efficient, and predictive monitoring solution for wind turbines, significantly advancing the state of the art in wind farm monitoring technology. [0055] FIG.3 is a schematic flow diagram showing illustrative DL-LM-DFOS component flow according to aspects of the present disclosure. [0056] Data Collection: During operation, the system begins by collecting data from two primary sources. The first is the existing optical fiber cables installed throughout the wind turbine. When combined with a DFOS system, these optical fiber cables act as sensors, generating high-dimensional data that
23028 captures detailed information about strain, temperature, vibration, and more at thousands of points along the length of the optical fiber. This comprehensive coverage is a significant advancement over traditional sensors, which may only monitor a limited number of points. [0057] The second source of data is textual data associated with wind turbine operation, such as maintenance logs, operational notes, or alarm messages. This data provides valuable context that can aid in the interpretation of the sensor data. [0058] Raw Sensor Data Processing: In this operation, raw sensor data, which is high-dimensional and time-series in nature, is fed into a deep learning system/module. This system/module is designed to handle such complex data effectively. The deep learning system/module includes two main components: Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. [0059] The CNN component is responsible for automatically extracting spatial features from the raw sensor data. CNNs are particularly effective at identifying patterns in spatial data, making them ideal for analyzing the high-dimensional data from the fiber optic sensors. The CNN component has multiple layers, including convolutional layers, pooling layers, and fully connected layers. The convolutional layers apply a series of filters to the data to identify patterns, the pooling layers reduce the dimensionality of the data while preserving the most important features, and the fully connected layers combine these features to make predictions. [0060] The LSTM component is responsible for analyzing the temporal aspects of the sensor data. LSTMs are a type of recurrent neural network that are capable of learning long-term dependencies in time- series data. This makes them ideal for analyzing the sensor data, which is collected over time and may contain temporal patterns that are indicative of potential issues. The combination of CNNs and LSTMs in this way is a novel aspect of this system. By using CNNs to extract spatial features and LSTMs to analyze temporal patterns, the system can effectively learn from the high-dimensional, time-series sensor data. [0061] FIG. 4 is a schematic flow diagram showing illustrative convolutional neural network and long short-term memory (CNN-LSTM) architecture according to aspects of the present disclosure.
23028 [0062] Textual Data Processing: In this step, the associated textual data, such as maintenance logs, operational notes, or alarm messages, is inputted into a large language model. This model is a type of transformer-based neural network, specifically designed to understand and generate human-like text. [0063] The large language model processes the textual data in the following ways. [0064] Tokenization: The textual data is first broken down into smaller pieces, or tokens. These tokens can be as short as one character or as long as one word. [0065] Embedding: Each token is then mapped to a high-dimensional vector that represents it in a way the model can understand. This process is known as embedding. [0066] Contextual Understanding: The model processes the sequence of embedded tokens, taking into account the order of the tokens and the relationships between them. This allows the model to understand the context of each token within the text. [0067] Interpretation: The model uses its understanding of the text's context to interpret the text, providing additional context that can help in diagnosing potential issues. [0068] FIG. 5 is a schematic flow diagram showing illustrative large language model architecture according to aspects of the present disclosure. [0069] One particularly distinguishing aspect of our inventive system and method results from the domain-specific fine-tuning of the large language model. Specifically, the model is fine-tuned on a dataset of wind turbine maintenance logs and operational notes. This allows the model to better understand the specific language and terminology used in this domain, improving its ability to interpret the textual data and provide useful context for diagnosing potential issues. This domain- specific fine-tuning of the large language model, combined with the deep learning module (CNN + LSTM) and distributed fiber optic sensing data, provides a comprehensive, efficient, and predictive monitoring solution for wind turbines. This represents a significant advancement in wind farm monitoring technology.
23028 [0070] Data Storage: The processed sensor data and interpreted textual data are then stored in separate data storage systems. This allows the data to be easily accessed and analyzed in the future. [0071] Predictive Monitoring: The stored data is provided to a predictive monitoring module, which analyzes the data to detect potential issues. By learning from the sensor data and associated textual data, the system can detect potential issues before they lead to failures. The processed sensor data (output from the CNN-LSTM model) and the interpreted textual data (output from the large language model) are inputted into the feature extraction process. The feature extraction process extracts relevant features from both types of data. The extracted features are then used to make a prediction about potential issues. If a potential issue is predicted, an alert is triggered. [0072] FIG. 6 is a schematic flow diagram showing illustrative predictive monitoring flow according to aspects of the present disclosure. [0073] Issue Detection: If the predictive monitoring module detects a potential issue, it alerts the relevant personnel or systems. This allows for early intervention, potentially preventing equipment failure and downtime [0074] FIG. 7 is a schematic diagram showing illustrative feature hierarchy systems and methods according to aspects of the present disclosure. [0075] FIG.8 is a schematic block diagram of an illustrative computing system that may be programmed with instructions that when executed produce the methods/algorithms according to aspects of the present invention. [0076] As may be immediately appreciated, such a computer system may be integrated into another system such as a router and may be implemented via discrete elements or one or more integrated components. The computer system may comprise, for example, a computer running any of a number of operating systems. The above-described methods of the present disclosure may be implemented on the computer system 800 as stored program control instructions.
23028 [0077] Computer system 800 includes processor 810, memory 820, storage device 830, and input/output structure 840. One or more input/output devices may include a display 845. One or more busses 850 typically interconnect the components, 810, 820, 830, and 840. Processor 810 may be a single or multi core. Additionally, the system may include accelerators etc., further comprising the system on a chip. [0078] Processor 810 executes instructions in which embodiments of the present disclosure may comprise steps described in one or more of the Drawing figures. Such instructions may be stored in memory 820 or storage device 830. Data and/or information may be received and output using one or more input/output devices. [0079] Memory 820 may store data and may be a computer-readable medium, such as volatile or non- volatile memory. Storage device 830 may provide storage for system 800 including for example, the previously described methods. In various aspects, storage device 830 may be a flash memory device, a disk drive, an optical disk device, or a tape device employing magnetic, optical, or other recording technologies. [0080] Input/output structures 840 may provide input/output operations for system 800. [0081] While we have presented our inventive concepts and description using specific examples, our invention is not so limited. Accordingly, the scope of our invention should be considered in view of the following claims.