CN113076655A - Multi-source heterogeneous oil consumption data feature extraction and fusion method - Google Patents

Abstract

The invention discloses a method for extracting and fusing characteristics of multi-source heterogeneous fuel consumption data, which comprises the following steps of: preprocessing the oil consumption data; carrying out time domain consistency operation on the various preprocessed oil consumption data; extracting statistical characteristic vectors and time sequence characteristic vectors of each frame data; fusing the statistical feature vector and the time series feature vector; and taking the fused features as vectors as input of a machine learning method. The method applies framing operation to the ship oil consumption data, and extracts statistical characteristics and time series characteristics; then, the statistical features and the time series features are fused. The method can process a large amount of existing sensor oil consumption data, and greatly improves the data volume for oil consumption modeling. According to the invention, the quality of the oil consumption data is greatly improved by carrying out feature extraction and fusion on the multi-source heterogeneous oil consumption data, so that the accuracy of oil consumption modeling is improved.

Description

Multi-source heterogeneous oil consumption data feature extraction and fusion method

Technical Field

The invention belongs to the technical field of multi-source heterogeneous data fusion, and particularly relates to a multi-source heterogeneous oil consumption data feature extraction and fusion method.

Background

The ship oil consumption evaluation mainly comprises two parts of data processing and evaluation modeling. The data processing is a key technology in the oil consumption evaluation of the ship, and the evaluation precision is directly influenced by the effect of the data processing. In marine practice, ship fuel consumption data may be divided into cabin log data and sensor acquisition data. Cabin log data are manually filled by crews according to a fixed format at a specified time, and data errors inevitably exist; and the sampling period is longer, and the oil consumption condition of the ship is not accurately depicted. Therefore, most ship fuel consumption estimation models are modeled by using sensor data.

With the continuous development of sensor technology, a large number of sensor devices, such as a doppler log, a barometer, a gps (global positioning system) global positioning system, an automatic Identification system (ais), and the like, are installed on modern ships, and these sensors can accurately measure and record ship state and environmental information related to oil consumption. However, the sampling frequency of the sensor varies according to the manufacturer and the manufacturing standard of the sensor. Thus, sensor-based ship fuel consumption data may be considered multi-source heterogeneous data. Before using these data to model ship fuel consumption, feature extraction and fusion must be performed on them.

In prior studies, Dario and Antonio have redefined variables such as wind direction, displacement, etc., for sensor-based fuel consumption data. The different characteristics are then selected to ensure that the correct characteristics are used for the assessment of fuel consumption of the vessel. Jianqin Zheng and Haoran Zheng regularize the oil consumption data, and accelerate the convergence speed and precision of the model. Trodden and Murphy reject dirty data using a Kalman filter. And removing the data exceeding the normal operation area of the ship by Lokuralute and Mo, normalizing the data, and scaling the variables of different scales to the same scale. Mueller-bau et al rejected singular values in the data using MATLAB software and analyzed the importance of different variables. The method comprises the steps that the brain conducts statistical analysis on oil consumption data, the data distribution condition of the oil consumption data is analyzed, correlation characteristics among different characteristics are researched, and then a Gaussian mixture model and a principal component analysis algorithm are used for dividing a working area of a ship host. According to the existing oil consumption data processing method, a large amount of existing sensor data cannot be used for oil consumption modeling, and the oil consumption evaluation accuracy is generally low.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a multi-source heterogeneous fuel consumption data feature extraction and fusion method, which can be used for modeling the fuel consumption of a ship by using a large amount of existing sensor fuel consumption data and can greatly improve the estimation accuracy of the fuel consumption of the ship by combining a related machine learning method

In order to achieve the purpose, the technical means adopted by the invention are as follows: a multi-source heterogeneous fuel consumption data feature extraction and fusion method comprises the following steps:

s1, preprocessing the oil consumption data

The fuel consumption data is sensor data related to fuel consumption, and the preprocessing of the fuel consumption data comprises two parts of supplementing missing values and deleting abnormal values. Firstly, an interpolation method is used for supplementing missing values generated due to sensor faults, and then data exceeding a normal operation area of a ship in oil consumption data are taken as abnormal data to be eliminated. The oil consumption data comprises ship wind speed data, trim data, rudder angle data, host fuel data, pitch data, navigational speed data and draft data.

S2, carrying out time domain consistency operation on the various preprocessed oil consumption data

The time-domain coherency operation includes a framing operation and a frame shifting operation. The frame division operation is that the time domain is moved and divided by adopting the frame length d with equal length; and the frame shift operation is that a certain overlapping area exists between two adjacent frames in the frame shift process, the overlapping area is called frame shift, and the frame shift length is 20-60% of the frame length. Through the operation, the consistency of the sensor data with different sampling frequencies on the time domain is ensured.

S3, extracting the characteristic of each frame data

And performing feature extraction operation on each frame data according to the data obtained in the step S2, wherein the feature extraction includes statistical feature extraction and time series feature extraction. The extraction process of the two features is as follows:

s31, extracting statistical feature vectors

The statistical characteristics of each frame data comprise mean, variance, mode, median, upper edge max and upper quartile point Q₃Lower quartile point Q₁And lower edge min.

And (4) taking the non-standard normal distribution of each frame data into consideration, and extracting the average value, the variance, the mode and the median as a statistical feature vector A.

Considering that each frame of data has outlier data, an upper edge, an upper quartile point, a lower quartile point and a lower edge are extracted as statistical feature vectors B.

Sorting the data according to the numerical value to obtain an upper quartile locus Q₃And lower quartile point Q₁. And will go up to quartile point Q₃And lower quartile point Q₁The interval between is defined as:

IQR＝Q₃-Q₁

according to the upper quartile point and the lower quartile point obtained by arrangement, the calculation formulas of the upper edge and the lower edge of the intra-frame data are obtained as follows:

Max＝Q₃+IQR

Min＝Q₁-IQR

s32, extracting time series characteristic vector

The time sequence characteristics of data in a frame are extracted by adopting an improved hierarchical clustering method, and Euclidean distance is taken as similarity measurement, and the specific process is as follows:

s321, inputting the framed data D of the frame to set the number of time-series features to be k.

S322, calculating the iteration time epoch according to the following formula:

epoch＝length(D)-k

wherein length (d) represents the number of points in a frame of data.

S323, calculating adjacent time data points respectivelyEuclidean distance dist (D)_i,D_i+1) That is, the euclidean distance between data point i and data point i +1 is calculated and stored in variable dis (i):

dis(i)＝dist(D_i,D_i+1)

and S324, merging adjacent data points with the shortest Euclidean distance, merging the average values of the data point i and the data point i +1, and covering the merged data point on the data point i.

D_i＝(D_i+D_i+1)/2

S325, turning to the step S323 until k time series characteristics are obtained:

c₁,c₂,…,c_k-1,c_k

s4, fusing the statistical feature vector and the time series feature vector

The statistical features and time series features obtained in step S3 are fused as input to the relevant machine learning method. And the characteristic fusion is to merge the obtained statistical characteristic vector of each frame of data with the time sequence characteristic vector to obtain a fused characteristic vector. The fusion feature vector of the statistical feature vector A and the time series feature is as follows:

(mean,variance,mode,median,c₁,c₂,…,c_k)

the fusion feature vector of the statistical feature vector B and the time series feature is as follows:

(min,Q₁,Q₃,max,c₁,c₂,…,c_k)

s5, using the fused feature as the input of the machine learning method

And (4) using the fused features obtained in the step (S4) as input of a machine learning method to realize modeling of the oil consumption of the ship, wherein the machine learning method comprises a linear regression method, a support vector regression method and an artificial neural network method.

Compared with the prior art, the invention has the following beneficial effects:

1. the method applies framing operation to the ship oil consumption data, and extracts statistical characteristics and time series characteristics; then, the statistical features and the time series features are fused. The method can process a large amount of existing sensor oil consumption data, and greatly improves the data volume for oil consumption modeling.

2. According to the invention, the quality of the oil consumption data is greatly improved by carrying out feature extraction and fusion on the multi-source heterogeneous oil consumption data, so that the accuracy of oil consumption modeling is improved.

Drawings

FIG. 1 is an overview of the modeling process of the present invention for fuel consumption estimation.

FIG. 2 is a schematic diagram of a multi-source data acquisition module of the present invention.

Fig. 3 is a schematic diagram of the overlapping framing method of the present invention.

Fig. 4 is a schematic diagram of time series feature extraction according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, a method for extracting and fusing characteristics of multi-source heterogeneous fuel consumption data includes the following steps:

s1, preprocessing the data of the fuel consumption related sensor

As shown in fig. 2, various sensors installed on the ship collect fuel consumption related information, including but not limited to ship speed information, fuel consumption information, wind speed information, etc. The data preprocessing operations include, but are not limited to: and (4) supplementing the missing value in the oil consumption data acquired by the shipborne sensor completely by using an interpolation method. And then, according to the normal operation area of the case ship, eliminating data outside the area as an abnormal value. For example, each ship has a designed sailing speed, a power of a commonly used power plant, a steering rate, a draft and the like according to the difference of the ship model, the ship shape, the ship length and the power plant configuration, and all the parameters are changed within a certain range, so that data beyond the conventional change range are removed.

S2, performing framing operation on the preprocessed data

The sampling frequency of the shipborne sensor is different, and the data of each sensor is not uniform in time domain. The invention performs framing operation on the sensor data, including but not limited to a moving overlapping framing method, and unifies the time domains of the sensors. The framing method is shown in fig. 3. Firstly, counting the sampling period of each shipborne navigation parameter sensor. At present, the sampling period of most navigation parameter sensors is 1-3 seconds. According to the invention, different navigation parameter sensors are framed in time domain in a length of 30-90 seconds, and the frame shift is set to be 20% -60% of the frame length.

S3, extracting the characteristic of each frame data

And performing feature extraction operation on each frame data according to the data obtained in the step S2, wherein the feature extraction includes statistical feature extraction and time series feature extraction. The extraction process of the two features is as follows:

s31, extracting statistical feature vectors

The statistical characteristics of each frame data comprise mean, variance, mode, median, upper edge max and upper quartile point Q₃Lower quartile point Q₁And lower edge min.

And (4) taking the non-standard normal distribution of each frame data into consideration, and extracting the average value, the variance, the mode and the median as a statistical feature vector A.

Considering that each frame of data has outlier data, an upper edge, an upper quartile point, a lower quartile point and a lower edge are extracted as statistical feature vectors B.

Sorting the data according to the numerical value to obtain an upper quartile locus Q₃And lower quartile point Q₁. And will go up to quartile point Q₃And lower quartile point Q₁The interval between is defined as:

IQR＝Q₃-Q₁

according to the upper quartile point and the lower quartile point obtained by arrangement, the calculation formulas of the upper edge and the lower edge of the intra-frame data are obtained as follows:

Max＝Q₃+IQR

Min＝Q₁-IQR

s32, extracting time series characteristic vector

As shown in fig. 4, an improved hierarchical clustering method is used to extract time series characteristics of data in a frame, and euclidean distance is used as a similarity measure, and the specific process is as follows:

s321, inputting the framed data D of the frame to set the number of time-series features to be k.

S322, calculating the iteration time epoch according to the following formula:

epoch＝length(D)-k

wherein length (d) represents the number of points in a frame of data.

S323, respectively calculating Euclidean distance dist (D) of adjacent time data points_i,D_i+1) That is, the euclidean distance between data point i and data point i +1 is calculated and stored in variable dis (i):

dis(i)＝dist(D_i,D_i+1)

and S324, merging adjacent data points with the shortest Euclidean distance, merging the average values of the data point i and the data point i +1, and covering the merged data point on the data point i.

D_i＝(D_i+D_i+1)/2

S325, turning to the step S323 until k time series characteristics are obtained:

c₁,c₂,…,c_k-1,c_k

s4, fusing the statistical feature vector and the time series feature vector

The statistical features and time series features obtained in step S3 are fused as input to the relevant machine learning method. And the characteristic fusion is to merge the obtained statistical characteristic vector of each frame of data with the time sequence characteristic vector to obtain a fused characteristic vector. The fusion feature vector of the statistical feature vector A and the time series feature is as follows:

(mean,variance,mode,median,c₁,c₂,…,c_k)

the fusion feature vector of the statistical feature vector B and the time series feature is as follows:

(min,Q₁,Q₃,max,c₁,c₂,…,c_k)

s5, using the fused feature as the input of the machine learning method

And (4) using the fused features obtained in the step (S4) as input of a machine learning method to realize modeling of the oil consumption of the ship, wherein the machine learning method comprises a linear regression method, a support vector regression method and an artificial neural network method.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (1)

1. A multi-source heterogeneous fuel consumption data feature extraction and fusion method is characterized by comprising the following steps: the method comprises the following steps:

s1, preprocessing the oil consumption data

The method comprises the following steps that oil consumption data are sensor data related to oil consumption, and preprocessing of the oil consumption data comprises two parts of missing value supplement and abnormal value deletion; firstly, supplementing a missing value generated by sensor faults by using an interpolation method, and then removing data exceeding a normal operation area of a ship in oil consumption data as abnormal data; the oil consumption data comprises ship wind speed data, trim data, rudder angle data, host fuel data, pitch data, navigational speed data and draft data;

s2, carrying out time domain consistency operation on the various preprocessed oil consumption data

The time domain consistency operation comprises framing operation and frame shifting operation; the frame division operation is that the time domain is moved and divided by adopting the frame length d with equal length; the frame shift operation is that a certain overlap area exists between two adjacent frames in the frame moving and dividing process, the overlap area is called frame shift, and the frame shift length is 20-60% of the frame length; through the operation, the consistency of the sensor data with different sampling frequencies on the time domain is ensured;

s3, extracting the characteristic of each frame data

According to the data obtained in the step S2, performing feature extraction operation on each frame of data, wherein the feature extraction comprises statistical feature extraction and time series feature extraction; the extraction process of the two features is as follows:

s31, extracting statistical feature vectors

The statistical characteristics of each frame data comprise mean, variance, mode, median, upper edge max and upper quartile point Q₃Lower quartile point Q₁And a lower edge min;

taking the non-standard normal distribution of each frame data into consideration, and extracting the average value, the variance, the mode and the median as a statistical feature vector A;

considering that each frame of data has outlier data, extracting an upper edge, an upper quartile point, a lower quartile point and a lower edge as a statistical feature vector B;

sorting the data according to the numerical value to obtain an upper quartile locus Q₃And lower quartile point Q₁(ii) a And will go up to quartile point Q₃And lower quartile point Q₁The interval between is defined as:

IQR＝Q₃-Q₁

according to the upper quartile point and the lower quartile point obtained by arrangement, the calculation formulas of the upper edge and the lower edge of the intra-frame data are obtained as follows:

Max＝Q₃+IQR

Min＝Q₁-IQR

s32, extracting time series characteristic vector

The time sequence characteristics of data in a frame are extracted by adopting an improved hierarchical clustering method, and Euclidean distance is taken as similarity measurement, and the specific process is as follows:

s321, inputting the framed data D of a frame to set the number of time sequence features as k;

s322, calculating the iteration time epoch according to the following formula:

epoch＝length(D)-k

wherein length (D) represents the number of one frame data point;

s323, respectively calculating Euclidean distance dist (D) of adjacent time data points_i,D_i+1) That is, the euclidean distance between data point i and data point i +1 is calculated and stored in variable dis (i):

dis(i)＝dist(D_i,D_i+1)

s324, merging adjacent data points with the shortest Euclidean distance, merging the average values of the data point i and the data point i +1, and covering the merged data point with the data point i;

D_i＝(D_i+D_i+1)/2

s325, turning to the step S323 until k time series characteristics are obtained:

c₁,c₂,…，c_k-1，c_k

s4, fusing the statistical feature vector and the time series feature vector

Fusing the statistical characteristics and the time series characteristics obtained in the step S3 as the input of a relevant machine learning method; the feature fusion is to merge the statistical feature vector of each frame data and the time sequence feature vector to obtain a fused feature vector; the fusion feature vector of the statistical feature vector A and the time series feature is as follows:

(mean，variance，mode，median,c₁,c₂,…，c_k)

the fusion feature vector of the statistical feature vector B and the time series feature is as follows:

(min,Q₁,Q₃,max,c₁,c₂,…，c_k)

s5, using the fused feature as the input of the machine learning method

And (4) using the fused features obtained in the step (S4) as input of a machine learning method to realize modeling of the oil consumption of the ship, wherein the machine learning method comprises a linear regression method, a support vector regression method and an artificial neural network method.

Images