WO2024021107A1 - 工业数据的存储方法及装置 - Google Patents

工业数据的存储方法及装置 Download PDF

Info

Publication number
WO2024021107A1
WO2024021107A1 PCT/CN2022/109203 CN2022109203W WO2024021107A1 WO 2024021107 A1 WO2024021107 A1 WO 2024021107A1 CN 2022109203 W CN2022109203 W CN 2022109203W WO 2024021107 A1 WO2024021107 A1 WO 2024021107A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
threshold
storage
server
unstructured data
Prior art date
Application number
PCT/CN2022/109203
Other languages
English (en)
French (fr)
Inventor
王璐
于志强
于琪
Original Assignee
西门子股份公司
西门子（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西门子股份公司, 西门子（中国）有限公司 filed Critical 西门子股份公司
Priority to PCT/CN2022/109203 priority Critical patent/WO2024021107A1/zh
Publication of WO2024021107A1 publication Critical patent/WO2024021107A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control

Definitions

  • the present invention mainly relates to the field of industrial digitization, and in particular, to an industrial data storage method and device.
  • Unstructured data is an irregular or incomplete data structure. There is no predefined data model and it is inconvenient to use two-dimensional logical tables in the database.
  • the data to be represented mainly includes images, videos, audios, log files and binary files.
  • unstructured data usually cannot be saved in traditional databases due to the huge amount of data, and because of the Unstructured it is difficult to discover the relationship between them.
  • the present invention provides an industrial data storage method and device to efficiently store and recall unstructured data in industrial sites.
  • the present invention proposes a storage method for industrial data.
  • the storage method includes: acquiring unstructured data in industrial field equipment at an edge device; comparing the usage frequency of the unstructured data with a first a threshold, sending unstructured data with a usage frequency lower than the first threshold to the server, and saving unstructured data with a usage frequency higher than the first threshold to the edge device; in the server Unstructured data whose usage frequency is lower than the first threshold is stored in multiple storage nodes of the server.
  • Unstructured data whose usage frequency is lower than the first threshold is stored in multiple storage nodes of the server.
  • Multi-level storage reduces the amount of data storage on edge devices and improves the storage efficiency of unstructured data.
  • the storage node includes a main storage unit and an archive storage unit.
  • the method further includes: comparing the usage interval of the unstructured data with a second threshold, and setting the usage interval to be less than the second threshold.
  • the unstructured data is stored in the main storage unit, and the unstructured data whose usage interval is greater than the second threshold is stored in the archive storage unit.
  • data that will be used in the short term is stored in the main storage unit, and data that is not used in the long term is stored in the archive storage unit, which reduces the storage load of the main storage unit and improves data storage efficiency.
  • the method further includes: acquiring structured data in industrial field devices at the edge device, matching the structured data with historical data, and uploading and storing the matched first metadata in the server,
  • the first metadata includes a correspondence between the structured data and historical data.
  • the method further includes: obtaining the data of the third-party software, matching the data of the third-party software with the timestamp and tag, uploading and storing the matched second metadata in the server, the The second metadata includes the correspondence between the data of the third-party software and the timestamp and tag.
  • the data attributes of external third-party software are stored on the edge and server, which enriches the data types and improves the processing capabilities of edge devices.
  • the method further includes: obtaining video data in the storage node, converting the video data into a video stream, and pushing the video stream to a video player in response to the request.
  • efficient calls to video data in the server can be achieved.
  • the present invention also proposes an industrial data storage device.
  • the storage device includes: an acquisition module, which acquires unstructured data in industrial field equipment at the edge device; and a comparison module, which compares the usage frequency of the unstructured data with a first threshold, sending unstructured data with a usage frequency lower than the first threshold to the server, and saving unstructured data with a usage frequency higher than the first threshold to the edge device; storage module , the server stores unstructured data whose usage frequency is lower than the first threshold in multiple storage nodes of the server.
  • the storage node includes a main storage unit and an archive storage unit
  • the device further includes: comparing the usage interval of the unstructured data with a second threshold, and setting the usage interval to be less than the second threshold.
  • the unstructured data is stored in the main storage unit, and the unstructured data whose usage interval is greater than the second threshold is stored in the archive storage unit.
  • the device further includes: acquiring structured data in industrial field devices at the edge device, matching the structured data with historical data, and uploading and storing the matched first metadata in the server,
  • the first metadata includes a correspondence between the structured data and historical data.
  • the device further includes: obtaining the data of the third-party software, matching the data of the third-party software with the timestamp and tag, uploading and storing the matched second metadata in the server, the The second metadata includes the correspondence between the data of the third-party software and the timestamp and tag.
  • the device further includes: acquiring video data in the storage node, converting the video data into a video stream, and pushing the video stream to a video player in response to the request.
  • the present invention also proposes a data storage system.
  • the data storage system includes an edge device and a server.
  • the edge device acquires unstructured data in industrial field devices; compares the usage frequency of the unstructured data with a first threshold. , sending unstructured data with a usage frequency lower than the first threshold to the server, and saving unstructured data with a usage frequency higher than the first threshold to the edge device; the server will Unstructured data whose usage frequency is lower than the first threshold is stored in multiple storage nodes of the server.
  • the present invention also proposes an electronic device, including a processor, a memory and instructions stored in the memory, wherein when the instructions are executed by the processor, the above method is implemented.
  • the present invention also proposes a computer-readable storage medium on which computer instructions are stored, which execute the method as described above when executed.
  • Figure 1 is a flow chart of an industrial data storage method according to an embodiment of the present invention.
  • Figure 2 is a schematic diagram of the implementation environment of an industrial data storage method according to an embodiment of the present invention.
  • Figure 3 is a schematic diagram of an industrial data storage device according to an embodiment of the present invention.
  • Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present invention.
  • Figure 1 is a flow chart of an industrial data storage method 100 according to an embodiment of the present invention. As shown in Figure 1, the storage method 100 includes:
  • Step 110 Obtain unstructured data in the industrial field device at the edge device.
  • Unstructured data is data with irregular or incomplete data structure, no predefined data model, and inconvenience to be represented by two-dimensional logical tables in the database. It mainly includes images. , video, audio, log files and binary files.
  • Edge devices are located at industrial sites to capture unstructured data from industrial field devices. Edge devices can obtain unstructured data from historical data of industrial field devices and unstructured data from real-time data of industrial field devices.
  • Figure 2 is a schematic diagram of the implementation environment of an industrial data storage method according to an embodiment of the present invention. As shown in Figure 2, the first field device 21 generates unstructured data during operation.
  • the unstructured data It can come from historical data, such as videos, images, audio, models, files, etc., and unstructured data can also come from real-time data, such as real-time images collected by cameras, real-time audio collected by microphones, etc.
  • the edge device 23 includes the first A transceiver 231 , through which the edge device 23 obtains unstructured data in the first field device 21 .
  • the method further includes: acquiring structured data in the industrial field device at the edge device, matching the structured data with historical data, uploading and storing the matched first metadata in the server, the first metadata Including the correspondence between structured data and historical data.
  • the second field device 22 such as PLC (programmable logic controller), driver, AGV (automatic guided vehicle), sensor, CNC (computer numerical control), motor, robot, etc. is generated during operation. Structured data can also be called time series data.
  • the edge device 23 includes a second transceiver 235. The edge device 23 obtains the structured data in the second field device 22 through the second transceiver 235.
  • the event processing unit 236 will Match the data with historical data, for example, match the data sensed by the sensor with the historical data, and match a certain data with the singular point data in the historical data.
  • the corresponding relationship between a certain data and the singular point data is the first metadata.
  • the event processing unit 236 sends the first metadata to the edge-side metadata storage unit 237 for storage, and then uploads it to the server-side metadata storage unit MS in the storage node of the server 25 .
  • storing the data attributes of key structured data on the edge and server can greatly reduce the amount of data and improve the storage efficiency of industrial data.
  • the method further includes: obtaining the data of the third-party software, matching the data of the third-party software with the timestamp and tag, uploading and storing the matched second metadata in the server, where the second metadata includes the third metadata. Correspondence between data from third-party software and timestamps and tags.
  • the third transceiver 238 acquires third-party software 24, such as ERP (Enterprise Resource Planning System), MES (Manufacturing Execution System), WMS (Warehouse Management System), PLM (Product Lifecycle Management System) ), SCADA (Supervisory Control and Data Acquisition System), etc.
  • the matching unit 239 matches the data of the third-party software 24 with timestamps and labels, and the corresponding relationship between the matched data, timestamps, and labels is the second metadata. Also called data attributes, the matching unit 239 sends the second metadata to the edge-side metadata storage unit 237 for storage, and then uploads it to the server-side metadata storage unit MS in the storage node of the server 25 .
  • the data attributes of external third-party software are stored on the edge and server, which enriches the data types and improves the processing capabilities of edge devices.
  • Step 120 Compare the usage frequency of unstructured data with a first threshold, send the unstructured data with a usage frequency lower than the first threshold to the server, and save the unstructured data with a usage frequency higher than the first threshold to in edge devices.
  • the first threshold can be input by the user, preset by the system, or generated by the data model. If the usage frequency is lower than the first threshold, it indicates that the usage frequency of the unstructured data is low. For unstructured data with low usage frequency, The data is uploaded to the server to avoid occupying the storage space of the edge device. If the frequency of use is higher than the first threshold, it indicates that the frequency of use of the unstructured data is high. For unstructured data with a high frequency of use, Save it to the edge device to avoid delays caused by data communication. As shown in Figure 2, the first transceiver 231 sends the unstructured data to the first cache 232.
  • the first cache 232 compares the usage frequency of the unstructured data with a first threshold, and sets the usage frequency lower than the first threshold.
  • the unstructured data is sent to the second cache 233, and the second cache 233 sends the unstructured data with a usage frequency lower than the first threshold to the main storage unit VS of the storage node in the server 25 through the upload scheduler 234.
  • Step 130 The server stores unstructured data whose usage frequency is lower than the first threshold in multiple storage nodes of the server.
  • the server can include a management node and multiple storage nodes.
  • the management node is used to manage multiple storage nodes, such as discovery, registration, load balancing, backup, etc.
  • the storage node is used to store uploaded unstructured data. Therefore, Unstructured data whose usage frequency is less than the first threshold is stored in the server, and unstructured data whose usage frequency is greater than the first threshold is stored on the edge device, realizing multi-level storage of unstructured data in edge devices and servers. It reduces the amount of data storage on edge devices and improves the storage efficiency of unstructured data.
  • the server 25 includes a management node MN, storage nodes N1, N2, and N3.
  • the management node MN includes a version control unit 251, an archive unit 252, a discovery unit 253, a registration unit 254, a load balancing unit 255, and a backup unit 256.
  • each storage node N1, N2, N3 includes an archive storage unit AS, a version mirror SS, a main storage unit VS and a server-side metadata storage unit MS.
  • the version control unit 251 is connected to the version mirror SS of each storage node to implement storage.
  • the archive unit 252 is connected to the archive storage unit AS of each storage node and is used to realize the archive storage function (archiving, compression, encryption, etc.) of the storage node.
  • the discovery unit 253 and the registration unit 254 are used for discovery and registration.
  • the load balancing unit 255 is used to obtain the free storage capacity of each storage node, and determine the storage node to store unstructured data based on the free storage capacity.
  • the server 25 also includes a backup node RN, in which the data of each storage node is backed up.
  • the backup unit 256 in the management node MN is used to control the backup node RN.
  • the storage node includes a main storage unit and an archive storage unit
  • the method further includes: comparing the usage interval of unstructured data with a second threshold, and storing the unstructured data whose usage interval is less than the second threshold. in the main storage unit, and store the unstructured data whose usage interval is greater than the second threshold in the archive storage unit.
  • the usage interval is less than the second threshold, it indicates that the structured data will be used in the short term, and the unstructured data whose usage interval is less than the second threshold is stored in the main storage unit VS.
  • the interval is greater than the second threshold, it indicates that the structured data has not been used for a long time, and the unstructured data whose use interval is greater than the second threshold is stored in the archive storage unit.
  • data that will be used in the short term is stored in the main storage unit, and data that is not used in the long term is stored in the archive storage unit, which reduces the storage load of the main storage unit and improves data storage efficiency.
  • the method further includes: obtaining video data in the storage node, converting the video data into a video stream, and pushing the video stream to the video player in response to the request.
  • the streaming media conversion unit 257 obtains unstructured data, such as video, in the storage node, converts the video data into a video stream, and pushes the video stream to the video player through the fourth transceiver 258 in response to the request. 26 in.
  • efficient calls to video data in the server can be achieved.
  • the unstructured data stored in the first cache 232 of the edge device 23 can also be accessed through the data access unit 27 , or the data stored in each storage node in the server 25 can be accessed.
  • Embodiments of the present invention provide a storage method for industrial data. By storing unstructured data with a usage frequency less than a first threshold in a server, and storing unstructured data with a usage frequency greater than the first threshold on an edge device, It realizes multi-level storage of unstructured data in edge devices and servers, reduces the amount of data storage on edge devices, and improves the storage efficiency of unstructured data.
  • FIG. 3 is a schematic diagram of an industrial data storage device 300 according to an embodiment of the present invention. As shown in Figure 3, the storage device 300 includes:
  • the acquisition module 310 acquires unstructured data in industrial field devices at the edge device;
  • the comparison module 320 compares the usage frequency of unstructured data with a first threshold, sends unstructured data with a usage frequency lower than the first threshold to the server, and saves unstructured data with a usage frequency higher than the first threshold. to edge devices;
  • the storage module 330 stores unstructured data whose usage frequency is lower than the first threshold in multiple storage nodes of the server.
  • the storage node includes a main storage unit and an archive storage unit.
  • the device 300 further includes: comparing the usage interval time of unstructured data with a second threshold, and classifying unstructured data whose usage interval is less than the second threshold. Store in the main storage unit, and store unstructured data whose usage interval is greater than the second threshold in the archive storage unit.
  • the apparatus 300 further includes: acquiring structured data in industrial field devices at the edge device, matching the structured data with historical data, and uploading and storing the matched first metadata in the server.
  • the first metadata includes the correspondence between the structured data and historical data.
  • the device 300 further includes: obtaining the data of the third-party software, matching the data of the third-party software with the timestamp and tag, and uploading and storing the matched second metadata in the server.
  • the second metadata includes a correspondence between the data of the third-party software and the timestamp and tag.
  • the apparatus 300 further includes: obtaining video data in the storage node, converting the video data into a video stream, and pushing the video stream to the video player in response to the request.
  • the present invention also proposes a data storage system.
  • the data storage system includes an edge device 23 and a server 25 as shown in Figure 2.
  • the edge device 23 acquires unstructured data in industrial field devices; compares the usage frequency of unstructured data. and a first threshold, sending unstructured data with a usage frequency lower than the first threshold to the server, and saving unstructured data with a usage frequency higher than the first threshold to the edge device; the server 25 sends the unstructured data with a usage frequency lower than the first threshold to the edge device.
  • the unstructured data of the first threshold is stored in multiple storage nodes of the server 25 .
  • FIG. 4 is a schematic diagram of an electronic device 400 according to an embodiment of the present invention.
  • the electronic device 400 includes a processor 410 and a memory 420 .
  • the memory 420 stores instructions, and when the instructions are executed by the processor 410 , the method 100 as described above is implemented.
  • the present invention also proposes a computer-readable storage medium on which computer instructions are stored, and when executed, the computer instructions execute the method 100 as described above.
  • Some aspects of the method and device of the present invention may be executed entirely by hardware, may be entirely executed by software (including firmware, resident software, microcode, etc.), or may be executed by a combination of hardware and software.
  • the above hardware or software may be referred to as “data block”, “module”, “engine”, “unit”, “component” or “system”.
  • the processor may be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLCs), field programmable gate arrays (FPGAs), processors , controller, microcontroller, microprocessor or combination thereof.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DAPDs digital signal processing devices
  • PLCs programmable logic devices
  • FPGAs field programmable gate arrays
  • processors controller, microcontroller, microprocessor or combination thereof.
  • aspects of the invention may be embodied as a computer product embodied in one or more computer-readable media, the product including computer-readable program code.
  • computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), optical disks (e.g., compact disks (CD), digital versatile disks (DVD), ...), smart cards and flash memory devices (e.g. cards, sticks, key drives).
  • a flowchart is used here to illustrate operations performed by methods according to embodiments of the present application. It should be understood that the preceding operations are not necessarily performed in exact order. Instead, the various steps can be processed in reverse order or simultaneously. At the same time, other operations may be added to these processes, or a step or steps may be removed from these processes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明提出了一种工业数据的存储方法，所述存储方法包括：在边缘设备获取工业现场设备中的非结构化数据；比较所述非结构化数据的使用频率与一第一阈值，将使用频率低于所述第一阈值的非结构化数据发送至服务器，并将使用频率高于所述第一阈值的非结构化数据保存至所述边缘设备中；在所述服务器将使用频率低于所述第一阈值的非结构化数据存储于所述服务器的多个存储节点中。

Description

工业数据的存储方法及装置 技术领域
本发明主要涉及工业数字化领域，尤其涉及一种工业数据的存储方法及装置。
背景技术
工业领域，例如图像处理和预测性维护中越来越多的使用到非结构化数据，非结构化数据是数据结构不规则或不完整，没有预定义的数据模型，不方便用数据库二维逻辑表来表现的数据，主要包括图像、视频、音频、日志文件和二进制文件，不同于结构化数据或时序数据，非结构化数据通常由于数据量巨大无法保存于传统的数据库中，并且因为这些数据的非结构化很难发现他们之间的关系。
发明内容
为了解决上述技术问题，本发明提供一种工业数据的存储方法及装置，以高效地存储和调用工业现场中的非结构化数据。
为实现上述目的，本发明提出了一种工业数据的存储方法，所述存储方法包括：在边缘设备获取工业现场设备中的非结构化数据；比较所述非结构化数据的使用频率与一第一阈值，将使用频率低于所述第一阈值的非结构化数据发送至服务器，并将使用频率高于所述第一阈值的非结构化数据保存至所述边缘设备中；在所述服务器将使用频率低于所述第一阈值的非结构化数据存储于所述服务器的多个存储节点中。为此，通过将使用频率小于第一阈值的非结构化数据存储于服务器中，使用频率大于第一阈值的非结构化数据存储于边缘设备上，实现了非结构化数据在边缘设备和服务器的多层级存储，降低了边缘设备的数据存储量，提高了非结构化数据的存储效率。
可选地，所述存储节点包括主存储单元和归档存储单元，所述方法还包括：比较所述非结构化数据的使用间隔时间与一第二阈值，将使用间隔时间小于所述第二阈值的非结构化数据存储于所述主存储单元，并将使用间隔时间大于所述第二阈值的非结构化数据存储于所述归档存储单元。为此，将短期会用的数据存储于主存储单元中，将长期不用的数据存储于归档存储单元中，降低了主存储单元的存储负载，提高了数据的存储效率。
可选地，所述方法还包括：在边缘设备获取工业现场设备中的结构化数据，匹配所述结构化数据与历史数据，将匹配之后的第一元数据上传并存储于所述服务器中，所述第一元数据包括所述结构化数据与历史数据之间的对应关系。为此，在边缘端和服务器端存储 关键结构化数据的数据属性，可以极大地降低数据量，提高工业数据的存储效率。
可选地，所述方法还包括：获取第三方软件的数据，匹配所述第三方软件的数据与时间戳和标签，将匹配之后的第二元数据上传并存储于所述服务器中，所述第二元数据包括所述第三方软件的数据与时间戳和标签之间的对应关系。为此，在边缘端和服务器端存储外部第三方软件的数据属性，丰富了数据类型，提高了边缘设备的处理能力。
可选地，所述方法还包括：获取所述存储节点中的视频数据，将所述视频数据转换为视频流，并响应于请求将所述视频流推送至视频播放器中。为此，可以实现对服务器中的视频数据的高效调用。
本发明还提出了一种工业数据的存储装置，所述存储装置包括：获取模块，在边缘设备获取工业现场设备中的非结构化数据；比较模块，比较所述非结构化数据的使用频率与一第一阈值，将使用频率低于所述第一阈值的非结构化数据发送至服务器，并将使用频率高于所述第一阈值的非结构化数据保存至所述边缘设备中；存储模块，在所述服务器将使用频率低于所述第一阈值的非结构化数据存储于所述服务器的多个存储节点中。
可选地，所述存储节点包括主存储单元和归档存储单元，所述装置还包括：比较所述非结构化数据的使用间隔时间与一第二阈值，将使用间隔时间小于所述第二阈值的非结构化数据存储于所述主存储单元，并将使用间隔时间大于所述第二阈值的非结构化数据存储于所述归档存储单元。
可选地，所述装置还包括：在边缘设备获取工业现场设备中的结构化数据，匹配所述结构化数据与历史数据，将匹配之后的第一元数据上传并存储于所述服务器中，所述第一元数据包括所述结构化数据与历史数据之间的对应关系。
可选地，所述装置还包括：获取第三方软件的数据，匹配所述第三方软件的数据与时间戳和标签，将匹配之后的第二元数据上传并存储于所述服务器中，所述第二元数据包括所述第三方软件的数据与时间戳和标签之间的对应关系。
可选地，所述装置还包括：获取所述存储节点中的视频数据，将所述视频数据转换为视频流，并响应于请求将所述视频流推送至视频播放器中。
本发明还提出了数据存储系统，所述数据存储系统包括边缘设备和服务器，所述边缘设备获取工业现场设备中的非结构化数据；比较所述非结构化数据的使用频率与一第一阈值，将使用频率低于所述第一阈值的非结构化数据发送至所述服务器，并将使用频率高于所述第一阈值的非结构化数据保存至所述边缘设备中；所述服务器将使用频率低于所述第一阈值的非结构化数据存储于所述服务器的多个存储节点中。
本发明还提出了一种电子设备，包括处理器、存储器和存储在所述存储器中的指令， 其中所述指令被所述处理器执行时实现如上所述的方法。
本发明还提出了一种计算机可读存储介质，其上存储有计算机指令，所述计算机指令在被运行时执行如上所述的方法。
附图说明
以下附图仅旨在于对本发明做示意性说明和解释，并不限定本发明的范围。其中，
图1是根据本发明的一实施例的一种工业数据的存储方法的流程图；
图2是根据本发明的一实施例的一种工业数据的存储方法的实施环境的示意图；
图3是根据本发明的一实施例的一种工业数据的存储装置的示意图；
图4是根据本发明的一实施例的一种电子设备的示意图。
附图标记说明
100工业数据的存储方法
110-130步骤
21第一现场设备
22第二现场设备
23边缘设备
231第一收发器
232第一缓存
233第二缓存
234上传调度器
235第二收发器
236事件处理单元
237边缘端元数据存储单元
238第三收发器
239匹配单元
24第三方软件
25服务器
251版本控制单元
252归档单元
253发现单元
254注册单元
255负载平衡单元
256备份单元
257流媒体转换单元
258第四收发器
26视频播放器
27数据访问单元
N1,N2,N3存储节点
RN(Replication Node)备份节点
MN(Management Node)管理节点
AS(Archive Storage)归档存储单元
SS(Snapshots)版本镜像
VS(Volume Storage)主存储单元
MS(Metadata Storage)服务器端元数据存储单元
300工业数据的存储装置
310获取模块
320比较模块
330存储模块
400电子设备
410处理器
420存储器
具体实施方式
为了对本发明的技术特征、目的和效果有更加清楚的理解，现对照附图说明本发明的具体实施方式。
在下面的描述中阐述了很多具体细节以便于充分理解本发明，但是本发明还可以采用其它不同于在此描述的其它方式来实施，因此本发明不受下面公开的具体实施例的限制。
如本申请和权利要求书中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其他的步骤或元素。
本发明提出了一种工业数据的存储方法，图1是根据本发明的一实施例的一种工业数据的存储方法100的流程图，如图1所示，存储方法100包括：
步骤110，在边缘设备获取工业现场设备中的非结构化数据。
工业现场设备在运行过程中会产生非结构化数据，非结构化数据是数据结构不规则或不完整，没有预定义的数据模型，不方便用数据库二维逻辑表来表现的数据，主要包括图像、视频、音频、日志文件和二进制文件。边缘设备位于工业现场，以获取工业现场设备中的非结构化数据。边缘设备可以获取工业现场设备的历史数据中的非结构化数据，也可以获取工业现场设备的实时数据中的非结构化数据。图2是根据本发明的一实施例的一种工业数据的存储方法的实施环境的示意图，如图2所示，第一现场设备21在运行过程中产生了非结构化数据，非结构化数据可以来自于历史数据，例如视频、图像、音频、模型、文件等，非结构化数据也可以来自于实时数据，例如来自于摄像头采集的实时图像，麦克风采集的实时音频等，边缘设备23包括第一收发器231，边缘设备23通过第一收发器231获取第一现场设备21中的非结构化数据。
在一些实施例中，方法还包括：在边缘设备获取工业现场设备中的结构化数据，匹配结构化数据与历史数据，将匹配之后的第一元数据上传并存储于服务器中，第一元数据包括结构化数据与历史数据之间的对应关系。具体地，如图2所示，第二现场设备22例如PLC(可编程逻辑控制器)，驱动器，AGV(自动引导车)，传感器，CNC(计算机数控)，电机，机器人等在运行过程中产生结构化数据，也可以称之为时序数据，边缘设备23包括第二收发器235，边缘设备23通过第二收发器235获取第二现场设备22中的结构化数据，事件处理单元236将结构化数据与历史数据进行匹配，例如将传感器感测的数据与历史数据进行匹配，匹配到某一数据与历史数据中的奇点数据，某一数据与奇点数据的对应关系即为第一元数据，也称之为数据属性，事件处理单元236将第一元数据发送至边缘端元数据存储单元237存储，随后上传至服务器25的存储节点中的服务器端元数据存储单元MS中。为此，在边缘端和服务器端存储关键结构化数据的数据属性，可以极大地降低数据量，提高工业数据的存储效率。
在一些实施例中，方法还包括：获取第三方软件的数据，匹配第三方软件的数据与时间戳和标签，将匹配之后的第二元数据上传并存储于服务器中，第二元数据包括第三方软件的数据与时间戳和标签之间的对应关系。具体地，如图2所示，第三收发器238获取第三方软件24，例如ERP(企业资源规划系统)，MES(制造执行系统)，WMS(仓储管理系统)，PLM(产品生命周期管理系统)，SCADA(监视控制与数据采集系统)等的数据，匹配单元239匹配第三方软件24的数据与时间戳和标签，匹配到的数据与时间戳和标签 的对应关系即为第二元数据，也称之为数据属性，匹配单元239将第二元数据发送至边缘端元数据存储单元237存储，随后上传至服务器25的存储节点中的服务器端元数据存储单元MS中。为此，在边缘端和服务器端存储外部第三方软件的数据属性，丰富了数据类型，提高了边缘设备的处理能力。
步骤120，比较非结构化数据的使用频率与一第一阈值，将使用频率低于第一阈值的非结构化数据发送至服务器，并将使用频率高于第一阈值的非结构化数据保存至边缘设备中。
第一阈值可以由用户输入，也可以由系统预设，或者由数据模型生成，若使用频率低于第一阈值，表明该非结构化数据的使用频率偏低，对于使用频率偏低的非结构化数据，将其上传至服务器上，避免占用边缘设备的存储空间，若使用频率高于第一阈值，表明该非结构化数据的使用频率偏高，对于使用频率偏高的非结构化数据，将其保存至边缘设备中，避免数据通信导致的延迟。如图2所示，第一收发器231将非结构化数据发送至第一缓存232中，第一缓存232比较非结构化数据的使用频率与一第一阈值，将使用频率低于第一阈值的非结构化数据发送至第二缓存233，第二缓存233通过上传调度器234将使用频率低于第一阈值的非结构化数据发送至服务器25中存储节点的主存储单元VS中。
步骤130，在服务器将使用频率低于第一阈值的非结构化数据存储于服务器的多个存储节点中。
服务器可以包括管理节点和多个存储节点，管理节点用于对多个存储节点进行管理，例如发现，注册，负载平衡，备份等，存储节点用于存储上传来的非结构化数据，由此，将使用频率小于第一阈值的非结构化数据存储于服务器中，使用频率大于第一阈值的非结构化数据存储于边缘设备上，实现了非结构化数据在边缘设备和服务器的多层级存储，降低了边缘设备的数据存储量，提高了非结构化数据的存储效率。如图2所示，服务器25包括管理节点MN，存储节点N1，N2，N3，管理节点MN包括版本控制单元251，归档单元252，发现单元253，注册单元254，负载平衡单元255和备份单元256，各存储节点N1，N2，N3包括归档存储单元AS，版本镜像SS，主存储单元VS和服务器端元数据存储单元MS，版本控制单元251连接至各存储节点的版本镜像SS，用于实现存储节点的镜像恢复功能，归档单元252连接至各存储节点的归档存储单元AS，用于实现存储节点的归档存储功能(归档，压缩，加密等)，发现单元253和注册单元254用于发现和注册新的存储节点，负载平衡单元255用于获取各存储节点的空余存储容量，并根据空余存储容量来确定将非结构化数据存储的存储节点。服务器25还包括备份节点RN，备份节点RN中备份有各存储节点的数据，管理节点MN中的备份单元256用于对备份节点RN进行控 制。
在一些实施例中，存储节点包括主存储单元和归档存储单元，方法还包括：比较非结构化数据的使用间隔时间与一第二阈值，将使用间隔时间小于第二阈值的非结构化数据存储于主存储单元，并将使用间隔时间大于第二阈值的非结构化数据存储于归档存储单元。具体地，如图2所示，若使用间隔时间小于第二阈值，表明该结构化数据短期会用，将使用间隔时间小于第二阈值的非结构化数据存储于主存储单元VS中，若使用间隔时间大于第二阈值，表明该结构化数据长期不用，将使用间隔时间大于第二阈值的非结构化数据存储于归档存储单元。为此，将短期会用的数据存储于主存储单元中，将长期不用的数据存储于归档存储单元中，降低了主存储单元的存储负载，提高了数据的存储效率。
在一些实施例中，方法还包括：获取存储节点中的视频数据，将视频数据转换为视频流，并响应于请求将视频流推送至视频播放器中。如图2所示，流媒体转换单元257获取存储节点中的非结构化数据，例如视频，将视频数据转换为视频流，并响应于请求通过第四收发器258将视频流推送至视频播放器26中。为此，可以实现对服务器中的视频数据的高效调用。可选地，还可以通过数据访问单元27访问边缘设备23的第一缓存232中存储的非结构化数据，或者访问服务器25中各存储节点中存储的数据。
本发明的实施例提供了一种工业数据的存储方法，通过将使用频率小于第一阈值的非结构化数据存储于服务器中，使用频率大于第一阈值的非结构化数据存储于边缘设备上，实现了非结构化数据在边缘设备和服务器的多层级存储，降低了边缘设备的数据存储量，提高了非结构化数据的存储效率。
本发明还提出了一种工业数据的存储装置，图3是根据本发明的一实施例的一种工业数据的存储装置300的示意图，如图3所示，存储装置300包括：
获取模块310，在边缘设备获取工业现场设备中的非结构化数据；
比较模块320，比较非结构化数据的使用频率与一第一阈值，将使用频率低于第一阈值的非结构化数据发送至服务器，并将使用频率高于第一阈值的非结构化数据保存至边缘设备中；
存储模块330，在服务器将使用频率低于第一阈值的非结构化数据存储于服务器的多个存储节点中。
在一些实施例中，存储节点包括主存储单元和归档存储单元，装置300还包括：比较非结构化数据的使用间隔时间与一第二阈值，将使用间隔时间小于第二阈值的非结构化数据存储于主存储单元，并将使用间隔时间大于第二阈值的非结构化数据存储于归档存储单元。
在一些实施例中，装置300还包括：在边缘设备获取工业现场设备中的结构化数据，匹配所述结构化数据与历史数据，将匹配之后的第一元数据上传并存储于所述服务器中，所述第一元数据包括所述结构化数据与历史数据之间的对应关系。
在一些实施例中，装置300还包括：获取第三方软件的数据，匹配所述第三方软件的数据与时间戳和标签，将匹配之后的第二元数据上传并存储于所述服务器中，所述第二元数据包括所述第三方软件的数据与时间戳和标签之间的对应关系。
在一些实施例中，装置300还包括：获取存储节点中的视频数据，将视频数据转换为视频流，并响应于请求将视频流推送至视频播放器中。
本发明还提出了一种数据存储系统，数据存储系统包括如图2所示的边缘设备23和服务器25，边缘设备23获取工业现场设备中的非结构化数据；比较非结构化数据的使用频率与一第一阈值，将使用频率低于第一阈值的非结构化数据发送至服务器，并将使用频率高于第一阈值的非结构化数据保存至边缘设备中；服务器25将使用频率低于第一阈值的非结构化数据存储于服务器25的多个存储节点中。
本发明还提出一种电子设备400。图4是根据本发明的一实施例的一种电子设备400的示意图。如图4所示，电子设备400包括处理器410和存储器420，存储器420存储中存储有指令，其中指令被处理器410执行时实现如上文所述的方法100。
本发明还提出一种计算机可读存储介质，其上存储有计算机指令，计算机指令在被运行时执行如上文所述的方法100。
本发明的方法和装置的一些方面可以完全由硬件执行、可以完全由软件(包括固件、常驻软件、微码等)执行、也可以由硬件和软件组合执行。以上硬件或软件均可被称为“数据块”、“模块”、“引擎”、“单元”、“组件”或“系统”。处理器可以是一个或多个专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理器件(DAPD)、可编程逻辑器件(PLC)、现场可编程门阵列(FPGA)、处理器、控制器、微控制器、微处理器或者其组合。此外，本发明的各方面可能表现为位于一个或多个计算机可读介质中的计算机产品，该产品包括计算机可读程序编码。例如，计算机可读介质可包括，但不限于，磁性存储设备(例如，硬盘、软盘、磁带……)、光盘(例如，压缩盘(CD)、数字多功能盘(DVD)……)、智能卡以及闪存设备(例如，卡、棒、键驱动器……)。
在此使用了流程图用来说明根据本申请的实施例的方法所执行的操作。应当理解的是，前面的操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各种步骤。同时，或将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。
应当理解，虽然本说明书是按照各个实施例描述的，但并非每个实施例仅包含一个独 立的技术方案，说明书的这种叙述方式仅仅是为清楚起见，本领域技术人员应当将说明书作为一个整体，各实施例中的技术方案也可以经适当组合，形成本领域技术人员可以理解的其他实施方式。
以上所述仅为本发明示意性的具体实施方式，并非用以限定本发明的范围。任何本领域的技术人员，在不脱离本发明的构思和原则的前提下所作的等同变化、修改与结合，均应属于本发明保护的范围。

Claims (13)

  1. 一种工业数据的存储方法(100)，其特征在于，所述存储方法(100)包括：
    在边缘设备获取工业现场设备中的非结构化数据(110)；
    比较所述非结构化数据的使用频率与一第一阈值，将使用频率低于所述第一阈值的非结构化数据发送至服务器，并将使用频率高于所述第一阈值的非结构化数据保存至所述边缘设备中(120)；
    在所述服务器将使用频率低于所述第一阈值的非结构化数据存储于所述服务器的多个存储节点中(130)。
  2. 根据权利要求1所述的存储方法(100)，其特征在于，所述存储节点包括主存储单元和归档存储单元，所述方法(100)还包括：比较所述非结构化数据的使用间隔时间与一第二阈值，将使用间隔时间小于所述第二阈值的非结构化数据存储于所述主存储单元，并将使用间隔时间大于所述第二阈值的非结构化数据存储于所述归档存储单元。
  3. 根据权利要求1或2所述的存储方法(100)，其特征在于，所述方法(100)还包括：在边缘设备获取工业现场设备中的结构化数据，匹配所述结构化数据与历史数据，将匹配之后的第一元数据上传并存储于所述服务器中，所述第一元数据包括所述结构化数据与历史数据之间的对应关系。
  4. 根据权利要求3所述的存储方法(100)，其特征在于，所述方法(100)还包括：获取第三方软件的数据，匹配所述第三方软件的数据与时间戳和标签，将匹配之后的第二元数据上传并存储于所述服务器中，所述第二元数据包括所述第三方软件的数据与时间戳和标签之间的对应关系。
  5. 根据权利要求1所述的存储方法(100)，其特征在于，所述方法(100)还包括：获取所述存储节点中的视频数据，将所述视频数据转换为视频流，并响应于请求将所述视频流推送至视频播放器中。
  6. 一种工业数据的存储装置(300)，其特征在于，所述存储装置(300)包括：
    获取模块(310)，在边缘设备获取工业现场设备中的非结构化数据；
    比较模块(320)，比较所述非结构化数据的使用频率与一第一阈值，将使用频率低于所述第一阈值的非结构化数据发送至服务器，并将使用频率高于所述第一阈值的非结构化数据保存至所述边缘设备中；
    存储模块(330)，在所述服务器将使用频率低于所述第一阈值的非结构化数据存储于所述服务器的多个存储节点中。
  7. 根据权利要求6所述的存储装置(300)，其特征在于，所述存储节点包括主存储单元和归档存储单元，所述装置(300)还包括：比较所述非结构化数据的使用间隔时间与一第二阈值，将使用间隔时间小于所述第二阈值的非结构化数据存储于所述主存储单元，并将使用间隔时间大于所述第二阈值的非结构化数据存储于所述归档存储单元。
  8. 根据权利要求6或7所述的存储装置(300)，其特征在于，所述装置(300)还包括：在边缘设备获取工业现场设备中的结构化数据，匹配所述结构化数据与历史数据，将匹配之后的第一元数据上传并存储于所述服务器中，所述第一元数据包括所述结构化数据与历史数据之间的对应关系。
  9. 根据权利要求8所述的存储装置(300)，其特征在于，所述装置(300)还包括：获取第三方软件的数据，匹配所述第三方软件的数据与时间戳和标签，将匹配之后的第二元数据上传并存储于所述服务器中，所述第二元数据包括所述第三方软件的数据与时间戳和标签之间的对应关系。
  10. 根据权利要求6所述的存储装置(300)，其特征在于，所述装置(300)还包括：获取所述存储节点中的视频数据，将所述视频数据转换为视频流，并响应于请求将所述视频流推送至视频播放器中。
  11. 数据存储系统，所述数据存储系统包括边缘设备(23)和服务器(25)，其特征在于，
    所述边缘设备(23)获取工业现场设备中的非结构化数据；比较所述非结构化数据的使用频率与一第一阈值，将使用频率低于所述第一阈值的非结构化数据发送至所述服务器(25)，并将使用频率高于所述第一阈值的非结构化数据保存至所述边缘设备(23)中；
    所述服务器(25)将使用频率低于所述第一阈值的非结构化数据存储于所述服务器(25)的多个存储节点中。
  12. 一种电子设备(400)，包括处理器(410)、存储器(420)和存储在所述存储器(420)中的指令，其中所述指令被所述处理器(410)执行时实现如权利要求1-5任一项所述的方法。
  13. 一种计算机可读存储介质，其上存储有计算机指令，所述计算机指令在被运行时执行根据权利要求1-5中任一项所述的方法。
PCT/CN2022/109203 2022-07-29 2022-07-29 工业数据的存储方法及装置 WO2024021107A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/109203 WO2024021107A1 (zh) 2022-07-29 2022-07-29 工业数据的存储方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/109203 WO2024021107A1 (zh) 2022-07-29 2022-07-29 工业数据的存储方法及装置

Publications (1)

Publication Number Publication Date
WO2024021107A1 true WO2024021107A1 (zh) 2024-02-01

Family

ID=89705102

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/109203 WO2024021107A1 (zh) 2022-07-29 2022-07-29 工业数据的存储方法及装置

Country Status (1)

Country Link
WO (1) WO2024021107A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190339688A1 (en) * 2016-05-09 2019-11-07 Strong Force Iot Portfolio 2016, Llc Methods and systems for data collection, learning, and streaming of machine signals for analytics and maintenance using the industrial internet of things
CN110582096A (zh) * 2018-06-11 2019-12-17 大唐移动通信设备有限公司 一种数据请求的处理方法及通信网络系统
CN110659315A (zh) * 2019-08-06 2020-01-07 上海孚典智能科技有限公司 基于非易失性存储系统的高性能非结构化数据库服务
CN113655968A (zh) * 2021-08-24 2021-11-16 上海晋朔信息科技有限公司 非结构化数据存储方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190339688A1 (en) * 2016-05-09 2019-11-07 Strong Force Iot Portfolio 2016, Llc Methods and systems for data collection, learning, and streaming of machine signals for analytics and maintenance using the industrial internet of things
CN110582096A (zh) * 2018-06-11 2019-12-17 大唐移动通信设备有限公司 一种数据请求的处理方法及通信网络系统
CN110659315A (zh) * 2019-08-06 2020-01-07 上海孚典智能科技有限公司 基于非易失性存储系统的高性能非结构化数据库服务
CN113655968A (zh) * 2021-08-24 2021-11-16 上海晋朔信息科技有限公司 非结构化数据存储方法

Similar Documents

Publication Publication Date Title
US11941017B2 (en) Event driven extract, transform, load (ETL) processing
CN108241555B (zh) 一种分布式数据库的备份、恢复方法、装置和服务器
JP7200259B2 (ja) データオブジェクトストアのためのデータ保持ハンドリング
WO2019024673A1 (zh) 一种区块链块数据的归档方法、装置及查询方法、装置
US10455264B2 (en) Bulk data extraction system
CN110505495B (zh) 多媒体资源抽帧方法、装置、服务器及存储介质
US20190238605A1 (en) Verification of streaming message sequence
WO2018233630A1 (zh) 故障发现
CN110958300B (zh) 一种数据的上传方法、系统、装置、电子设备和计算机可读介质
WO2017124800A1 (zh) 一种视频补录方法和装置
WO2018068639A1 (zh) 数据恢复方法、装置和存储介质
WO2017101126A1 (zh) 一种监控系统的数据存储方法及系统
CN116049109A (zh) 一种基于过滤器的文件校验方法、系统、设备和介质
US20120265908A1 (en) Server and method for buffering monitored data
US9424362B2 (en) Storing and publishing contents of a content store
WO2024021107A1 (zh) 工业数据的存储方法及装置
US20210058652A1 (en) System and method of building a distributed network for essence management and access
US10129328B2 (en) Centralized management of webservice resources in an enterprise
US11294866B2 (en) Lazy optimistic concurrency control
US8538907B2 (en) Autonomous intelligent content items
US10114704B1 (en) Updating database records while maintaining accessible temporal history
US20180096079A1 (en) Re-execution of an analytical process based on lineage metadata
US20160283476A1 (en) Appending a user identifier to an image
CN111427851A (zh) 一种hdfs跨外部存储系统多层级存储效率优化的方法和设备
CN113312414B (zh) 数据处理方法、装置、设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22952554

Country of ref document: EP

Kind code of ref document: A1