US20210263652A1 - Sensor storage system - Google Patents
Sensor storage system Download PDFInfo
- Publication number
- US20210263652A1 US20210263652A1 US16/796,196 US202016796196A US2021263652A1 US 20210263652 A1 US20210263652 A1 US 20210263652A1 US 202016796196 A US202016796196 A US 202016796196A US 2021263652 A1 US2021263652 A1 US 2021263652A1
- Authority
- US
- United States
- Prior art keywords
- storage
- data
- integrated processor
- storage modules
- fabric
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000004744 fabric Substances 0.000 claims abstract description 40
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000012546 transfer Methods 0.000 claims abstract description 6
- 238000001816 cooling Methods 0.000 claims description 14
- 238000000034 method Methods 0.000 claims description 13
- 238000000638 solvent extraction Methods 0.000 claims description 7
- 230000002093 peripheral effect Effects 0.000 claims description 6
- 230000003068 static effect Effects 0.000 claims description 4
- 230000008878 coupling Effects 0.000 claims 2
- 238000010168 coupling process Methods 0.000 claims 2
- 238000005859 coupling reaction Methods 0.000 claims 2
- 230000006870 function Effects 0.000 abstract description 3
- 230000007246 mechanism Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 3
- 239000007788 liquid Substances 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000005055 memory storage Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/20—Cooling means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
- G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/40—Bus structure
- G06F13/4004—Coupling between buses
- G06F13/4022—Coupling between buses using switching circuits, e.g. switching matrix, connection or expansion network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
- G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
- G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
- G06F13/4295—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus using an embedded synchronisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
- G06F15/17306—Intercommunication techniques
- G06F15/17331—Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0613—Improving I/O performance in relation to throughput
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0688—Non-volatile semiconductor memory arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/06—Answer-back mechanisms or circuits
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/35—Switches specially adapted for specific applications
- H04L49/356—Switches specially adapted for specific applications for storage area networks
- H04L49/358—Infiniband Switches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2200/00—Indexing scheme relating to G06F1/04 - G06F1/32
- G06F2200/20—Indexing scheme relating to G06F1/20
- G06F2200/201—Cooling arrangements using cooling fluid
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/0026—PCI express
Definitions
- Embodiments described herein generally relate to a sensor storage system, and in an embodiment, but not by way of limitation, a sensor storage system that is made up of removable storage elements of non-volatile memory storage modules interconnected to a network fabric.
- Modern sensor systems e.g., radar systems
- cluster fabric-enabled storage having write rate capabilities and capacities that are sufficient to record and store sensor data in real time for extended periods.
- the aggregate data rates that must be supported by such storage systems are a function of the number of data sources, the streaming rate of the digitized data, and the additional processed data that must also be collected.
- FIG. 1 is an illustration of a storage module and controller module combination.
- FIG. 2 is an illustration of a single storage module.
- FIG. 3 is an illustration of a stack of storage modules.
- FIG. 4 is an illustration of a stack of storage modules, a controller module, and a network fabric.
- FIG. 5 is an example of a manner in which several PCI data busses can be used to maximize storage.
- One or more embodiments relate to a sensor storage system, and in particular, a sensor storage system that is made up of removable storage elements of non-volatile memory storage modules interconnected to a network fabric.
- a Sensor Storage Subsystem provides a cartridge, module, or unit that includes a removable or fixed (e.g., soldered-down integrated circuits) set of storage elements composed of non-volatile memory express NVMe M.2 and/or U.2 form factor modules (and future form factor modules) and/or ruler form factor modules, interconnected on a PCIe (peripheral component interconnect express) fabric, sourced by an integrated processing system providing direct connection to a cluster fabric performing sensor (e.g., radar) signal and data processing.
- An SSS embodiment operates at extended temperatures making it suited for demanding ground mobile radar deployments.
- An SSS embodiment also supports the encryption of source data and system initialization (boot).
- Data writing capacity and rates can be arbitrarily scaled to meet requirements by interconnecting one or more SSS's and by utilizing data centric publish subscribe (DCPS) mechanisms and remote direct memory access (RDMA) transfers to allocate capacity and write rate.
- DCPS data centric publish subscribe
- RDMA remote direct memory access
- the M.2 and/or U.2 modules and NVMe infrastructure use a PCIe interconnect structure that provides high write data rates and capacity in a compact package.
- An embodiment has the capability to provide an integrated (multiple type) cluster fabric attached into an extended-temperature, rugged, and modular storage system.
- the system uses a cartridge containing removable NVMe solid state devices as a scalable solution for sensor data storage.
- the system provides extendable, software-controlled capabilities that are integrated into a unit support boot, system initialization, and encryption of data.
- An embodiment provides a configurable and scalable high-speed data recording and system initialization mechanism that can operate directly on a cluster fabric such as InfiniBand or RoCE (RDMA over Converged Ethernet), and provides Network Attached Storage (NAS) via Ethernet or RoCE. It can be provisioned for various forms of solid-state storage in removable media cartridges or modules. It is a liquid and/or air-cooled rugged solution for demanding environments.
- the modular design implementation uses standard single board computer designs running a Linux operating system.
- the solid-state storage options that are available include M.2, U.2, and ruler form factor (e.g., Enterprise and Data Center SSD Form Factor (EDSFF))NVMe devices.
- EDSFF Enterprise and Data Center SSD Form Factor
- encryption is supported. Capacity can be increased by attaching external electrical connectors across units. In addition, this feature also allows for different storage form factors to be used simultaneously.
- Data are transferred from one or more hosts to the SSS using mechanisms based on Data Centric Publish Subscribe (DCPS) notification messages followed by RDMA transfers made by the SSS to acquire the data.
- DCPS Data Centric Publish Subscribe
- the software contained within the SSS moves these data to the storage devices of the SSS using operations to maximize the number of devices written to in order to provide high write-rate capabilities necessary for the collection of sensor data, for example, by utilizing multiple PCIe data busses.
- the configuration of the SSS allows, in the alternative, traditional NAS operation for lower demand collection needs. Ethernet, RoCE, and InfiniBand are supported network and fabric options.
- Software configuration over a network allows for dynamic and static partitioning and volume configuration of the storage. Off-load of the data is provided using the same means in read modes as provided for in collection modes.
- the SSS can be configured to generate notification messages to external hosts and source the previously stored data via RDMA to these hosts over the fabric or operate as Network Attached Storage (NAS) using conventional means such Network File System (NFS)
- an embodiment of the sensor storage subsystem permits multiple storage modules to be attached to a cluster fabric.
- the SSS supports multiple types of fabric interfaces, and supports multiple form factors of NVMe storage using different tray configurations.
- the SSS can include one or more of an OpenVPX or other standard form factor replaceable/upgradable controllers.
- the SSS is scalable (both upwards and downwards) in capacity and in performance.
- the SSS uses DCPS mechanisms followed by remote direct memory access (RDMA) data transfers over a cluster fabric interconnect.
- RDMA remote direct memory access
- NFS Network attached storage operation
- the SSS offers an SBC solution for controllers that makes the controllers upgradeable to future processing technologies and fabric interconnects as well as being able to support other storage technologies.
- the SSS is scalable in capacity and in performance by using multiple units and connecting those units together using an external PCIe fabric.
- the trays or units in the SSS are replaceable, which allows alternate solid-state drive (SSD) form factors to be used for each scalable recorder element.
- FIGS. 1, 2, 3, and 4 An example embodiment of a sensor storage subsystem (SSS) is illustrated in FIGS. 1, 2, 3, and 4 .
- FIG. 1 illustrates a side view of an SSS including a controller module 110 that includes a processor 105 , a storage module 120 , and a backplane 130 .
- FIG. 2 illustrates an example of a front view of a single storage module 120 , which supports multiple types of fabric interfaces such as InfiniBand 112 and Ethernet 114 .
- a PCIe expansion port 118 is also provided.
- a power port/input 116 is present.
- the unit is liquid-cooled via a coolant-in port 119 A and a coolant-out port 119 B.
- the cooling system can be an air-based cooling system, a liquid-based cooling system, or a combination of an air-based and liquid-based cooling system.
- FIG. 3 illustrates several storage modules 120 in a stacked arrangement.
- FIG. 4 illustrates a further embodiment of the SSS of FIGS. 1, 2, and 3 .
- a plurality of storage modules 120 is coupled together and coupled to a network fabric 140 .
- the storage modules 120 and the control module 110 are independently removable. Also, the storage modules 120 are more often removed than the control module 110 . Therefore, since it is less complex to disconnect air-cooled connections than it is to disconnect liquid-cooled connections, the storage modules 120 are typically air-cooled rather that liquid-cooled. Access to network fabrics is typically done using a network switch, however it is possible to use point-to-point connections for some use cases.
- the network fabric includes a plurality of hosts from which the SSS collects and processes sensor data.
- Hosts can include such devices as radar units, computing complexes that use a network fabric (such as computing complexes that are used in High Performance Computing (HPC) systems), and/or in systems that utilize network-connected computing clusters, such as computing systems that are used in weather forecasting, and science and research applications.
- Each storage module 120 includes a plurality of form factor non-volatile memory (NVMe) storage units or soldered-down integrated circuits.
- the storage modules 120 can include one or more NVMe M.2 or U.2 modules.
- the network fabric 140 can include a peripheral component interconnect express (PCIe) fabric. Additionally, the network fabric 140 can include Ethernet 114 , remote direct memory access (RDMA) over Converged Ethernet (RoCE), or InfiniBand 112 .
- PCIe peripheral component interconnect express
- RDMA remote direct memory access
- RoCE Converged Ethernet
- InfiniBand 112 InfiniBand 112 .
- FIG. 4 further illustrates a manner in which a processor 105 is integrated into a control module 110 , and further how the processor 105 can be coupled to the network fabric 140 and storage modules 120 .
- the integrated processor 105 is configured for control functions and data processing.
- the integrated processor configuration includes instructions such that the plurality of storage devices receive data via a data centric publish subscribe (DCPS) notification, and the DCPS notification is followed by a remote direct memory access (RDMA) transfer to the SSS processors.
- the SSS processors then optionally encrypt and/or directly store the captured data to storage devices in the storage module.
- the integrated processor can also be configured for booting the system, initializing the system, and encrypting the sensed data.
- the SSS can be useful in collecting sensed data from a radar system.
- the SSS can be used as a storage system in any computing complex that uses a network fabric, such as computing complexes that are used in High Performance Computing (HPC) systems, and/or in systems that utilize network-connected computing clusters, such as computing systems that are used in weather forecasting, and science and research applications.
- HPC High Performance Computing
- the integrated processor includes instructions for dynamic partitioning, static partitioning, and/or volume configuration.
- software executing on the integrated processor(s) can determine how to allocate storage capacity.
- the software can execute this in several ways. It can be executed statically at system initialization from predetermined configuration files. It can also be executed dynamically while the SSS is actively processing data to be stored. In both cases, the software may designate and use portions of the total amount of storage available and collect these portions into volumes, and these volumes are used as the storage units for allocation. Remote control and configuration of the SSS is accomplished using one or more of the available network connections (e.g., Ethernet or InfiniBand) and a protocol determined to be acceptable to the SSS and the host(s) (e.g., a radar system or other computing complex).
- the available network connections e.g., Ethernet or InfiniBand
- a protocol determined to be acceptable to the SSS and the host(s) e.g., a radar system or other computing complex.
- the integrated processor includes instructions for maximizing the number of the storage modules to which data are written.
- several PCIe data busses can be used to maximize storage.
- An example is illustrated in FIG. 5 .
- Another feature of the SSS is its ability to maximize the data write rate to available storage. This maximizing can be accomplished using software executing on the integrated processors, which distribute data across several storage devices to effectively improve write rate performance by parallelizing write operations. This is executed in a manner that does not require specific (storage system) formatting of the storage devices as might be the case if traditional RAID mechanisms were used. This enables data recovery to be accomplished using ordinary file reading mechanisms provided by the host (e.g., radar) operating system.
- the number of devices used can be determined by the SSS software or preconfigured by configuration files.
- the SSS is configured to permit network-attached storage for lower demand collection needs. That is, the storage modules can be partitioned according to each particular use case. For example, the SSS can be configured to behave as traditional network-attached storage using IP protocols (e.g., NFS).
- IP protocols e.g., NFS
- the SSS software incorporates this feature, and it is configurable to be used in conjunction with other operations that are executed by the SSS (e.g., DCPS/RDMA) simultaneously.
- the SSS is then able to allocate storage amounts to be used for different types of storage access methods on more than one network connection at a time.
- Engines may be hardware, software, or firmware communicatively coupled to one or more processors in order to carry out the operations described herein.
- Engines may be hardware engines, and as such engines may be considered tangible entities capable of performing specified operations and may be configured or arranged in a certain manner.
- circuits may be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner as an engine.
- the whole or part of one or more computing platforms may be configured by firmware or software (e.g., instructions, an application portion, or an application) as an engine that operates to perform specified operations.
- the software may reside on a machine-readable medium.
- the software when executed by the underlying hardware of the engine, causes the hardware to perform the specified operations.
- the term hardware engine is understood to encompass a tangible entity, be that an entity that is physically constructed, specifically configured (e.g., hardwired), or temporarily (e.g., transitorily) configured (e.g., programmed) to operate in a specified manner or to perform part of or all operations described herein.
- each of the engines need not be instantiated at any one moment in time.
- the engines comprise a general-purpose hardware processor configured using software; the general-purpose hardware processor may be configured as respective different engines at different times.
- Software may accordingly configure a hardware processor, for example, to constitute a particular engine at one instance of time and to constitute a different engine at a different instance of time.
- the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.”
- the term “or” is used to refer to a nonexclusive or, such that “A or B” includes “A but not B,” “B but not A,” and “A and B,” unless otherwise indicated.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Hardware Design (AREA)
- Signal Processing (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Storage Device Security (AREA)
- Techniques For Improving Reliability Of Storages (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Measurement Of Radiation (AREA)
Abstract
Description
- Embodiments described herein generally relate to a sensor storage system, and in an embodiment, but not by way of limitation, a sensor storage system that is made up of removable storage elements of non-volatile memory storage modules interconnected to a network fabric.
- Modern sensor systems (e.g., radar systems) are challenged to provide cluster fabric-enabled storage having write rate capabilities and capacities that are sufficient to record and store sensor data in real time for extended periods. The aggregate data rates that must be supported by such storage systems are a function of the number of data sources, the streaming rate of the digitized data, and the additional processed data that must also be collected.
- Prior methods of meeting high speed recording and storage needs have typically used hard drives, or solid-state drives in RAID (redundant array of independent discs) configurations on conventional storage interfaces (SATA (serial advanced technology attachment) or SAS (serial attached SCSI (small computer system interface)))) to produce the aggregate data rates and capacities required. These systems manifest themselves as direct-attached storage or network-attached storage driven by a single board computer (SBC) or server system. Software running on the SBC or server provides a means of receiving the sensor data and writing that data to the storage array. Rugged solutions exist, but they have limitations in their connectivity, capacity, supported data rates, and ability to support removable media. These prior solutions are challenged to provide both high performance computing (HPC) fabric and Ethernet fabric integration, both of which can support remote direct memory access (RDMA) methods, and the evolving size, weight, and power needs at extended temperatures and in ground mobile environments.
- Simply put, there are no available rugged data recording and boot solutions that can connect directly to RDMA-enabled cluster and network fabrics, support removable solid-state non-volatile memory (NVMe) media, and meet challenging data rate and environmental requirements.
- In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. Like numerals having different letter suffixes may represent different instances of similar components. Some embodiments are illustrated by way of example, and not limitation, in the figures of the accompanying drawings.
-
FIG. 1 is an illustration of a storage module and controller module combination. -
FIG. 2 is an illustration of a single storage module. -
FIG. 3 is an illustration of a stack of storage modules. -
FIG. 4 is an illustration of a stack of storage modules, a controller module, and a network fabric. -
FIG. 5 is an example of a manner in which several PCI data busses can be used to maximize storage. - One or more embodiments relate to a sensor storage system, and in particular, a sensor storage system that is made up of removable storage elements of non-volatile memory storage modules interconnected to a network fabric.
- In an embodiment, a Sensor Storage Subsystem (SSS) provides a cartridge, module, or unit that includes a removable or fixed (e.g., soldered-down integrated circuits) set of storage elements composed of non-volatile memory express NVMe M.2 and/or U.2 form factor modules (and future form factor modules) and/or ruler form factor modules, interconnected on a PCIe (peripheral component interconnect express) fabric, sourced by an integrated processing system providing direct connection to a cluster fabric performing sensor (e.g., radar) signal and data processing. An SSS embodiment operates at extended temperatures making it suited for demanding ground mobile radar deployments. An SSS embodiment also supports the encryption of source data and system initialization (boot). Data writing capacity and rates can be arbitrarily scaled to meet requirements by interconnecting one or more SSS's and by utilizing data centric publish subscribe (DCPS) mechanisms and remote direct memory access (RDMA) transfers to allocate capacity and write rate. The M.2 and/or U.2 modules and NVMe infrastructure use a PCIe interconnect structure that provides high write data rates and capacity in a compact package.
- An embodiment has the capability to provide an integrated (multiple type) cluster fabric attached into an extended-temperature, rugged, and modular storage system. The system uses a cartridge containing removable NVMe solid state devices as a scalable solution for sensor data storage. The system provides extendable, software-controlled capabilities that are integrated into a unit support boot, system initialization, and encryption of data.
- An embodiment provides a configurable and scalable high-speed data recording and system initialization mechanism that can operate directly on a cluster fabric such as InfiniBand or RoCE (RDMA over Converged Ethernet), and provides Network Attached Storage (NAS) via Ethernet or RoCE. It can be provisioned for various forms of solid-state storage in removable media cartridges or modules. It is a liquid and/or air-cooled rugged solution for demanding environments. The modular design implementation uses standard single board computer designs running a Linux operating system. The solid-state storage options that are available include M.2, U.2, and ruler form factor (e.g., Enterprise and Data Center SSD Form Factor (EDSFF))NVMe devices. Additionally, encryption is supported. Capacity can be increased by attaching external electrical connectors across units. In addition, this feature also allows for different storage form factors to be used simultaneously.
- Data are transferred from one or more hosts to the SSS using mechanisms based on Data Centric Publish Subscribe (DCPS) notification messages followed by RDMA transfers made by the SSS to acquire the data. The software contained within the SSS moves these data to the storage devices of the SSS using operations to maximize the number of devices written to in order to provide high write-rate capabilities necessary for the collection of sensor data, for example, by utilizing multiple PCIe data busses. The configuration of the SSS allows, in the alternative, traditional NAS operation for lower demand collection needs. Ethernet, RoCE, and InfiniBand are supported network and fabric options. Software configuration over a network allows for dynamic and static partitioning and volume configuration of the storage. Off-load of the data is provided using the same means in read modes as provided for in collection modes. Specifically, the SSS can be configured to generate notification messages to external hosts and source the previously stored data via RDMA to these hosts over the fabric or operate as Network Attached Storage (NAS) using conventional means such Network File System (NFS) to enable offloading of data.
- In summary, an embodiment of the sensor storage subsystem (SSS) permits multiple storage modules to be attached to a cluster fabric. The SSS supports multiple types of fabric interfaces, and supports multiple form factors of NVMe storage using different tray configurations. The SSS can include one or more of an OpenVPX or other standard form factor replaceable/upgradable controllers. The SSS is scalable (both upwards and downwards) in capacity and in performance. The SSS uses DCPS mechanisms followed by remote direct memory access (RDMA) data transfers over a cluster fabric interconnect. Network attached storage operation (e.g., NFS) is also supported simultaneously. The SSS offers an SBC solution for controllers that makes the controllers upgradeable to future processing technologies and fabric interconnects as well as being able to support other storage technologies. The SSS is scalable in capacity and in performance by using multiple units and connecting those units together using an external PCIe fabric. The trays or units in the SSS are replaceable, which allows alternate solid-state drive (SSD) form factors to be used for each scalable recorder element.
- An example embodiment of a sensor storage subsystem (SSS) is illustrated in
FIGS. 1, 2, 3, and 4 .FIG. 1 illustrates a side view of an SSS including acontroller module 110 that includes aprocessor 105, astorage module 120, and abackplane 130.FIG. 2 illustrates an example of a front view of asingle storage module 120, which supports multiple types of fabric interfaces such as InfiniBand 112 and Ethernet 114. APCIe expansion port 118 is also provided. A power port/input 116 is present. In an embodiment, the unit is liquid-cooled via a coolant-inport 119A and a coolant-outport 119B. The cooling system can be an air-based cooling system, a liquid-based cooling system, or a combination of an air-based and liquid-based cooling system.FIG. 3 illustratesseveral storage modules 120 in a stacked arrangement. -
FIG. 4 illustrates a further embodiment of the SSS ofFIGS. 1, 2, and 3 . InFIG. 4 , a plurality ofstorage modules 120 is coupled together and coupled to anetwork fabric 140. In an embodiment, thestorage modules 120 and thecontrol module 110 are independently removable. Also, thestorage modules 120 are more often removed than thecontrol module 110. Therefore, since it is less complex to disconnect air-cooled connections than it is to disconnect liquid-cooled connections, thestorage modules 120 are typically air-cooled rather that liquid-cooled. Access to network fabrics is typically done using a network switch, however it is possible to use point-to-point connections for some use cases. The network fabric includes a plurality of hosts from which the SSS collects and processes sensor data. Hosts can include such devices as radar units, computing complexes that use a network fabric (such as computing complexes that are used in High Performance Computing (HPC) systems), and/or in systems that utilize network-connected computing clusters, such as computing systems that are used in weather forecasting, and science and research applications. Eachstorage module 120 includes a plurality of form factor non-volatile memory (NVMe) storage units or soldered-down integrated circuits. Thestorage modules 120 can include one or more NVMe M.2 or U.2 modules. Thenetwork fabric 140 can include a peripheral component interconnect express (PCIe) fabric. Additionally, thenetwork fabric 140 can includeEthernet 114, remote direct memory access (RDMA) over Converged Ethernet (RoCE), orInfiniBand 112. -
FIG. 4 further illustrates a manner in which aprocessor 105 is integrated into acontrol module 110, and further how theprocessor 105 can be coupled to thenetwork fabric 140 andstorage modules 120. In an embodiment, theintegrated processor 105 is configured for control functions and data processing. Specifically, the integrated processor configuration includes instructions such that the plurality of storage devices receive data via a data centric publish subscribe (DCPS) notification, and the DCPS notification is followed by a remote direct memory access (RDMA) transfer to the SSS processors. The SSS processors then optionally encrypt and/or directly store the captured data to storage devices in the storage module. In another embodiment, the integrated processor can also be configured for booting the system, initializing the system, and encrypting the sensed data. - As noted above, the SSS can be useful in collecting sensed data from a radar system. In other embodiments, the SSS can be used as a storage system in any computing complex that uses a network fabric, such as computing complexes that are used in High Performance Computing (HPC) systems, and/or in systems that utilize network-connected computing clusters, such as computing systems that are used in weather forecasting, and science and research applications.
- In another embodiment, the integrated processor includes instructions for dynamic partitioning, static partitioning, and/or volume configuration. For example, software executing on the integrated processor(s) can determine how to allocate storage capacity. The software can execute this in several ways. It can be executed statically at system initialization from predetermined configuration files. It can also be executed dynamically while the SSS is actively processing data to be stored. In both cases, the software may designate and use portions of the total amount of storage available and collect these portions into volumes, and these volumes are used as the storage units for allocation. Remote control and configuration of the SSS is accomplished using one or more of the available network connections (e.g., Ethernet or InfiniBand) and a protocol determined to be acceptable to the SSS and the host(s) (e.g., a radar system or other computing complex).
- In another embodiment, the integrated processor includes instructions for maximizing the number of the storage modules to which data are written. In one instance, several PCIe data busses can be used to maximize storage. An example is illustrated in
FIG. 5 . Another feature of the SSS is its ability to maximize the data write rate to available storage. This maximizing can be accomplished using software executing on the integrated processors, which distribute data across several storage devices to effectively improve write rate performance by parallelizing write operations. This is executed in a manner that does not require specific (storage system) formatting of the storage devices as might be the case if traditional RAID mechanisms were used. This enables data recovery to be accomplished using ordinary file reading mechanisms provided by the host (e.g., radar) operating system. The number of devices used can be determined by the SSS software or preconfigured by configuration files. - In another embodiment, the SSS is configured to permit network-attached storage for lower demand collection needs. That is, the storage modules can be partitioned according to each particular use case. For example, the SSS can be configured to behave as traditional network-attached storage using IP protocols (e.g., NFS). The SSS software incorporates this feature, and it is configurable to be used in conjunction with other operations that are executed by the SSS (e.g., DCPS/RDMA) simultaneously. The SSS is then able to allocate storage amounts to be used for different types of storage access methods on more than one network connection at a time.
- Examples, as described herein, may include, or may operate on, logic or several components, circuits, or engines, which for the sake of consistency are termed engines, although it will be understood that these terms may be used interchangeably. Engines may be hardware, software, or firmware communicatively coupled to one or more processors in order to carry out the operations described herein. Engines may be hardware engines, and as such engines may be considered tangible entities capable of performing specified operations and may be configured or arranged in a certain manner. In an example, circuits may be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner as an engine. In an example, the whole or part of one or more computing platforms (e.g., a standalone, client or server computing platform) or one or more hardware processors may be configured by firmware or software (e.g., instructions, an application portion, or an application) as an engine that operates to perform specified operations. In an example, the software may reside on a machine-readable medium. In an example, the software, when executed by the underlying hardware of the engine, causes the hardware to perform the specified operations. Accordingly, the term hardware engine is understood to encompass a tangible entity, be that an entity that is physically constructed, specifically configured (e.g., hardwired), or temporarily (e.g., transitorily) configured (e.g., programmed) to operate in a specified manner or to perform part of or all operations described herein.
- Considering examples in which engines are temporarily configured, each of the engines need not be instantiated at any one moment in time. For example, where the engines comprise a general-purpose hardware processor configured using software; the general-purpose hardware processor may be configured as respective different engines at different times. Software may accordingly configure a hardware processor, for example, to constitute a particular engine at one instance of time and to constitute a different engine at a different instance of time.
- The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments that may be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements in addition to those shown or described. However, also contemplated are examples that include the elements shown or described. Moreover, also contemplated are examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.
- Publications, patents, and patent documents referred to in this document are incorporated by reference herein in their entirety, as though individually incorporated by reference. In the event of inconsistent usages between this document and those documents so incorporated by reference, the usage in the incorporated reference(s) are supplementary to that of this document; for irreconcilable inconsistencies, the usage in this document controls.
- In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.” In this document, the term “or” is used to refer to a nonexclusive or, such that “A or B” includes “A but not B,” “B but not A,” and “A and B,” unless otherwise indicated. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Also, in the following claims, the terms “including” and “comprising” are open-ended, that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to suggest a numerical order for their objects.
- The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with others. Other embodiments may be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. However, the claims may not set forth every feature disclosed herein as embodiments may feature a subset of said features. Further, embodiments may include fewer features than those disclosed in a particular example. Thus, the following claims are hereby incorporated into the Detailed Description, with a claim standing on its own as a separate embodiment. The scope of the embodiments disclosed herein is to be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
Claims (20)
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/796,196 US11822826B2 (en) | 2020-02-20 | 2020-02-20 | Sensor storage system |
EP21712605.1A EP4107629A1 (en) | 2020-02-20 | 2021-02-19 | Sensor storage system |
KR1020227028076A KR20220123551A (en) | 2020-02-20 | 2021-02-19 | sensor storage system |
PCT/US2021/018748 WO2021168227A1 (en) | 2020-02-20 | 2021-02-19 | Sensor storage system |
JP2022549835A JP7511658B2 (en) | 2020-02-20 | 2021-02-19 | Sensor Storage System |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/796,196 US11822826B2 (en) | 2020-02-20 | 2020-02-20 | Sensor storage system |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210263652A1 true US20210263652A1 (en) | 2021-08-26 |
US11822826B2 US11822826B2 (en) | 2023-11-21 |
Family
ID=74885023
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/796,196 Active 2040-09-21 US11822826B2 (en) | 2020-02-20 | 2020-02-20 | Sensor storage system |
Country Status (5)
Country | Link |
---|---|
US (1) | US11822826B2 (en) |
EP (1) | EP4107629A1 (en) |
JP (1) | JP7511658B2 (en) |
KR (1) | KR20220123551A (en) |
WO (1) | WO2021168227A1 (en) |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100049821A1 (en) * | 2008-08-21 | 2010-02-25 | Tzah Oved | Device, system, and method of distributing messages |
US20120120603A1 (en) * | 2010-11-16 | 2012-05-17 | International Business Machines Corporation | Automatically reconfigurable liquid-cooling apparatus for an electronics rack |
US20130132582A1 (en) * | 2011-11-18 | 2013-05-23 | Electronics And Telecommunications Research Institute | Apparatus and method for supporting qos in middleware for data distribution service |
US20130282853A1 (en) * | 2012-04-20 | 2013-10-24 | Electronics And Telecommunications Research Institute | Apparatus and method for processing data in middleware for data distribution service |
US20140201562A1 (en) * | 2010-10-10 | 2014-07-17 | Liqid Inc. | Systems and methods for optimizing data storage among a plurality of solid state memory subsystems |
US20140310457A1 (en) * | 2013-04-16 | 2014-10-16 | International Business Machines Corporation | Logical region allocation with immediate availability |
US20150067059A1 (en) * | 2013-08-30 | 2015-03-05 | Real-Time Innovations, Inc. | Scalable Content Distribution for Sparse Publish-Subscribe Systems Using the Object Management Group Real-Time Publish-Subscribe Protocol |
US20180068206A1 (en) * | 2016-09-08 | 2018-03-08 | Mentor Graphics Corporation | Object recognition and classification using multiple sensor modalities |
US20190349929A1 (en) * | 2018-05-09 | 2019-11-14 | General Electric Company | Systems and methods for medical body area network frequency band switching |
US20200019433A1 (en) * | 2017-03-22 | 2020-01-16 | Huawei Technologies Co., Ltd. | Resource Management Method and Apparatus |
US20200097054A1 (en) * | 2018-09-21 | 2020-03-26 | Super Micro Computer, Inc. | Adaptable storage bay for solid state drives |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1849092A4 (en) | 2005-01-06 | 2010-01-27 | Tervela Inc | End-to-end publish/subscribe middleware architecture |
JP6127416B2 (en) | 2012-09-07 | 2017-05-17 | 富士通株式会社 | Electronics |
US9887008B2 (en) | 2014-03-10 | 2018-02-06 | Futurewei Technologies, Inc. | DDR4-SSD dual-port DIMM device |
DE102017113022A1 (en) | 2017-06-13 | 2018-12-13 | Infineon Technologies Ag | Apparatus and method for processing radar signals |
US10785301B2 (en) | 2017-08-03 | 2020-09-22 | Toshiba Memory Corporation | NVM express over fabrics |
US10719474B2 (en) | 2017-10-11 | 2020-07-21 | Samsung Electronics Co., Ltd. | System and method for providing in-storage acceleration (ISA) in data storage devices |
US10992967B2 (en) | 2017-12-05 | 2021-04-27 | Sony Interactive Entertainment LLC | Ultra high-speed low-latency network storage |
US10649940B2 (en) | 2018-03-05 | 2020-05-12 | Samsung Electronics Co., Ltd. | Modular system architecture for supporting multiple solid-state drives |
WO2021166227A1 (en) | 2020-02-21 | 2021-08-26 | アイ ピース, インコーポレイテッド | Solution conveyance apparatus |
-
2020
- 2020-02-20 US US16/796,196 patent/US11822826B2/en active Active
-
2021
- 2021-02-19 WO PCT/US2021/018748 patent/WO2021168227A1/en unknown
- 2021-02-19 EP EP21712605.1A patent/EP4107629A1/en active Pending
- 2021-02-19 KR KR1020227028076A patent/KR20220123551A/en not_active Application Discontinuation
- 2021-02-19 JP JP2022549835A patent/JP7511658B2/en active Active
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100049821A1 (en) * | 2008-08-21 | 2010-02-25 | Tzah Oved | Device, system, and method of distributing messages |
US20140201562A1 (en) * | 2010-10-10 | 2014-07-17 | Liqid Inc. | Systems and methods for optimizing data storage among a plurality of solid state memory subsystems |
US20120120603A1 (en) * | 2010-11-16 | 2012-05-17 | International Business Machines Corporation | Automatically reconfigurable liquid-cooling apparatus for an electronics rack |
US20130132582A1 (en) * | 2011-11-18 | 2013-05-23 | Electronics And Telecommunications Research Institute | Apparatus and method for supporting qos in middleware for data distribution service |
US20130282853A1 (en) * | 2012-04-20 | 2013-10-24 | Electronics And Telecommunications Research Institute | Apparatus and method for processing data in middleware for data distribution service |
US20140310457A1 (en) * | 2013-04-16 | 2014-10-16 | International Business Machines Corporation | Logical region allocation with immediate availability |
US20150067059A1 (en) * | 2013-08-30 | 2015-03-05 | Real-Time Innovations, Inc. | Scalable Content Distribution for Sparse Publish-Subscribe Systems Using the Object Management Group Real-Time Publish-Subscribe Protocol |
US20180068206A1 (en) * | 2016-09-08 | 2018-03-08 | Mentor Graphics Corporation | Object recognition and classification using multiple sensor modalities |
US20200019433A1 (en) * | 2017-03-22 | 2020-01-16 | Huawei Technologies Co., Ltd. | Resource Management Method and Apparatus |
US20190349929A1 (en) * | 2018-05-09 | 2019-11-14 | General Electric Company | Systems and methods for medical body area network frequency band switching |
US20200097054A1 (en) * | 2018-09-21 | 2020-03-26 | Super Micro Computer, Inc. | Adaptable storage bay for solid state drives |
Also Published As
Publication number | Publication date |
---|---|
JP7511658B2 (en) | 2024-07-05 |
JP2023515483A (en) | 2023-04-13 |
WO2021168227A1 (en) | 2021-08-26 |
US11822826B2 (en) | 2023-11-21 |
KR20220123551A (en) | 2022-09-07 |
EP4107629A1 (en) | 2022-12-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11650764B2 (en) | Solid state drive multi-card adapter with integrated processing | |
US20190095294A1 (en) | Storage unit for high performance computing system, storage network and methods | |
US9507529B2 (en) | Apparatus and method for routing information in a non-volatile memory-based storage device | |
US9304704B2 (en) | Communication with two or more storage devices via one SAS communication port | |
US9645940B2 (en) | Apparatus and method for accessing a non-volatile memory blade using multiple controllers in a non-volatile memory based storage device | |
US20170220506A1 (en) | Modular Software Defined Storage Technology | |
US9792048B2 (en) | Identifying disk drives and processing data access requests | |
US9152603B1 (en) | System and method for increasing application compute client data I/O bandwidth performance from data file systems and/or data object storage systems by hosting/bundling all of the data file system storage servers and/or data object storage system servers in the same common global shared memory compute system as the application compute clients | |
US20170270044A1 (en) | Active Storage Unit and Array | |
US20180107388A1 (en) | Transfer of object memory references in a data storage device | |
US20140149662A1 (en) | Systems and methods for smart storage interconnection in a heterogeneous storage environment | |
KR101824671B1 (en) | Apparatus and method for routing information in a non-volatile memory-based storage device | |
US11822826B2 (en) | Sensor storage system | |
US10579275B2 (en) | Storage system and storage control method | |
US9489151B2 (en) | Systems and methods including an application server in an enclosure with a communication link to an external controller | |
JP6734058B2 (en) | Control device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: RAYTHEON COMPANY, MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DUBE, RUSSELL E.;REEL/FRAME:051875/0488 Effective date: 20200220 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: AWAITING TC RESP, ISSUE FEE PAYMENT VERIFIED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |