NZ778588A - Systems and methods for processing metadata - Google Patents

Systems and methods for processing metadata

Info

Publication number
NZ778588A
NZ778588A NZ778473A NZ77847319A NZ778588A NZ 778588 A NZ778588 A NZ 778588A NZ 778473 A NZ778473 A NZ 778473A NZ 77847319 A NZ77847319 A NZ 77847319A NZ 778588 A NZ778588 A NZ 778588A
Authority
NZ
New Zealand
Prior art keywords
neural network
module
storage device
trained neural
lbas
Prior art date
Application number
NZ778473A
Inventor
Ferguson Anne L Chang
Horst Christopher Sarubin
Original Assignee
Nbcuniversal Media Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nbcuniversal Media Llc filed Critical Nbcuniversal Media Llc
Publication of NZ778588A publication Critical patent/NZ778588A/en

Links

Abstract

method for tracking containers. The method includes manufacturing containers (24), including forming the containers (24) and serializing them with machine-readable codes (26). The method further includes using the machine-readable codes (26) to store data associated with the containers (24), and supplying the containers (24) to a customer. The method still further comprises receiving from the customer, data obtained from customer-readings of the machine- readable codes (26); and receiving from one or more other locations in a distribution chain in which the containers (24) travel, data obtained from readings of the machine-readable codes (26) at those locations. The method still further comprises comparing the data from the customer- readings and other readings of the machine-readable codes (26) across product brands, product distribution channels, and/or container types, and providing the data to the customer. In at least some embodiments, the method further comprises receiving from the customer a compensation for the containers (24), for example, a per-refill compensation. d supplying the containers (24) to a customer. The method still further comprises receiving from the customer, data obtained from customer-readings of the machine- readable codes (26); and receiving from one or more other locations in a distribution chain in which the containers (24) travel, data obtained from readings of the machine-readable codes (26) at those locations. The method still further comprises comparing the data from the customer- readings and other readings of the machine-readable codes (26) across product brands, product distribution channels, and/or container types, and providing the data to the customer. In at least some embodiments, the method further comprises receiving from the customer a compensation for the containers (24), for example, a per-refill compensation.

Description

MODULE AND METHOD FOR DETECTING MALICIOUS ACTIVITIES IN A STORAGE DEVICE Field of the Invention This invention relates to a module and method for detecting malicious activities in a storage device whereby the module is provided within a controller of the storage device. The module is configured to monitor, using a trained neural network, appropriate logical block addresses (LBAs) of the file system of the storage device that contain sensitive data or information for malicious activities.
Summary of Prior Art Storage devices typically comprise of solid state devices (SSDs), hard disk drives (HDDs), optical drives or a magnetic disc drives. Regardless of the type of storage device, these devices are typically addressed linearly by their logical block addresses (LBAs). For HDDs, optical drives or magnetic disk drives, LBAs specify the location of specific blocks of data stored within the drive. As an example, LBA 0 would refer to the first sector on the first track accessible by the first head in the disc drive as such, when LBA 0 is accessed by a host device, the content contained at LBA 0 would be provided to the host device.
However, unlike the disk drives described above, SSDs comprise non-volatile memories that are electrically erasable and re-programmable and as such, would not have tracks or heads as referred to in the logical block addressing system. Hence, SSDs have to make use of a flash translation layer (FTL) as provided within the SSD’s flash memory controller to map a host device’s file system logical block addresses to the physical addresses of the flash memory (logical-to-physical mapping). In other words, the host device will still utilize existing LBA addressing methodologies to address the SSD for read/write/overwrite operations. These commands from the host device will be intercepted by the FTL and the FTL will maintain a map of the relationship between LBAs to physical block addresses (PBAs) of the flash memory. The PBAs will then be utilized by the SSD’s controller to carry out the received commands.
Recently, SSDs have become more widely used as storage devices as SSDs offer numerous advantages over traditional mechanical hard disk drives. For example, SSDs are much faster than HDDs, and are able to deliver up to 100 times the performance of HDDs and this translates to faster boot times and faster file transfers. SSDs also consume much lesser power than HDDs resulting in improved power and heat efficiencies. As a result, SSDs are now widely used in industrial, medical or military applications.
Typically, most SSDs will be used with a host device and may be used to store the host device’s operating system, i.e. used as the host’s system drive, whereby code associated with the operating system is stored within the SSD and will be accessed when the host device boots up. When the SSD is used as the host’s system drive, the SSD will have a master boot record (MBR) stored at a logical block address (LBA) 0 and the host device’s operating system code stored elsewhere in the storage device. When a host device accesses the storage device for the first time, instructions will be sent to LBA 0 to instruct the SSD to send the contents at LBA 0 to the host device. This enables the host device to read the MBR from LBA 0 whereby the MBR will typically contain computer-readable program code that, when executed by the host device, provides the host device with the ability to read the other parts of the operating system code from the storage device and boot up the host device.
Alternatively, an SSD may also be used as a secondary storage medium such as USB flash drives, memory cards or external storage devices to expand the storage capacity accessible by the host device. When such an SSD is accessed for the first time by a host device, the content at LBA 0 of the storage device would indicate to the host device that it is to be used as a secondary storage medium.
In order to access information contained within sensitive applications, malicious third parties have resorted to various means and ways to infect the MBR of such storage devices.
A common method involves the malicious third party gaining system level access to the storage device before de-rooting the MBR or other boot sectors of the storage device and causing a compromised operating system to be installed within.
To protect the operating system from being tampered with and to prevent access to important private information if the storage device is misplaced, it has been proposed by those skilled in the art that the operating system code (including the MBR) be encrypted by software installed within the storage device and be subjected to authentication procedures, so that the MBR and operating system code are accessible only to authorized users. As the controller will not be able to read the MBR prior to authenticating a user of the storage device, the storage device can store an “alternate” master boot record (MBR) that causes authentication information to be collected and validated by an authentication program running in the storage device.
After successfully authenticating a user of the storage device, the storage device remaps LBA 0 to the original MBR, so that the storage device can receive the actual MBR and boot up as normal. The downside to this approach is that if the user’s authentication credentials are compromised, this means that the MBR and the operating system code will also be jeopardized.
Additionally, solutions proposed by those skilled in the art require the operating system, partition system or boot area of the monitored storage device to be known and pre-loaded into these solutions before the storage device may be adequately protected by these solutions. In other words, existing solutions are unable to automatically identify the operating system, partition system or boot area of the monitored storage device and such information has to be provided by the user to the existing monitoring solution. To the controller of the storage device, all information contained within the storage device comprises the user’s data and it is by default unable to differentiate this data. This becomes particularly problematic when the operating system, partition system or boot area of the monitored storage device is modified or incorrectly selected by the user and as a result, the storage device may become inadvertently compromised.
For the above reasons, those skilled in the art are constantly striving to come up with a module and method that is capable of detecting malicious activities in a storage device even though the operating system, partition system or boot area of the storage device is not provided to the controller of the device by a user.
Summary of the Invention The above and other problems are solved and an advance in the art is made by systems and methods provided by embodiments in accordance with the invention.
A first advantage of embodiments of modules and methods in accordance with the invention is that the module is able to identify the type of operating system installed within the memory device.
A second advantage of embodiments of modules and methods in accordance with the invention is that the module is able to detect malicious activities taking place within specific locations within the storage device automatically and efficiently.
A third advantage of embodiments of modules and methods in accordance with the invention is that the logical block addresses of the storage device will be monitored at the firmware level and does not require the operating system to be booted up before malicious activities may be detected and thwarted.
A fourth advantage of embodiments of modules and methods in accordance with the invention is that content contained within a storage device (that has configured to act as the host device’s system device) will still be protected from malicious third parties even though the storage device is removed from the host device and reconfigured as a secondary storage device as the module as the module is configured detect malicious activities in both configurations.
A fifth advantage of embodiments of modules and methods in accordance with the invention is that the module may not be disabled at the operating system level as the module is implemented as part of the storage device controller’s firmware.
The above advantages are provided by embodiments of a method in accordance with the invention operating in the following manner.
According to a first aspect of the invention, a module for detecting malicious activity in a storage device is disclosed, whereby the module is provided within a controller of the storage device, the module being configured to: retrieve a first logical block address (LBA0) sent to the controller from a host device and retrieve content at the first LBA0, whereby the retrieved content is used to prime a trained neural network retrieved from a data module provided within the controller; identify, using content at the first LBA0 and the trained neural network, logical block addresses (LBAs) of the storage device that are to be monitored; mirror instructions sent to the identified LBAs by the host device to the controller, and mirror contents of the mirrored LBAs; determine, using the trained neural network, if malicious activity is occurring at the storage device based on the mirrored instructions and contents, wherein the neural network is trained for different types of operating systems or secondary storage operations based on average read/write/overwrite access of contents at LBAs related to master boot records, master file tables, boot sectors, BIOS parameter blocks or extended BIOS parameter blocks of file systems associated with the operating systems or the secondary storage operations.
With regard to the first aspect of the invention, the priming of the associated trained neural network comprises the module being configured to: select a set of magic numbers from the content at the first LBA0, whereby the selected set of magic numbers are used with a magic number lookup table to determine a type of operating system or a type of secondary storage operation associated with a file system of the storage controller, whereby the magic number lookup table is obtained from the data module; and prime the trained neural network to detect malicious activities related to the determined type of operating system or secondary storage operation from the data module.
With regard to the first aspect of the invention, the identifying LBAs of the storage device that are to be monitored comprises the module being configured to: identify, based on a determined type of operating system or secondary storage operation associated with the primed trained neural network, LBAs that contain critical data whereby the critical data comprises at least a master file table, a master boot record, a boot sector, a BIOS parameter block or an extended BIOS parameter block of a file system associated with the storage device.
With regard to the first aspect of the invention, the module is further configured to: optimize the trained neural network using the malicious activity determined to have occurred at the storage device based on the mirrored instructions and contents.
With regard to the first aspect of the invention, the module is further configured to: lockdown the storage device in response to a determination that malicious activity is determined to have occurred at the storage device based on the mirrored instructions and contents.
With regard to the first aspect of the invention, the trained neural network comprises one of an artificial neural network, a Recurrent Neural Network (RNN) or a Convolutional Neural Network (CNN).
According to a second aspect of the invention, a method for detecting malicious activity in a storage device is disclosed, the method comprising the steps of: retrieving, using a module provided within a controller of the storage device, a first logical block address (LBA0) sent to the controller from a host device; retrieving, using the module, content at the first LBA0, whereby the retrieved content is used to prime a trained neural network retrieved from a data module provided within the controller; identifying using content at the first LBA0 and the trained neural network, using the module, logical block addresses (LBAs) of the storage device that are to be monitored; mirroring, using the module, instructions sent to the identified LBAs by the host device to the controller, and mirror contents of the mirrored LBAs; and determining, using the trained neural network, if malicious activity is occurring at the storage device based on the mirrored instructions and contents, wherein the neural network is trained for different types of operating systems or secondary storage operations based on average read/write/overwrite access of contents at LBAs related to master boot records, master file tables, boot sectors, BIOS parameter blocks or extended BIOS parameter blocks of file systems associated with the operating systems or the secondary storage operations.
With regard to the second aspect of the invention, the priming of the associated trained neural network comprises the steps of: selecting, using the module, a set of magic numbers from the content at the first LBA0, whereby the selected set of magic numbers are used with a magic number lookup table to determine a type of operating system or a type of secondary storage operation associated with a file system of the storage controller, whereby the magic number lookup table is obtained from the data module; and priming, using the module, the trained neural network to detect malicious activities related to the determined type of operating system or secondary storage operation from the data module.
With regard to the second aspect of the invention, the identifying LBAs of the storage device that are to be monitored comprises the steps of: identifying, using the module, based on a determined type of operating system or secondary storage operation associated with the primed trained neural network, LBAs that contain critical data whereby the critical data comprises at least a master file table, a master boot record, a boot sector, a BIOS parameter block or an extended BIOS parameter block of a file system associated with the storage device.
With regard to the second aspect of the invention, the method further comprises the step of: optimizing, using the module, the trained neural network using the malicious activity determined to have occurred at the storage device based on the mirrored instructions and contents.
With regard to the second aspect of the invention, the method further comprises the step of: locking down, using the module, the storage device in response to a determination that malicious activity is determined to have occurred at the storage device based on the mirrored instructions and contents.
With regard to the second aspect of the invention, the trained neural network comprises one of an artificial neural network, a Recurrent Neural Network (RNN) or a Convolutional Neural Network (CNN).
Brief Description of the Drawings The above and other problems are solved by features and advantages of a system and method in accordance with the present invention described in the detailed description and shown in the following drawings.
Figure 1 illustrating a block diagram of modules contained within a storage device in accordance with embodiments of the invention; Figure 2 illustrating a block diagram of modules contained within a controller of a storage device in accordance with embodiments of the invention; Figure 3 illustrating an exemplary boot sector of a file system in a storage device in accordance with embodiments of the invention; Figure 4 illustrating a flow chart of a process or a method for detecting malicious activity in a storage device in accordance with embodiments of the invention; and Figure 5 illustrating a flow chart of a process or a method for priming a trained neural network in accordance with embodiments of the invention.
Detailed Description This invention relates to a module and method for detecting malicious activities in a storage device whereby the module is provided within a controller of the storage device. The module is configured to monitor, using a trained neural network, appropriate logical block addresses (LBAs) of the file system of the storage device that contain sensitive data or information for malicious activities wherein the neural network is trained for different types of operating systems or secondary storage operations based on average read/write/overwrite access of contents at LBAs related to master boot records, master file tables, boot sectors, BIOS parameter blocks or extended BIOS parameter blocks of file systems associated with the operating systems or the secondary storage operations.
The present invention will now be described in detail with reference to several embodiments thereof as illustrated in the accompanying drawings. In the following description, numerous specific features are set forth in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to one skilled in the art, that embodiments may be realised without some or all of the specific features. Such embodiments should also fall within the scope of the current invention. Further, certain process steps and/or structures in the following may not been described in detail and the reader will be referred to a corresponding citation so as to not obscure the present invention unnecessarily.
Further, one skilled in the art will recognize that many functional units in this description have been labelled as modules throughout the specification. The person skilled in the art will also recognize that a module may be implemented as circuits, logic chips or any sort of discrete component. Still further, one skilled in the art will also recognize that a module may be implemented in software which may then be executed by a variety of processor architectures.
In embodiments of the invention, a module may also comprise computer instructions, firmware or executable code that may instruct a computer processor to carry out a sequence of events based on instructions received. The choice of the implementation of the modules is left as a design choice to a person skilled in the art and does not limit the scope of this invention in any way.
An exemplary process or method for detecting malicious activity in a storage device in accordance with embodiments of the invention is set out in the steps below. The steps of the process or method as implemented by a module provided within a controller of the storage device are as follows: Step 1: retrieve a first logical block address (LBA0) sent to the controller from a host device and retrieve content at the first LBA0, whereby the retrieved content is used to prime a trained neural network retrieved from a data module provided within the controller; Step 2: identify, using content at the first LBA0 and the trained neural network, logical block addresses (LBAs) of the storage device that are to be monitored; Step 3: mirror instructions sent to the identified LBAs by the host device to the controller, and mirror contents of the mirrored LBAs; Step 4: determine, using the trained neural network, if malicious activity is occurring at the storage device based on the mirrored instructions and contents, wherein the neural network is trained for different types of operating systems or secondary storage operations based on average read/write/overwrite access of contents at LBAs related to master boot records, master file tables, boot sectors, BIOS parameter blocks or extended BIOS parameter blocks of file systems associated with the operating systems or the secondary storage operations.
In accordance with embodiments of the invention, the steps set out above may be carried out or executed by modules contained within controller 105 of storage device 100, as illustrated in Figure 1, whereby storage device 100 additionally comprises cache 107, flash memories 110a-h and interface 120. Storage device 100 may comprise various types of solid state devices/drives, cache 107 may comprise a dynamic Random-Access-Memory (DRAM) and is used for caching both user data and internal SSD meta data. Flash memories 110a-h may comprise any type of electronic non-volatile computer memory storage medium that can be electronically erased and reprogrammed such as NAND or NOR flash memories. Interface 120 acts as the physical interface between a host system and storage device 100 whereby existing storage standards and interfaces such as, but not limited to, small computer system interface (SCSI) protocol, serial advanced technology attachment (SATA) protocol, serial attached SCSI (SAS), Non-Volatile Memory express (NVMe), Peripheral Component Interconnect express (PCIe) or any similar interface may be used as the link for communicatively connecting storage device 100 to a host device such as a computer.
Controller 105 is a complex embedded system with standalone processing and works with firmware and modules contained within controller 105 to manage all aspects of storage device 100, including protecting and controlling content stored in flash memories 110a-h. This controller is most commonly implemented as a SoC (System-On-Chip) design which consists of multiple hardware-accelerated functional blocks/ modules that are coupled to one or more embedded processor cores.
The functional blocks contained within controller 105 are illustrated in Figure 2. In particular, Figure 2 shows that controller 105 may comprise micro-controller 205, buffer 210, flash interface modules (FIMs) 215a-c, and threat detection module 250. Micro-controller 205 comprises a processor located inside controller 105 and is tasked to receive and manipulate incoming data. Herein the term “processor” is used to refer generically to any device or component that can process such instructions and may include: a microprocessor, microcontroller, programmable logic device or other computational device. That is, micro- controller 205 may be provided by any suitable logic circuitry for receiving inputs, processing them in accordance with instructions stored in memory and generating outputs. In this embodiment, micro-controller 205 may be a single core processor with memory addressable space. Buffer 210 may be treated as a data module as it may comprise SRAM (static RAM) for executing controller 105’s firmware or storing data/information that is to be accessed by module 250. Threat detection module 250 is used to mirror inputs/outputs at controller 105; train and load an appropriate trained neural network to detect malicious activities that may take place within the storage device and related tasks in accordance with embodiments of the invention. FIMs 215a-c act as the physical and logical interconnects between controller 105 and the flash memories 110a-h allowing the controller to communicate with multiple flash memories simultaneously. One skilled in the art will recognize that the various memory components described above comprise non-transitory computer-readable media and shall be taken to comprise all computer-readable media except for a transitory, propagating signal.
Typically, the instructions are stored as program code in the memory components but can also be hardwired. Although not shown, controller 105 also comprises a flash translation layer (FTL) for translating LBAs from a host device to physical block addresses (PBAs) of the flash memories. The detailed workings of the FTL are omitted for brevity as it is known to those skilled in the art.
When a host device (not shown) is booted up and under the assumption that storage device 100 is to be used as the host device’s system device, its operating system would not be contained within the host device’s ROM or RAM. It would be stored within storage device 100. As such, to facilitate the loading of the operating system, the host device will execute a firmware stored in the host device’s ROM which will send appropriate instructions to LBA ‘0’ (LBA0) of storage device 100. Upon receiving these instructions which are addressed to its LBA ‘0”, storage device 100 will send back data that is stored at a physical block address that corresponds to LBA ‘0’. Behind the scenes, the logical block address received by controller 105 of storage device 100 will be converted into a suitable physical block address by the FTL provided within controller 105. However, to the host device, all this is unseen and it is understood that when instructions are addressed to LBAs of storage device 100, all the necessary translations between the LBAs to PBAs will automatically take place in controller 105.
Typically in a system device, the first LBA in the LBA scheme, i.e. LBA ‘0’, may contain, but is not only limited to, the master boot record (MBR), which comprises computer-readable program code that, when executed by the host device, provides the host device with the ability to read the other parts of the operating system code from the storage device 100 and boot up the host device. LBA is a common scheme that is used for specifying the location of blocks of data stored within storage device 100 and provides a simple linear addressing method for the host device to access content stored within storage device 100 without the host device having to be aware of the storage device’s physical sector positions or PBAs. As such, when various partitions, file systems or any other special areas of the storage device are accessed by the host device, the LBAs associated with these areas will be sent by the host device to controller 105 of storage device 100 so that controller 105 may use this information to retrieve the relevant data/information for the host device.
As controller 105 receives the LBAs and their related instructions from the host device and returns the contents of the LBAs to the host device based on the received instructions, threat detection module 250 is configured to mirror all this and this may be done by making a record of all inputs and outputs taking place at controller 105. In other words, threat detection module 250 may achieve this by recording the instructions received by controller 105 and the LBAs that the instructions are directed to. Data and/or information provided at these LBAs may then be recorded by module 250 as well before the data and/or information is sent back to the requesting host device.
An exemplary boot sector of a file system as provided at logical block address ‘0’ (LBA0) is shown in Figure 3. As illustrated, it is shown that boot sector 300 comprises multiple fields having multiple lengths and their individual offsets. Each field would have its own typical value which would be associated with a particular meaning or command. These values may comprise, but are not limited to, hex numbers or magic numbers that have been generated based on each particular meaning/command for each type of operating system/file system/storage system and as such, each meaning or command would be associated with a unique value. For example, a value of “EB” may be associated with “Bytes per sector”, a value of “52” may be associated with “Sectors per Cluster”, a value of “67” may be associated with “OEM ID”, a value of “J9” may be associated with “BPB”, a value of “34” may be associated with “extended BPB” and etc.
In this exemplary embodiment of the invention, of particular interest would be key information contained at data 305 of LBA0 as shown in Figure 3, e.g. may comprise the BIOS parameter block (BPB) and the extended BPB, and one skilled in the art will recognize that this is just an example and that other contents at LBA0 may be used as well. Based on the information contained in data 305, the following information about the file system of the storage device may be determined: number of bytes per sector, number of sectors per cluster, the type of media descriptor, the total number of sectors, the location of the master file table (MFT) or its equivalent structure, the location of the copy of the master file table, the number of clusters per MFT record, the number of clusters per index buffer, the type of file system, the operating system and the volume’s serial number. This information may then be used to determine the operating system of the storage device’s file system, the storage device’s file system and/or the operation of the storage device’s file system. One skilled in the art will recognize that other information may be included within data 305 and within the content of LBA0 without departing from the invention.
In accordance embodiments of the invention, a magic number lookup table may be preloaded into cache 107 or buffer 210. As known to one skilled in the art, magic numbers refer to constant numerical values that were generated for specific information or data. As such, these magic numbers may be used to identify a particular file format or protocol or may refer to distinctive unique values that are unlikely to be mistaken for other meanings. In this embodiment of the invention, each of the magic numbers in the preloaded magic number lookup table refers to a particular type of operating system and/or a type of file system such as a secondary storage file system. It should be noted that this lookup table may be updated periodically as required, or whenever a new operating system, secondary storage file system, or other types of file systems are introduced. The information in the magic number lookup table may then be matched with the contents found at LBA0 and based on the resulting match, module 250 may then determine the type of operating system/ file system/ storage system that is associated with the storage device.
By doing so, threat detection module 250 may then utilize this information to prime a trained neural network contained within module 250 to detect malicious activities for a specific type of operating system or secondary storage operation as each file system would have its own unique list of LBAs that contain critical content, information or data. In embodiments of the invention, critical content or records comprises, but is not limited to, data that affects the user of the storage device, data that affects the normal operation of the storage device and/or any such similar data. In other words, this information may be used by module 250 to identify LBAs in the file system of storage device 100 that are to be closely monitored by the trained neural network whereby these LBAs may be unique to the type of file system installed within the storage device. Additionally, a trained neural network that has been optimized for the identified type of operating system or file system may also be selected and loaded. Once the parameters described above have been initialized, the primed trained neural network may then be used by module 250 to monitor storage device 100 for malicious activities.
In embodiment's of the invention, a neural network model is provided within buffer 210 or cache 107 and this neural network may comprise, but is not limited to, an artificial neural network such as a recurrent neural network (RNN), a recursive neural network or a convolutional neural network (CNN). This neural network model would have been pre-trained before it is used to detect malicious activities taking place in a file system of a storage device.
In particular, the neural network model would have been trained based on average read/write/overwrite access of contents at specific LBAs or PBAs relating to master boot records, master file tables, boot sectors, BIOS parameter blocks and/or extended BIOS parameter blocks of file systems associated with each type of operating system or the secondary storage file systems (which are installed in storage devices used as secondary storage operations).
In other words, the neural network model would have been trained based on various input vectors such as the average read/write/overwrite access of contents of LBAs that are commonly accessed during the operation of various types of operating systems and their corresponding file systems or during the operation of the storage device as a secondary storage system. As such, any activities that deviate from these conventional actions may cause the neural network to label the triggering activities as malicious activities. Labelled malicious activities together with LBAs accessed by these malicious activities may also be used to train the neural network whereby combinations of the data above may be provided to the neural network during its training phase to optimize the training of the neural network.
In further embodiments of the invention, the trained neural network may be further optimized using malicious activities detected during the normal operation of the storage device.
Such an on-the-fly optimization step would greatly improve the efficiency and effectiveness of the neural network.
In embodiments of the invention, the average read/write/overwrite access of contents of the LBAs of the various operating systems and file systems may be obtained by recording the inputs/outputs at the controller of a storage device having the various operating systems and file systems over a period of time. The average read/write/overwrite access may also be obtained from third parties resources and may be used train the neural network.
Additionally, as the LBAs accessed for each of the various operating systems, file systems and secondary storage systems differ from one system to the next, a record of the LBAs that contain critical data for each of these systems may be created whereby the critical data may comprise, but is not limited to, a master file table or its equivalent file structure, a master boot record, a boot sector, critical areas defined by an user, a secure area, a BIOS parameter block or an extended BIOS parameter block of a file system. This record may then be linked with the trained neural network model and be stored in cache 107 or buffer 210 or alternatively, may be used as part of the training data provided to train the neural network as described above. Hence, once a storage device’s function has been identified, i.e. to operate as a system device or secondary storage device, the LBAs of the storage device that contain critical data may then be identified from this record.
In summary, the neural network will be trained to protect certain areas of the file system and the training will be done based on the type of file system that is implemented on the storage device and LBAs that contain critical data. The information to identify the type of file system may be obtained from contents at the first LBA, i.e. LBA0. However, one skilled in the art will recognize that while the relevant content may initially be found at LBA0, for certain types of file systems, it may be so voluminous or due to the manner in which the information is structured, it may be distributed across multiple LBAs, e.g. from LBA “0”- LBA “48”. In embodiments of the invention, each neural network may be optimized for each type of file system as the critical LBAs vary from one file system to the next. As such, the type of trained neural network that is to be used may depend on the file system of the storage device and the performance of trained neural network may be more efficient and effective if a suitably trained neural network were to be selected to be used with the suitable file system and this action may be taken as the priming of the trained neural network.
Figure 4 illustrates process 400 for detecting malicious activities in a storage device that is communicatively connected to a host device in accordance with embodiments of the invention whereby process 400 may be implemented in threat detection module 250 as provided within a controller of a storage device. Process 400 begins at step 405 whereby a first logical block address (LBA) sent to the controller from the host device is copied by process 400. Process 400 then proceeds to retrieve content at the first LBA, whereby the retrieved content is used to prime a trained neural network retrieved from a data module provided within the controller. When this happens, a specific neural network that has been optimized for the identified file system/operation system/storage system is loaded and primed.
Based on the retrieved content and/or the primed trained neural network, a record of the LBAs that contain critical data for an associated file system is then loaded at step 410. This record is then used to identify LBAs of the storage device that are to be monitored by process 400. At step 415, process 400 then mirrors instructions sent to the LBAs under monitor and also mirrors contents from these LBAs that are subsequently sent to the host device. Process 400 then determines at step 420, based on the mirrored instructions and contents if malicious activities are taking place at the storage device.
If the process 400 determines at step 420 that malicious activities are taking place within the storage device, process 400 will then proceed to step 425 whereby a suitable alarm or warning will be raised or alternatively the storage device may be locked down. Process 400 then ends. Conversely, if no malicious activities are detected by process 400 at step 420, process 400 will then end. Process 400 will then repeat itself each time the storage device is booted up or started up so that it would be able to detect any malicious activities that may take place.
Figure 5 illustrates process 500 that may be implemented in module 250 for priming a trained neural network retrieved from a data module during the boot up or start-up of the associated host device. Process 500 begins at step 505 by selecting a set of values or magic numbers from the content found at the first LBA (as copied by process 400 in step 405) or at other LBAs if the content extends beyond the first LBA. These set of values or magic numbers are then compared with a magic number lookup table that was preloaded into a cache or buffer of the storage device. By matching the set of values/magic numbers with that contained in the magic number lookup table, process 500 is then able to determine the type of operating system and its file system or be able to determine the type of system configuration (e.g. secondary storage file system) associated with the file system of the storage device. LBAs that are critical to the identified file system are also identified at this step so that the trained neural network will be made aware that it has to monitor these LBAs. Process 500 then proceeds to prime the trained neural network based on this information at step 515 thereby accelerating the detection speed of the trained neural network as it would have been primed to be used with the appropriate file system.
Numerous other changes, substitutions, variations and modifications may be ascertained by the skilled in the art and it is intended that the present invention encompass all such changes, substitutions, variations and modifications as falling within the scope of the appended claims.

Claims (16)

CLAIMS :
1. A module for detecting malicious activity in a storage device, whereby the module is provided within a controller of the storage device, the module being configured to: retrieve a first logical block address (LBA0) sent to the controller from a host device and retrieve content at the first LBA0, whereby the retrieved content is used to prime a trained neural network retrieved from a data module provided within the controller; identify, using content at the first LBA0 and the trained neural network, logical block addresses (LBAs) of the storage device that are to be monitored; mirror instructions sent to the identified LBAs by the host device to the controller, and mirror contents of the mirrored LBAs; determine, using the trained neural network, if malicious activity is occurring at the storage device based on the mirrored instructions and contents, wherein the neural network is trained for different types of operating systems or secondary storage operations based on average read/write/overwrite access of contents at LBAs related to master boot records, critical records, master file tables, boot sectors, BIOS parameter blocks or extended BIOS parameter blocks of file systems associated with the operating systems or the secondary storage operations.
2. The module according to claim 1 wherein the priming of the associated trained neural network comprises the module being configured to: select a set of magic numbers from the content at the first LBA0, whereby the selected set of magic numbers are used with a magic number lookup table to determine a type of operating system or a type of secondary storage operation associated with a file system of the controller, whereby the magic number lookup table is obtained from the data module; and prime the trained neural network to detect malicious activities related to the determined type of operating system or secondary storage operation from the data module.
3. The module according to claim 2 wherein the priming the trained neural network comprises the module being configured to select a trained neural network that has been optimized for the determined type of operating system or the determined type of secondary storage operation associated with the file system of the controller.
4. The module according to claim 1 wherein the identifying LBAs of the storage device that are to be monitored comprises the module being configured to: identify, based on a determined type of operating system or secondary storage operation associated with the primed trained neural network, LBAs that contain critical data whereby the critical data comprises at least a master file table, a master boot record, a boot sector, a BIOS parameter block or an extended BIOS parameter block of a file system associated with the storage device.
5. The module according to claim 1 wherein the module is further configured to: optimize the trained neural network using the malicious activity determined to have occurred at the storage device based on the mirrored instructions and contents.
6. The module according to claims 1 or 5 wherein the module is further configured to: lockdown the storage device in response to a determination that malicious activity is determined to have occurred at the storage device based on the mirrored instructions and contents.
7. The module according to any one of claims 1 to 6 wherein the trained neural network comprises an artificial neural network.
8. The module according to claim 7 wherein the artificial neural network comprises a Recurrent Neural Network (RNN) or a Convolutional Neural Network (CNN).
9. A method for detecting malicious activity in a storage device comprising the steps of: retrieving, using a module provided within a controller of the storage device, a first logical block address (LBA0) sent to the controller from a host device; retrieving, using the module, content at the first LBA0, whereby the retrieved content is used to prime a trained neural network retrieved from a data module provided within the controller; identifying using content at the first LBA0 and the trained neural network, using the module, logical block addresses (LBAs) of the storage device that are to be monitored; mirroring, using the module, instructions sent to the identified LBAs by the host device to the controller, and mirror contents of the mirrored LBAs; and determining, using the trained neural network, if malicious activity is occurring at the storage device based on the mirrored instructions and contents, wherein the neural network is trained for different types of operating systems or secondary storage operations based on average read/write/overwrite access of contents at LBAs related to master boot records, critical records, master file tables, boot sectors, BIOS parameter blocks or extended BIOS parameter blocks of file systems associated with the operating systems or the secondary storage operations.
10. The method according to claim 9 wherein the priming of the associated trained neural network comprises the steps of: selecting, using the module, a set of magic numbers from the content at the first LBA0, whereby the selected set of magic numbers are used with a magic number lookup table to determine a type of operating system or a type of secondary storage operation associated with a file system of the controller, whereby the magic number lookup table is obtained from the data module; and priming, using the module, the trained neural network to detect malicious activities related to the determined type of operating system or secondary storage operation from the data module.
11. The method according to claim 10 wherein the priming the trained neural network comprises the step of selecting a trained neural network that has been optimized for the determined type of operating system or the determined type of secondary storage operation associated with the file system of the controller.
12. The method according to claim 9 wherein the identifying LBAs of the storage device that are to be monitored comprises the steps of: identifying, using the module, based on a determined type of operating system or secondary storage operation associated with the primed trained neural network, LBAs that contain critical data whereby the critical data comprises at least a master file table, a master boot record, a boot sector, a BIOS parameter block or an extended BIOS parameter block of a file system associated with the storage device.
13. The method according to claim 9 wherein the method further comprises the step of: optimizing, using the module, the trained neural network using the malicious activity determined to have occurred at the storage device based on the mirrored instructions and contents.
14. The method according to claims 9 or 13 wherein the method further comprises the step locking down, using the module, the storage device in response to a determination that malicious activity is determined to have occurred at the storage device based on the mirrored instructions and contents.
15. The method according to any one of claims 9 to 14 wherein the trained neural network comprises an artificial neural network.
16. The method according to claim 15 wherein the artificial neural network comprises a Recurrent Neural Network (RNN) or a Convolutional Neural Network (CNN).
NZ778473A 2018-12-28 2019-12-19 Systems and methods for processing metadata NZ778588A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US62/786,069 2018-12-28
US16/715,965 2019-12-16

Publications (1)

Publication Number Publication Date
NZ778588A true NZ778588A (en) 2021-08-27

Family

ID=

Similar Documents

Publication Publication Date Title
CN107632939B (en) Mapping table for storage device
US11126544B2 (en) Method and apparatus for efficient garbage collection based on access probability of data
US11657022B2 (en) Host and storage system for securely deleting files and operating method of the host
US10360144B2 (en) Storage apparatus and non-volatile memory device including a controller to selectively compress data based on an update frequency level
US20150169465A1 (en) Method and system for dynamic compression of address tables in a memory
US20130151761A1 (en) Data storage device storing partitioned file between different storage mediums and data management method
KR20110107856A (en) Logical address offset
CN101174473A (en) Method of providing block state information in semiconductor memory device including flash memory
US10459803B2 (en) Method for management tables recovery
US8954692B2 (en) File protecting method and system, and memory controller and memory storage apparatus thereof
US8433847B2 (en) Memory drive that can be operated like optical disk drive and method for virtualizing memory drive as optical disk drive
CN111984441B (en) Instant power-off recovery processing method and device and computer readable storage medium
US10884857B2 (en) Data storage device and method of operating
KR20210097493A (en) Data classifing method in memory systems and controller perfroming the same
US20170017406A1 (en) Systems and methods for improving flash-oriented file system garbage collection
US11347860B2 (en) Randomizing firmware loaded to a processor memory
US11055443B1 (en) Module and method for detecting malicious activities in a storage device
CN100518061C (en) Disk memory system with once written and multiple read and design method thereof
NZ778588A (en) Systems and methods for processing metadata
NZ778067B2 (en) Module and method for detecting malicious activities in a storage device
US20220206904A1 (en) Device and method for managing recovery information of auxiliary storage device
KR102497130B1 (en) Storage device and operating method thereof
US9274709B2 (en) Indicators for storage cells
RU2757659C1 (en) Magnetic drive with divided areas
CN116756106A (en) Compression method and device for ordered string table file