US20200218472A1 - Storage apparatus, storage control device, and recording medium - Google Patents
Storage apparatus, storage control device, and recording medium Download PDFInfo
- Publication number
- US20200218472A1 US20200218472A1 US16/720,815 US201916720815A US2020218472A1 US 20200218472 A1 US20200218472 A1 US 20200218472A1 US 201916720815 A US201916720815 A US 201916720815A US 2020218472 A1 US2020218472 A1 US 2020218472A1
- Authority
- US
- United States
- Prior art keywords
- command
- processing
- task
- controller
- status
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/541—Interprogram communication via adapters, e.g. between incompatible applications
Abstract
Description
- This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2019-1235, filed on Jan. 8, 2019, the entire contents of which are incorporated herein by reference.
- The embodiment discussed herein is related to a storage control technology.
- A firmware active application function is one of important functions of storage apparatuses. The firmware active application function revises firmware of a storage apparatus while keeping the storage apparatus online without stopping the storage apparatus.
- The storage apparatus includes a plurality of modules. Each module operates with independent firmware. Each module includes two regions to which firmware is written: an operating region and a non-operating region. Upon the firmware active application, revised firmware is written to the non-operating region. After that, the non-operating region is switched to the operating region. At the time of reboot, the storage apparatus starts up with the revised firmware.
- A storage control device includes two systems: a master system and another system that operates when the master system has an abnormality. The firmware is revised in one system after the other, allowing revision of the firmware while keeping the storage apparatus online.
- There is a technique that monitors an access to a target storage apparatus to which firmware is applied and provides instructions to apply the firmware to the target storage apparatus based on the monitoring result. This technique enables application of the firmware even when the target storage apparatus is in operation.
- Another technique improves an operability of a higher-layer program, which is operated by an user, without the user being directly aware of the operation of a lower-layer program in a system that provides a single function by calling a plurality of programs hierarchically. In this technique, when the lower-layer program receives a large number of requests, the lower-layer program identifies relationships between the timeout requirements of the requests and tasks that are the basis of the respective requests. According to the execution order of the tasks in a program that has issued the requests and the timeout requirement of each request, the lower-layer program controls the execution order of the requests so as to reduce the timeout as much as possible.
- Examples of the techniques are disclosed in Japanese Laid-open Patent Publication No. 2009-282834 and International Publication Pamphlet No. WO 2017/056208.
- According to an aspect of the embodiment, a storage apparatus includes a non-volatile storage device configured to store data; a memory; and a processor coupled to the memory and configured to control access to the non-volatile storage device, the processor configured to: receive an instruction from external software as a task, generate a command set for controlling the non-volatile storage device, execute a command included in the generated command set, control firmware revision processing of the storage apparatus, and stop execution of the command after performing processing for suppressing abnormality detection of the external software at a timing at which the command is not executable during the firmware revision processing.
- The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
- It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention.
-
FIG. 1 is a diagram illustrating a configuration of a storage apparatus according to an embodiment; -
FIG. 2 is a diagram illustrating a functional configuration of a CM; -
FIG. 3 is a diagram illustrating an example of a command set; -
FIG. 4 is a diagram illustrating status of a task managed by a command manager; -
FIG. 5 is a diagram illustrating a pause operation instep 1; -
FIG. 6 is a diagram illustrating a pause operation instep 2; -
FIG. 7 is a diagram illustrating a complete stop operation instep 1; -
FIG. 8 is a diagram illustrating a complete stop operation instep 2; -
FIG. 9 is a diagram illustrating a flow of asynchronous command processing; -
FIG. 10 is a flowchart illustrating a flow of firmware revision processing; -
FIGS. 11A and 11B are first diagrams illustrating an operation of firmware active application; -
FIGS. 11C and 11D are second diagrams illustrating an operation of firmware active application; -
FIGS. 11E and 11F are third diagrams illustrating an operation of firmware active application; -
FIGS. 11G and 11H are fourth diagrams illustrating an operation of firmware active application; -
FIG. 12 is a flowchart illustrating a flow of a pause instruction; and -
FIG. 13 is a diagram illustrating an example of a configuration of another storage apparatus. - In recent years, there have been an increasing number of environments that automate operations in cloud systems or use virtualization software. In an environment that automates operations in a cloud system or uses virtualization software, software that operates on a server performs configuration control over a storage apparatus. The configuration control refers to settings of an apparatus such as redundant arrays of inexpensive disks (RAID) settings and changes of the configuration of the apparatus such as changes of the RAID configuration. In some cases, the configuration control over the storage apparatus is performed during the firmware active application.
- However, the firmware active application involves reboot of a module to be revised. In order to maintain integrity of various control tables in the module between two systems, there is a timing at which the configuration control from external software is not acceptable. The external software refers to the software that operates on the server.
- The technique has an issue that, when the configuration control from the external software is not acceptable, the external software detects an abnormality and the operation of a system using the external software stops. A storage control device may accept the configuration control from the user as well as from the external software.
- An embodiment of a storage apparatus, a storage control device, and a storage control program disclosed in the present application will be hereinafter described in detail with reference to the drawings. The following embodiment does not limit the disclosed technology.
- A configuration of a storage apparatus according to an embodiment will be described.
FIG. 1 is a diagram illustrating a configuration of a storage apparatus according to an embodiment. As illustrated inFIG. 1 , astorage apparatus 1 according to the embodiment includes a controller enclosure (CE) 1 a and twenty drive enclosures (DEs) 1 b denoted asDE# 00 toDE# 19. The CE 1 a is a housing including two controller modules (CMs) 10 denoted asCM# 0 andCM# 1. TheDEs 1 b are housings each including hard disk drives (HDDs) 23. TheDEs 1 b may include other non-volatile storage devices such as solid-state drives (SSDs) instead of theHDDs 23. Alternatively, theDEs 1 b may include a plurality of types of non-volatile storage devices. - The
CMs 10 are control devices that control thestorage apparatus 1. TheCM# 0 and theCM# 1 constitute a redundant system. One of theCM# 0 and theCM# 1 operates as a master. EachCM 10 includes a central processing unit (CPU) 11, amemory 11 a, an input/output controller (IOC) 12, a CM-EXPander (EXP) 13, and a CM-basic input/output system (BIOS) 14. EachCM 10 also includes a bootup and utility device (BUD) 15 and two channel adapters (CAs) 16. - The
CPU 11 causes the correspondingCM 10 to operate as the control device by executing firmware stored in thememory 11 a. Thememory 11 a is a random-access memory (RAM) that stores, for example, the firmware to be executed by theCPU 11 and data to be used by the firmware. The firmware to be executed by theCPU 11 is read from a flash memory and stored in thememory 11 a, for example. Alternatively, the firmware to be executed by theCPU 11 may be read from a compact disc-recordable (CD-R), which is a computer-readable medium, and stored in thememory 11 a. - The
IOC 12 is a serial attached SCSI (SAS) controller. The CM-EXP 13 couples theCM 10 with theDEs 1 b by the SAS. The CM-BIOS 14 is a read-only memory (ROM) storing a BIOS. The BIOS starts an operating system (OS) that operates on theCPU 11. TheBUD 15 stores configuration setting information regarding thestorage apparatus 1, for example. EachCA 16 is an interface with an information processing apparatus such as a server. The information processing apparatus such as a server writes data to thestorage apparatus 1 and reads data from thestorage apparatus 1. - Each
DE 1 b includes I/O modules (IOMs) 21 denoted asIOM# 0 andIOM# 1. EachIOM 21 transfers an SAS frame to theHDD 23 specified by the correspondingCM 10. Each of theDEs 1 b denoted as theDE# 00 and theDE# 19 includes fan expander modules (FEMs) 22 denoted asFEM# 00 andFEM# 01. Each of theother DEs 1 b includes the plurality ofHDDs 23. EachFEM 22 includes the plurality ofHDDs 23 and transfers the SAS frame transferred by the correspondingIOM 21 to theHDD 23 specified by the correspondingCM 10. EachFEM 22 includes a fan and cools theDE 1 b. - The
storage apparatus 1 may include more orfewer DEs 1 b. More orfewer DEs 1 b may include theFEMs 22. - A functional configuration of CMs will be described below.
FIG. 2 is a diagram illustrating a functional configuration of one of CMs. The CMs illustrated inFIG. 2 may beCMs 10 illustrated inFIG. 1 . Since the twoCMs 10 include a like functional configuration, the functional configuration of one of theCMs 10 is illustrated inFIG. 2 . The functions of theCM 10 are implemented by theCPU 11 executing the firmware stored in thememory 11 a. As illustrated inFIG. 2 , theCM 10 includes a command-line interface (CLI) controller 41, acommand manager 42, acommand controller 43, aconfiguration controller 44, and anIO controller 45. - The CLI controller 41 receives a CLI command from an application programming interface (API)
provider 32. When the received CLI command is a synchronous command, the CLI controller 41 passes the received CLI command to thecommand controller 43. When the received CLI command is an asynchronous command, the CLI controller 41 generates a corresponding command set and passes the command set to thecommand manager 42. - The
API provider 32 is software that accepts API execution fromexternal software 31, converts the accepted API into one or more CLI commands, and transmits the CLI command(s) to thestorage apparatus 1. Theexternal software 31 operates on aserver 3. TheAPI provider 32 receives the status and result of the execution of the CLI command(s) from thestorage apparatus 1 and responds to theexternal software 31. Although theAPI provider 32 operates on theserver 3 in the embodiment, thestorage apparatus 1 may include the functions of theAPI provider 32. - The synchronous command is a command that the
external software 31 expects to receive a response at the time of the API return. The asynchronous command is a command that theexternal software 31 checks the progress of the API execution. There are two types of commands in the asynchronous command: a command that is converted from an API having a timeout requirement; and a command that is converted from an API having no timeout requirement. Thestorage apparatus 1 detects a timing at which no synchronous command is being processed, and revises the firmware. -
FIG. 3 is a diagram illustrating an example of a command set. As illustrated inFIG. 3 , the command set includes an API name, the number of commands in the command set, and commands corresponding to the number of commands. InFIG. 3 , the number of commands is denoted as “n” (a positive integer). Each command includes a command code and control data for the command code. The API name identifies the API that is the basis of the command set. - The
command manager 42 passes the commands included in the command set to thecommand controller 43 one by one and requests thecommand controller 43 for execution. Thecommand manager 42 associates an API with a task and manages task information using a queue. When thecommand manager 42 receives an inquiry about the progress of the API execution from theexternal software 31, thecommand manager 42 responds with the progress of the task through the CLI controller 41 and theAPI provider 32. -
FIG. 4 is a diagram illustrating status of a task managed by a command manager. The command manager illustrated inFIG. 4 may be thecommand manager 42 illustrated inFIG. 2 . As illustrated inFIG. 4 , the status of the task includes “Queued,”“Running,”“Success,” and “Error.” “Queued” indicates that the task information is queued and the task is in a pre-execution state. “Running” indicates that the task has started and the command execution has started. “Success” indicates that all commands of the task have ended normally. “Error” indicates that the command execution of the task has failed. The task information is synchronized between the twoCMs 10. - The
command controller 43 executes the synchronous command received from the CLI controller 41 and the command received from thecommand manager 42. Thecommand controller 43 notifies the CLI controller 41 of the execution result of the synchronous command, and notifies thecommand manager 42 of the execution result of the command. Thecommand controller 43 requests theconfiguration controller 44 to process a command related to the configuration control. When thecommand controller 43 is required to access theHDD 23 at the time of the execution of the synchronous command and at the time of the execution of the command, thecommand controller 43 requests theIO controller 45 to perform processing. When theserver 3 requests theIO controller 45 to read or write data, theIO controller 45 receives the request from theserver 3 through a path different from the path illustrated inFIG. 2 . - The
configuration controller 44 performs the configuration control and notifies thecommand controller 43 of the result of the configuration control. When theconfiguration controller 44 is required to access theHDD 23 at the time of the configuration control, theconfiguration controller 44 requests theIO controller 45 to perform processing. - When the
IO controller 45 receives a request for processing from thecommand controller 43, theIO controller 45 notifies thecommand controller 43 of the processing result. When theIO controller 45 receives a request for processing from theconfiguration controller 44, theIO controller 45 notifies theconfiguration controller 44 of the processing result. TheIO controller 45 receives a data reading or writing request from theserver 3 through the path different from the path illustrated inFIG. 2 , and controls writing or reading of data to or from theHDD 23. - The
command controller 43 includes arevision controller 50. Therevision controller 50 controls firmware active application. During the firmware active application processing, therevision controller 50 provides a stop instruction to thecommand manager 42 at a timing at which the configuration control performed using the API is not acceptable. - Examples of the timing at which the configuration control is not acceptable during the firmware active application processing include: the timing of separating and incorporating the
CM 10 during the firmware revision of theCM 10; and the timing of synchronizing a cache table during the firmware revision of theCM 10. Another example of the timing at which the configuration control is not acceptable during the firmware active application processing includes the timing of switching themaster CM 10 serving as the master. - Before starting the separation and incorporation of the
CM 10, accordingly, therevision controller 50 provides a pause instruction to thecommand manager 42. Before starting the synchronization of the cache table, therevision controller 50 also provides the pause instruction to thecommand manager 42. - When the
command manager 42 receives the pause instruction, asstep 1, thecommand manager 42 preferentially processes an API having the timeout requirement such that theexternal software 31 does not detect the timeout, and performs command sweeping processing before stopping. The command sweeping processing refers to processing performed until the status of the task becomes “Success” or “Error (failure).” In the case of an API having no timeout requirement, asstep 1, thecommand manager 42 stops the processing of the next command after receiving the processing result of a command being processed. -
FIG. 5 is a diagram illustrating a pause operation instep 1. As illustrated inFIG. 5 , when there is the timeout requirement and the status of the task indicates “Queued” or “Running,” thecommand manager 42 performs the command sweeping processing. Thecommand manager 42 continues to perform the processing until the status of the task is determined to be “Success” or “Error.” When there is the timeout requirement and the status of the task indicates “Success” or “Error,” thecommand manager 42 retains the task information, and deletes the task information after notification of “success” or “failure.” - When the
command manager 42 receives the pause instruction, there is no timeout requirement, and the status of the task indicates “Queued,” thecommand manager 42 maintains “Queued.” For example, thecommand manager 42 stops the execution of the task in “Queued” status. When thecommand manager 42 receives the pause instruction, there is no timeout requirement, and the status of the task indicates “Running,” thecommand manager 42 stops the processing upon receiving a command response. For example, thecommand manager 42 stops the execution of the next command of the task in “Running” status. When thecommand manager 42 receives the pause instruction, there is no timeout requirement, and the status of the task indicates “Success” or “Error,” thecommand manager 42 retains the task information, and deletes the task information after notification of “success” or “failure.” - After completion of the pause processing, the
command manager 42 responds with the completion of the pause in response to the status confirmation received from therevision controller 50. When therevision controller 50 receives the completion of the pause or when the first threshold time has elapsed, therevision controller 50 starts the processing for which therevision controller 50 has provided the pause instruction. For example, when therevision controller 50 has provided the pause instruction before starting the separation and incorporation of theCM 10, therevision controller 50 starts the separation and incorporation of theCM 10. When therevision controller 50 has provided the pause instruction before starting the synchronization of the cache table, therevision controller 50 starts the synchronization of the cache table. After completion of the separation and incorporation of theCM 10 or completion of the synchronization of the cache table, therevision controller 50 instructs thecommand manager 42 to resume. - When the
command manager 42 does not complete the pause operation instep 1 within the second threshold (<first threshold) time, thecommand manager 42 proceeds to step 2.FIG. 6 is a diagram illustrating a pause operation instep 2. As illustrated inFIG. 6 , when there is the timeout requirement and the status of the task indicates “Queued,” thecommand manager 42 maintains “Queued.” When there is the timeout requirement and the status of the task indicates “Running,” thecommand manager 42 performs the command sweeping processing and continues to perform the processing until the status of the task is determined to be “Success” or “Error.” When there is the timeout requirement and the status of the task indicates “Success” or “Error,” thecommand manager 42 retains the task information, and deletes the task information after notification of “success” or “failure.” - When there is no timeout requirement and the status of the task indicates “Queued” in the pause operation in
step 2, thecommand manager 42 maintains “Queued.” When there is no timeout requirement and the status of the task indicates “Running” in the pause operation instep 2, thecommand manager 42 stops the processing upon receiving a command response. When there is no timeout requirement and the status of the task indicates “Success” or “Error” in the pause operation instep 2, thecommand manager 42 retains the task information, and deletes the task information after notification of “success” or “failure.” - When the
master CM 10 is switched, the network is once reset. Thus, the communication between theexternal software 31 and themaster CM 10 is temporarily disconnected. After theexternal software 31 detects disconnection from thestorage apparatus 1, theexternal software 31 is not able to continue the processing and detects an abnormality depending on the number of connection retries. In order to avoid the abnormality detection of theexternal software 31 as much as possible, therevision controller 50 provides a complete stop instruction to thecommand manager 42 before switching and continues the firmware revision with no queued task left. - When the
command manager 42 receives the complete stop instruction, asstep 1, thecommand manager 42 performs the command sweeping processing before stopping regardless of whether or not there is the timeout requirement. -
FIG. 7 is a diagram illustrating a complete stop operation instep 1. As illustrated inFIG. 7 , when the status of the task indicates “Queued” or “Running,” thecommand manager 42 performs the command sweeping processing regardless of whether or not there is the timeout requirement. When the status of the task indicates “Success” or “Error,” thecommand manager 42 retains the task information, and deletes the task information after notification of “success” or “failure” regardless of whether or not there is the timeout requirement. - After completion of the complete stop operation, the
command manager 42 responds with the completion of the complete stop in response to the status confirmation received from therevision controller 50. When therevision controller 50 receives the completion of the complete stop or when the first threshold time has elapsed, therevision controller 50 starts switching theCM 10. After completion of the switching of theCM 10, therevision controller 50 instructs thecommand manager 42 to resume. - When the
command manager 42 does not complete the complete stop operation instep 1 within the second threshold time, thecommand manager 42 proceeds to step 2.FIG. 8 is a diagram illustrating a complete stop operation instep 2. As illustrated inFIG. 8 , when there is the timeout requirement and the status of the task indicates “Queued,” thecommand manager 42 maintains “Queued.” When there is the timeout requirement and the status of the task indicates “Running,” thecommand manager 42 performs the command sweeping processing. When there is the timeout requirement and the status of the task indicates “Success” or “Error,” thecommand manager 42 retains the task information, and deletes the task information after notification of “success” or “failure.” - When there is no timeout requirement and the status of the task indicates “Queued” in the complete stop operation in
step 2, thecommand manager 42 performs the command sweeping processing. When there is no timeout requirement and the status of the task indicates “Running” in the complete stop operation instep 2, thecommand manager 42 stops the processing upon receiving a command response. When there is no timeout requirement and the status of the task indicates “Success” or “Error” in the complete stop operation instep 2, thecommand manager 42 retains the task information, and deletes the task information after notification of “success” or “failure.” - A flow of asynchronous command processing will be described below.
FIG. 9 is a diagram illustrating a flow of asynchronous command processing. As illustrated inFIG. 9 , theexternal software 31 executes an API (t1). Theexternal software 31 also performs polling for the result of processing of thestorage apparatus 1 regarding the executed API (t1′). - The
API provider 32 accepts the API execution, converts the accepted API into one or more CLI commands, and instructs thestorage apparatus 1 to execute the CLI command(s) (t2). The CLI controller 41 generates a command set from an asynchronous command and requests thecommand manager 42 for execution (t3). - The
command manager 42 extracts commands one by one from the command set and requests thecommand controller 43 to process the command (t4). Thecommand controller 43 requests theconfiguration controller 44 to process the configuration control (t5). Thecommand manager 42 manages the progress of the command processing (t6). When thecommand manager 42 receives the processing result of the requested command from thecommand controller 43, thecommand manager 42 extracts the next command from the command set and requests thecommand controller 43 to process the command (t4). - In this way, the processing from t4 to t6 is repeated until the command processing is completed. However, when the firmware active application is performed, the command processing may be stopped. For example, an asynchronous command having no timeout requirement is more likely to be stopped than an asynchronous command having the timeout requirement.
- A flow of firmware revision processing will be described below.
FIG. 10 is a flowchart illustrating a flow of firmware revision processing. “Revise” inFIG. 10 refers to the revision of the firmware. As illustrated inFIG. 10 , therevision controller 50 revises theIOMs 21 and the FEMs 22 (step S1). - The
revision controller 50 revises the CM 10 (step S2). Therevision controller 50 provides the pause instruction to thecommand manager 42 at the timing of starting the separation and incorporation of theCM 10 during the revision of the CM 10 (step S3). When thecommand manager 42 receives the pause instruction, thecommand manager 42 performs the pause processing (step S4). After completion of the pause processing, thecommand manager 42 responds with the completion in response to the status confirmation received from the revision controller 50 (step S5). When therevision controller 50 receives the completion response from thecommand manager 42 or when the first threshold time has elapsed, therevision controller 50 starts the separation and incorporation of theCM 10. After completion of the separation and incorporation of theCM 10, therevision controller 50 instructs thecommand manager 42 to resume processing. - The
revision controller 50 provides the pause instruction to thecommand manager 42 at the timing of starting the synchronization of the cache table (step S3). When thecommand manager 42 receives the pause instruction, thecommand manager 42 performs the pause processing (step S4). After completion of the pause processing, thecommand manager 42 responds with the completion in response to the status confirmation received from the revision controller 50 (step S5). When therevision controller 50 receives the completion response from thecommand manager 42 or when the first threshold time has elapsed, therevision controller 50 starts the synchronization of the cache table. After completion of the synchronization of the cache table, therevision controller 50 instructs thecommand manager 42 to resume processing. - Before switching the
master CM 10, therevision controller 50 provides the complete stop instruction to the command manager 42 (step S6). When thecommand manager 42 receives the complete stop instruction, thecommand manager 42 performs the complete stop processing (step S7). After completion of the complete stop processing, thecommand manager 42 responds with the completion in response to the status confirmation received from the revision controller 50 (step S8). When therevision controller 50 receives the completion response from thecommand manager 42 or when the first threshold time has elapsed, therevision controller 50 switches the master CM 10 (step S9). After completion of the switching of themaster CM 10, therevision controller 50 instructs thecommand manager 42 to resume processing. - The
revision controller 50 revises the CM 10 (step S10). Therevision controller 50 provides the pause instruction to thecommand manager 42 at the timing of starting the separation and incorporation of theCM 10 during the revision of the CM 10 (step S11). When thecommand manager 42 receives the pause instruction, thecommand manager 42 performs the pause processing (step S12). After completion of the pause processing, thecommand manager 42 responds with the completion in response to the status confirmation received from the revision controller 50 (step S13). When therevision controller 50 receives the completion response from thecommand manager 42 or when the first threshold time has elapsed, therevision controller 50 starts the separation and incorporation of theCM 10. After completion of the separation and incorporation of theCM 10, therevision controller 50 instructs thecommand manager 42 to resume processing. - The
revision controller 50 provides the pause instruction to thecommand manager 42 at the timing of starting the synchronization of the cache table (step S11). When thecommand manager 42 receives the pause instruction, thecommand manager 42 performs the pause processing (step S12). After completion of the pause processing, thecommand manager 42 responds with the completion in response to the status confirmation received from the revision controller 50 (step S13). When therevision controller 50 receives the completion response from thecommand manager 42 or when the first threshold time has elapsed, therevision controller 50 starts the synchronization of the cache table. After completion of the synchronization of the cache table, therevision controller 50 instructs thecommand manager 42 to resume processing. - Before switching the
master CM 10, therevision controller 50 provides the complete stop instruction to the command manager 42 (step S14). When thecommand manager 42 receives the complete stop instruction, thecommand manager 42 performs the complete stop processing (step S15). After completion of the complete stop processing, thecommand manager 42 responds with the completion in response to the status confirmation received from the revision controller 50 (step S16). When therevision controller 50 receives the completion response from thecommand manager 42 or when the first threshold time has elapsed, therevision controller 50 switches the master CM 10 (step S17). After completion of the switching of themaster CM 10, therevision controller 50 instructs thecommand manager 42 to resume processing. After that, therevision controller 50 revises theIOMs 21 and the FEMs 22 (step S18). - During the firmware active application processing, the
revision controller 50 provides the stop instruction to thecommand manager 42 as described above at the timing at which the configuration control performed using the API is not acceptable. When thecommand manager 42 receives the stop instruction, thecommand manager 42 stops the processing after taking measures such that theexternal software 31 does not detect an abnormality. With this configuration, therevision controller 50 may minimize such a situation that theexternal software 31 performs the configuration control and detects an abnormality during the firmware active application. - An operation of a firmware active application will be described below with reference to
FIGS. 11A to 11H .FIGS. 11A to 11H are diagrams illustrating an operation of a firmware active application. InFIGS. 11A to 11H , “M” surrounded by a circle refers to themaster CM 10.FIG. 11A illustrates a state before the firmware active application processing starts. InFIG. 11A , theCM# 0, theIOM# 0, and theFEM# 00 operate as the master. - The
revision controller 50 first revises the firmware of theIOM# 1 system and theFEM# 01 system that do not operate as the master.FIG. 11B illustrates the firmware revision operation of theIOM# 1 system and theFEM# 01 system. As illustrated inFIG. 11B , the firmware of theIOM# 1 system and theFEM# 01 system is revised in theDE# 00 to theDE# 19. - Next, the
revision controller 50 revises the firmware of theCM# 1. During the firmware revision of theCM# 1, therevision controller 50 provides the pause instruction to thecommand manager 42 at the timing of separating and incorporating theCM# 1 and at the timing of synchronizing the cache table.FIG. 11C illustrates the firmware revision operation of theCM# 1. As illustrated inFIG. 11C , therevision controller 50 provides the pause instruction at the timing of separating and incorporating theCM# 1 and at the timing of synchronizing the cache table during the firmware revision executed by theCPU 11, theIOC 12, and the CM-EXP 13 of theCM# 1. - Next, the
revision controller 50 switches themaster CM 10. Therevision controller 50 provides the complete stop instruction to thecommand manager 42 before switching themaster CM 10.FIG. 11D illustrates the operation of switching themaster CM 10. As illustrated inFIG. 11D , themaster CM 10 is switched from theCM# 0 to theCM# 1 after the complete stop instruction is provided. - Next, the
revision controller 50 revises the firmware of theCM# 0. During the firmware revision of theCM# 0, therevision controller 50 provides the pause instruction to thecommand manager 42 at the timing of separating and incorporating theCM# 0 and at the timing of synchronizing the cache table.FIG. 11E illustrates the firmware revision operation of theCM# 0. As illustrated inFIG. 11E , therevision controller 50 provides the pause instruction at the timing of separating and incorporating theCM# 0 and at the timing of synchronizing the cache table during the firmware revision executed by theCPU 11, theIOC 12, and the CM-EXP 13 of theCM# 0. - Next, the
revision controller 50 switches back themaster CM 10. Therevision controller 50 provides the complete stop instruction to thecommand manager 42 before switching back themaster CM 10.FIG. 11F illustrates the switch-back operation of themaster CM 10. As illustrated inFIG. 11F , themaster CM 10 is switched back from theCM# 1 to theCM# 0 after the complete stop instruction is provided. - Next, the
revision controller 50 revises the firmware of theIOM# 0 system and theFEM# 00 system that do not operate as the master.FIG. 11G illustrates the firmware revision operation of theIOM# 0 system and theFEM# 00 system. As illustrated inFIG. 11G , the firmware of theIOM# 0 system and theFEM# 00 system is revised in theDE# 00 to theDE# 19. - The
revision controller 50 ends the firmware revision.FIG. 11H illustrates a state after completion of the firmware revision. InFIG. 11H , the state of thestorage apparatus 1 returns to the state inFIG. 11A , which illustrates the state before the firmware revision. - A flow of the pause instruction will be described below.
FIG. 12 is a flowchart illustrating a flow of a pause instruction. As illustrated inFIG. 12 , therevision controller 50 provides the pause instruction to the command manager 42 (step S21). Therevision controller 50 inquires of thecommand manager 42 about the status and acquires the status from the command manager 42 (step S22). On the basis of the acquired status, therevision controller 50 determines whether or not thecommand manager 42 has completed the pause processing (step S23). - When the
command manager 42 has completed the pause processing, therevision controller 50 proceeds to the next processing. When thecommand manager 42 has not completed the pause processing, therevision controller 50 determines whether or not the first threshold time has elapsed (step S24). When the first threshold time has not elapsed, therevision controller 50 returns to step S22. When the first threshold time has elapsed, therevision controller 50 proceeds to the next processing. - As described above, the
revision controller 50 proceeds to the next processing when thecommand manager 42 does not complete the pause processing within the first threshold time. This configuration may reduce the interruption time during the firmware revision processing. The complete stop instruction is also processed in a flow similar to the flow inFIG. 12 . - As described above, in the embodiment, the
API provider 32 accepts API execution of theexternal software 31 and converts the accepted API into CLI commands. The CLI controller 41 generates a command set from an asynchronous command among the CLI commands and passes the command set to thecommand manager 42. Thecommand manager 42 extracts commands one by one from the command set and requests thecommand controller 43 for execution. Accordingly, thecommand controller 43 executes the command. Thecommand controller 43 also controls the firmware revision of thestorage apparatus 1 and instructs thecommand manager 42 to stop the command execution at the timing at which the command execution is not possible during the revision processing. Thecommand manager 42 stops the command execution after performing the processing for suppressing the abnormality detection of theexternal software 31. With this configuration, theCM 10 may minimize such a situation that theexternal software 31 detects an abnormality and stops theserver 3. - In the embodiment, the
command controller 43 provides the pause instruction to thecommand manager 42 at the timing of separating and incorporating theCM 10 and at the timing of synchronizing the cache table between theCMs 10 during the firmware revision of theCM 10. Thecommand controller 43 provides the complete stop instruction to thecommand manager 42 at the timing of switching themaster CM 10. With this configuration, thecommand controller 43 may provide the stop instruction at an appropriate timing. - In the embodiment, when the
command manager 42 receives the pause instruction, thecommand manager 42 performs the sweeping processing for a task associated with an API having the timeout requirement and stops the processing. For a task that has no timeout requirement, when thecommand manager 42 receives the pause instruction and the status of the task remains in the execution state, thecommand manager 42 stops the processing upon receiving a response to the command being executed. With this configuration, thecommand manager 42 may minimize such a situation that theexternal software 31 detects an abnormality due to the timeout of the API execution. - In the embodiment, even for the API that has the timeout requirement, when the
command manager 42 receives the pause instruction and does not complete the sweeping processing within the second threshold time with the status of the task remaining in the pre-execution state, thecommand manager 42 leaves the status of the task in the pre-execution state and stops the processing. With this configuration, thecommand manager 42 may not cause a significant delay in the firmware revision processing. - In the embodiment, when the
command manager 42 receives the complete stop instruction, thecommand manager 42 performs the sweeping processing and stops the processing regardless of whether or not the API has the timeout requirement. With this configuration, thecommand controller 43 may switch themaster CM 10 while the asynchronous command is not executed. - In the embodiment, even for the API that has the timeout requirement, when the
command manager 42 does not complete the sweeping processing within the second threshold time and the status of the task remains in the pre-execution state, thecommand manager 42 leaves the status of the task in the pre-execution state and stops the processing. With this configuration, thecommand manager 42 may not cause a significant delay in the firmware revision processing. - The storage apparatus according to the embodiment may include various configurations other than the configuration illustrated in
FIG. 1 . An example of a configuration of another storage apparatus will be described below.FIG. 13 is a diagram illustrating an example of a configuration of another storage apparatus. As illustrated inFIG. 13 , astorage apparatus 2 includes 12CEs 2 a denoted asCE# 0 to CE#B, 192DEs 2 b denoted asDE# 00 to DE#BF, and a front enclosure (FE) 2 c. - The
FE 2 c is a housing including two front modules (FMs) 60 denoted asFM# 0 andFM# 1. EachFM 60 manages theCEs 2 a and relays communication between theCEs 2 a. TheFM# 0 includes a service controller (SVC) 61 denoted asSVC# 0 and two frontend routers (FRTs) 62 denoted asFRT# 0 andFRT# 1. TheFM# 1 includes anSVC 61 denoted asSVC# 1 and twoFRTs 62 denoted asFRT# 2 andFRT# 3. EachSVC 61 manages theCEs 2 a and monitors and controls modules mounted in theCEs 2 a. EachFRT 62 provides communication paths between theCEs 2 a. - Each
CE 2 a includes twoCMs 10 a denoted asCM# 0 andCM# 1. EachCM 10 a includes aCPU 11, twoIOCs 12, a CM-EXP 13, a CM-BIOS 14, fourCAs 16, and a non-transparent bridge (NTB) 17. TheNTB 17 is an interface coupled with theFRTs 62. - Each
DE 2 b includes twoIOMs 21 denoted asIOM# 0 andIOM# 1 and twoFEMs 22 denoted asFEM# 0 andFEM# 1. - The
storage apparatus 2 may include more orfewer CEs 2 a. Thestorage apparatus 2 may also include more orfewer DEs 2 b. - All examples and conditional language provided herein are intended for the pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed as limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although one or more embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
Claims (8)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JPJP2019-001235 | 2019-01-08 | ||
JP2019001235A JP7115322B2 (en) | 2019-01-08 | 2019-01-08 | Storage device, storage control device and storage control program |
JP2019-001235 | 2019-01-23 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20200218472A1 true US20200218472A1 (en) | 2020-07-09 |
US11113003B2 US11113003B2 (en) | 2021-09-07 |
Family
ID=71404408
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/720,815 Active 2039-12-31 US11113003B2 (en) | 2019-01-08 | 2019-12-19 | Storage apparatus, storage control device, and recording medium with execution command pausing or stopping |
Country Status (2)
Country | Link |
---|---|
US (1) | US11113003B2 (en) |
JP (1) | JP7115322B2 (en) |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6959413B2 (en) * | 2002-06-18 | 2005-10-25 | Lsi Logic Corporation | Method of handling unreadable blocks during rebuilding of a RAID device |
US7558915B2 (en) * | 2005-06-06 | 2009-07-07 | Dell Products L.P. | System and method for updating the firmware of a device in a storage network |
JP2009282834A (en) | 2008-05-23 | 2009-12-03 | Fujitsu Ltd | Disk array device, method for applying control firm, and control unit for controlling application of control firm |
JP5217966B2 (en) | 2008-11-28 | 2013-06-19 | 富士通株式会社 | Storage system update processing program, update processing method, and storage system |
JP5314731B2 (en) | 2011-06-10 | 2013-10-16 | 富士通株式会社 | Method and computer system for synchronizing basic input / output program between data processing devices |
US8694724B1 (en) * | 2011-09-06 | 2014-04-08 | Emc Corporation | Managing data storage by provisioning cache as a virtual device |
US9798534B1 (en) | 2015-07-01 | 2017-10-24 | EMC IP Holding Company LLC | Method and system to perform non-intrusive online disk firmware upgrades |
WO2017056208A1 (en) | 2015-09-30 | 2017-04-06 | 株式会社日立製作所 | Request execution order control system |
JP6555096B2 (en) * | 2015-11-18 | 2019-08-07 | 富士通株式会社 | Information processing apparatus and program update control method |
JP2018160156A (en) * | 2017-03-23 | 2018-10-11 | 東芝メモリ株式会社 | Memory system |
JP6812900B2 (en) | 2017-05-17 | 2021-01-13 | 富士通株式会社 | Storage device, storage control device, and storage control program |
-
2019
- 2019-01-08 JP JP2019001235A patent/JP7115322B2/en active Active
- 2019-12-19 US US16/720,815 patent/US11113003B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
JP2020112865A (en) | 2020-07-27 |
JP7115322B2 (en) | 2022-08-09 |
US11113003B2 (en) | 2021-09-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6273353B2 (en) | Computer system | |
US7770053B1 (en) | Systems and methods for maintaining data integrity during a migration | |
US10922135B2 (en) | Dynamic multitasking for distributed storage systems by detecting events for triggering a context switch | |
JP5959733B2 (en) | Storage system and storage system failure management method | |
US9069640B2 (en) | Patch applying method for virtual machine, storage system adopting patch applying method, and computer system | |
US11789840B2 (en) | Managing containers on a data storage system | |
US10809997B2 (en) | Information processing apparatus and program update control method | |
US10027532B2 (en) | Storage control apparatus and storage control method | |
JP2008112399A (en) | Storage virtualization switch and computer system | |
US9501372B2 (en) | Cluster system including closing a bus using an uncorrectable fault upon a fault detection in an active server | |
US20100138625A1 (en) | Recording medium storing update processing program for storage system, update processing method, and storage system | |
TW201437811A (en) | Methods and systems for recovering a computer system using a storage area network | |
US20240078198A1 (en) | Instant recovery as an enabler for uninhibited mobility between primary storage and secondary storage | |
US11113003B2 (en) | Storage apparatus, storage control device, and recording medium with execution command pausing or stopping | |
JP2010524108A (en) | Method and program for restoring automatic firmware of peer programmable hardware device | |
US20130167206A1 (en) | Storage system, method of controlling access to storage system and computer system | |
JP6674101B2 (en) | Control device and information processing system | |
EP3871087B1 (en) | Managing power request during cluster operations | |
US11301156B2 (en) | Virtual disk container and NVMe storage management system and method | |
JP6822706B1 (en) | Cluster system, server equipment, takeover method, and program | |
US11977458B2 (en) | System and method for storage awareness service failover | |
US20160259560A1 (en) | Information processing apparatus, information processing system, and control program | |
US20210191637A1 (en) | Storage control device and non-transitory computer-readable storage medium for storing control program | |
JP2018032061A (en) | Storage controller, and storage system | |
US10235316B1 (en) | Fabric management system and method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: FUJITSU LIMITED, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MUROYAMA, TOMOHIKO;REEL/FRAME:051398/0173 Effective date: 20191218 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |