JP4945774B2 - Failure information data collection method for disk array device and transport control processor core - Google Patents

Description

  The present invention relates to a technique for collecting memory dump data when a failure of a processor mounted on a disk array device occurs. Particularly in a multi-core processor, one processor core is a processor core for transport control. When a failure occurs in the transport control processor core, the disk array device and the transport control processor core can collect memory dump data including failure information of the transport control processor core. The present invention relates to a method for collecting failure information data.

  In recent years, with the development of information infrastructure, in an information society where the amount of data handled has been increasing every day, it is required to realize an information system with high reliability and high availability. In order to realize such an information system, disk array devices capable of always accessing large amounts of data and backing up data are rapidly spreading.

  With the rapid spread, disk array devices with significantly improved performance are equipped with a large number of device components, and these components are complicatedly related. For this reason, when any problem occurs, it takes a lot of resources, time, and labor to identify the cause and recognize the range of influence. Therefore, it is necessary to collect useful fault information data (such as memory dump data of the CPU memory) regarding the cause of the problem within a limited resource and time.

  FIG. 7 is a diagram for explaining collection of failure information data when a failure occurs. In the disk array device 50, a CM (Controller Module) 500 (a, b) is a component that manages the entire storage system, such as host I / O control and device maintenance control. The CPU 510 (a, b) is a processor that controls the CM 500 (a, b). The expanders 700 (a, b) are components that monitor and control a DE (Drive Enclosure) on which the disks 600 (a, b) are mounted. Of the illustrated disks 600 (a, b), the disk 600b is set in advance as a system disk.

  In the disk array device 50 of FIG. 7, only two components such as the CM 500 and the disk 600 are shown for simplicity of explanation, but in reality, various components are made redundant and complicated. Related.

  For example, when a failure occurs in the CPU 510b of the CM 500b, the CPU 510b transitions from the normal state to the failure information storage state. In the failure information storage state, the memory dump target data 525 on the memory 520 of the CPU 510b is automatically stored in the system disk (disk 600b) as useful failure information data 610 regarding the cause of the problem by the failure information storage function.

  If the failure factor is a firmware factor (software factor), after the failure information data 610 is stored, the CM 500b in which the failure has occurred is reset and automatically controlled. By this control, the CM 500b where the failure has occurred is restored and becomes in a normal state where it can operate.

  The failure information data 610 stored in the system disk (disk 600b) can be collected by a maintenance personal computer (maintenance PC 800) connected to the disk array device 50. For example, when a failure occurs in the CM 500b at the site where the disk array device 50 is installed, a CE (Customer Engineer) or SE (System Engineer) at the site connects the maintenance PC 800 to the disk array device 50 for maintenance. The failure information data 610 stored in the system disk (disk 600b) of the disk array device 50 is collected in the disk 801 of the maintenance PC 800 via the CGI screen. The collected failure information data 610 is transmitted to the developer for failure analysis.

  Note that, for example, Patent Document 1 and Patent Document 2 are documents that describe techniques related to collection of failure information data.

  In Patent Document 1, a memory dump routine, which is a module separated from the operation system, is prepared on a processing device in order to quickly analyze a failure at the time of a failure, and when a dump switch is pressed, There is described a technique for collecting a memory dump by restarting a processing apparatus while leaving data on and executing a dump routine.

In Patent Document 2, in order to reduce the stop time of the computer system when a failure occurs, dump data of a dumped processor to be dumped is temporarily saved on a storage device of the save processor, and dump data A technique is described in which the dumped processor is restarted without waiting for output, and dump data on the storage device of the save processor is output to an external storage device.
JP 2000-137630 A JP 2001-34508 A

  FIG. 8 is a diagram illustrating the problem of the present invention. In recent years, multi-core processors in which a plurality of processor cores are integrated in one package have become widespread. In a multi-core processor, each processor core functions independently without being influenced by other processor cores. In FIG. 8, it is assumed that the CPU 510b of the CM 500b is a dual-core processor having two processor cores (Application Core 511, Transport Core 512).

  In the CPU 510b, the application core 511 is a processor core on which application firmware (Application Firmware) for managing the entire storage system such as RAID control related to host I / O control, copy control function, and device maintenance control is mounted. The transport core 512 is a processor core on which SAS / SATA in the host interface and the disk interface, and transport firmware (Transport Firmware) that controls the FC (Fibre Channel) transport layer protocol are mounted.

  When a failure occurs in the application core 511, as in the case described with reference to FIG. 7, the application core 511 transits from the normal state to the failure information storage state, and the memory dump target data 525 is stored by the failure information storage function. The failure information data 610 is stored in the system disk (disk 600b). The data transfer at this time is controlled by the transport core 512.

  When a failure occurs in the transport core 512, the application core 511 transits from the normal state to the failure information storage state, and the failure information storage function stores the failure information in the memory 520 from the transport core 512 in which the failure has occurred. The memory dump target data 525 including the failure information of the transport core 512 is dumped and stored in the system disk (disk 600b) as the failure information data 610.

  However, in this case, since a failure has occurred in the transport core 512 that controls data transfer, the application core 511 having the failure information storage function cannot access the system disk (disk 600b), and the transport core There is a high possibility that memory dump target data 525 including 512 failure information cannot be transferred to the system disk (disk 600b).

  In this way, in a multi-core processor configuration, if that processor core is a processor core for transport control, if a failure occurs in that transport control processor core, the memory dump target data in the memory will fail. Problems that cannot be transferred to the system disk for storing information may occur.

  The technique described in Patent Document 1 is a technique for collecting a memory dump in a single processor single core configuration. In addition, the technique described in Patent Document 2 described above aims to reduce the stop time of a computer system. In a multiprocessor configuration, all processors are a system management processor, a dump data saving processor, a faulty processor, and a related processor. The purpose and device configuration are different.

  That is, the techniques described in Patent Document 1 and Patent Document 2 do not have the concept that a failure occurs in a specific transport control processor core in a multi-core processor configuration. The technique described in 2 cannot solve the above problem.

  The present invention solves the above problems, and in a disk array device having a multi-core processor, when that one processor core is a processor core for transport control, a failure occurs in the processor core for transport control. Provides a technology that can automatically store memory dump data including failure information of the processor core for transport control in the system disk as useful failure information data regarding the cause of the failure The purpose is to do.

  In order to solve the above problems, the present invention provides a multi-core processor configuration in which, when a failure occurs in the transport control processor core, the failure information of the transport control processor core is stored in a non-volatile target area on the memory. After saving to the target area and restarting, the data of the storage target area on the memory including the area where the fault information of the transport control processor core was saved is used as useful fault information data regarding the cause of the problem. It is characterized by being automatically stored in a system disk via a transport control processor core.

  Specifically, the present invention provides a multi-core processor in which one processor core is a transport control processor core and at least one processor core other than the transport control processor core is a processor core having a fault information data collection function. A disk array device comprising a multi-core processor memory and a system disk that stores memory dump data collected from the memory as failure information data, and the nonvolatile memory is not initialized when the multi-core processor is restarted The target area is information for managing the memory for each area, and includes at least information indicating whether or not the target area is a nonvolatile target area and information indicating whether or not the target area is a storage target area from which data is collected when a failure occurs. Management information and transport control process A processor core that stores information indicating whether or not there is a failure of the Sacore and has a failure information data collection function is an area that is a non-volatile target area and a storage target area when a failure occurs in the transport control processor core. In addition, the means for saving the failure information of the transport control processor core and the information indicating whether or not there is a failure of the transport control processor core are set as the failure of the transport control processor core. If the information indicating whether there is a failure in the transport control processor core at the time of restart indicates that the transport control processor core has failed, the memory management information indicates the storage target area. Collect the data recorded in the memory area set for the transport control process. Via Sakoa, characterized in that it comprises a means for storing the system disk.

  As a result, in a disk array device having a multi-core processor, when one processor core is a processor core for transport control, even if a failure occurs in the processor core for transport control, the transport control is performed. Memory dump data including failure information of the processor core for the system can be automatically stored on the system disk as useful failure information data relating to the cause of the problem.

  Further, according to the present invention, in the above disk array apparatus, the means for saving the fault information of the transport control processor core dynamically secures an area on the memory for saving the fault information of the transport control processor core. The reserved area is registered in the memory management information as a nonvolatile target area and a storage target area, and the fault information of the transport control processor core is saved in the reserved area.

  As a result, there is no need to previously set an area for saving fault information of the processor core for transport control in the memory, so that the memory area can be effectively used during normal operation.

  According to the present invention, in a disk array device having a multi-core processor, when one processor core is a processor core for transport control, even if a failure occurs in the processor core for transport control, the transport Memory dump data including failure information of the control processor core can be automatically stored in the system disk via the transport control processor core as useful failure information data relating to the cause of the problem.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration example of a disk array device according to an embodiment of the present invention. The disk array device 10 shown in FIG. 1 has a configuration that focuses on one of the CMs 100 in particular. In the disk array device 10, the CM 100 and the CM 100 'are components that manage the entire storage system, such as host I / O control and device maintenance control. The CPU 110 is a processor that controls the CM 100. The expander 300 is a component that monitors and controls a DE (not shown) on which a disk (not shown) is mounted. In the example of the disk array device 10 in FIG. 1, only a part of the components such as the CM 100 is described for the sake of simplicity, but in reality, various components are redundantly related in a complicated manner. It has a configuration.

  The system disk 200 is a disk that stores failure information data collected when a failure occurs in the disk array device 10. A dedicated disk may be prepared as the system disk 200, or a partial area of the disk in which data from the user host is stored may be set as the system disk 200 area in advance.

  In the CM 100, the CPU 110 is a dual core processor having two processor cores, an application core 111 and a transport core 112. The application core 111 is a processor core on which application firmware 130 for managing the entire storage system such as RAID control related to host I / O control, copy control function, and device maintenance control is mounted. The transport core 112 is a processor core on which the SAS / SATA in the host interface and the disk interface and the transport firmware 170 that controls the FC transport layer protocol are mounted.

  The application firmware 130 has a normal routine 140, a failure information storage routine 150, and a power-on routine 160. The normal routine 140 is a program that is executed during the normal operation of the CM 100. The failure information storage routine 150 is a program that is executed when a failure occurs in the CPU 110. The power-on routine 160 is a program that is executed when the CM 100 is started or restarted.

  The memory 120 of the CPU 110 stores a transport firmware failure determination flag 121 and a memory management table 122. The transport firmware failure determination flag 121 is a flag indicating whether or not the activation of the CM 100 is a restart due to a failure of the transport firmware. Here, “1” indicates a restart due to the occurrence of a failure in the transport firmware, and “0” indicates the other. The memory management table 122 is a table in which management information of the memory 120 is recorded.

  FIG. 2 is a diagram illustrating an example of a memory management table. The memory management table 122 is a table for managing the memory 120 for each area, and is constructed based on the memory descriptor when the CM 100 is activated. In the memory descriptor, the size of an area that needs to be allocated on the memory 120 is instructed.

  The memory management table 122 has information such as a table number, a pool name (Pool name), an allocate address (Allocate address), an allocate size (Allocate size), a storage flag, and a nonvolatile flag.

  The table number is an identification number assigned to each record in the memory management table 122. The pool name indicates the name of the memory area. The allocate address indicates the address of the memory area. The allocate size indicates the size of the memory area.

  The storage flag is information indicating whether or not the memory area is a storage target area for the system disk 200. Here, “1” indicates that the storage target area is in the system disk 200, and “0” indicates that the storage target area is not in the system disk 200. Data in the memory area specified as the storage target area in the system disk 200 is transferred to the system disk 200 as fault information data when a fault occurs.

  The nonvolatile flag is information indicating whether or not the memory area is a nonvolatile target area. Here, “1” indicates that it is a non-volatile target area, and “0” indicates that it is not a non-volatile target area. The memory area designated as the non-volatile target area is not initialized when the CM 100 is restarted due to a failure of the transport firmware, and data is retained. Conversely, a memory area that is not designated as a non-volatile target area is initialized even when the CM 100 is restarted due to a failure of the transport firmware.

  In the memory management table shown in FIG. 2, the memory area of the pool name “SYS-MEM-DESC” is the area of the memory management table 122. As shown in FIG. 2, since the nonvolatile flag of the pool name “SYS-MEM-DESC” is “1”, it is not initialized when the CM 100 is restarted due to a failure of the transport firmware. In other words, when the CM 100 is restarted due to the failure of the transport firmware, the memory management table 122 before the restart remains as it is without reconstructing the memory management table 122 from the memory descriptor. Although not particularly shown in FIG. 2, the area in which the transport firmware failure determination flag 121 is recorded is also designated as the non-volatile target area.

  FIG. 3 is a diagram illustrating a functional configuration example of each routine of the application firmware. The failure information storage routine 150 includes a failure information storage state notification processing unit 151, a transport firmware failure information save processing unit 152, a CM restart processing unit 153, and a failure information data storage processing unit 154.

  The failure information storage state notification processing unit 151 performs processing for notifying the other CM 100 ′ and the expander 300 that the application core 111 of the own CM 100 has transitioned from the normal state to the failure information storage state. The transport firmware failure information saving processing unit 152 performs processing for saving the failure information in the memory 120 when a failure occurs in the transport firmware 170. The CM restart processing unit 153 performs processing for restarting the CM 100. The failure information data storage processing unit 154 performs processing for storing the data in the storage target area of the memory 120 in the system disk 200 as failure information data.

  The power-on routine 160 includes a transport firmware failure determination processing unit 161. The transport firmware failure determination processing unit 161 determines whether or not the activation of the CM 100 is a restart due to the occurrence of a failure in the transport firmware 170.

  Here, an example of a series of operations when a failure occurs in the transport firmware according to the present embodiment will be described with reference to FIGS.

  The application core 111 that has detected the failure of the transport firmware 170 in the transport core 112 transitions from the normal state to the failure information storage state. That is, the application core 111 stops the processing of the normal routine 140 and starts the failure information storage routine 150. The failure information storage state notification processing unit 151 of the failure information storage routine 150 notifies the other CM 100 ′, the expander 300, and the like that the application core 111 of the own CM 100 has entered the failure information storage state.

  The reason that the application core 111 of the CM 100 is in the failure information storage state is notified to the other CM 100 ′, the expander 300, etc. The other CM 100 ′, the expander 300, etc. This is because it is determined that a failure due to hardware factors may have occurred, and the CM 100 that does not respond is disconnected in order to avoid the risk. If the CM 100 in which a software-caused failure occurs enters the failure information storage state, the CM 100 'and the expander 300 are not disconnected while the failure information data is being stored.

  The application core 111 of the CM 100 in which the failure has occurred collects the transport firmware failure information from the transport core 112 in which the failure has occurred, and saves it in the memory 120 under the control of the internal bus. That is, the transport firmware failure information save processing unit 152 of the failure information storage routine 150 sends a transport firmware failure information collection instruction including information specifying a save area on the memory 120 to the transport core 112.

  In the memory management table 122 shown in FIG. 2, the pool name “TFW-INFO” indicates a save area for transport firmware failure information. As shown in FIG. 2, since the nonvolatile flag of the pool name “TFW-INFO” is “1”, the save area for the transport firmware failure information is not initialized when the CM 100 is restarted due to a failure of the transport firmware. . Further, since the storage flag of the pool name “TFW-INFO” is “1”, the transport firmware failure information saved in the memory area is stored in the system disk 200 as failure information data.

  The area for saving the transport firmware failure information on the memory 120 may be set in advance or may be dynamically secured. When an area for saving the transport firmware failure information is set in advance, it may be indicated by a memory descriptor.

  When the area for saving the transport firmware fault information is dynamically secured, the transport firmware fault information save processing unit 152 of the fault information storage routine 150 refers to the memory management table 122 and performs fast boot or the like. The area on the memory 120 that is not a storage target area in the system disk 200 (storage flag is “0”) is secured and the transport firmware fault information is saved. At this time, a record of an area for saving the transport firmware failure information is generated in the memory management table 122, and both the storage flag and the non-volatile flag are set to “1”.

  The CM restart processing unit 153 of the failure information storage routine 150 sets the transport firmware failure determination flag 121 of the nonvolatile target area on the memory 120 to “1”, and resets the own CM 100 to another CM 100 ′ or the expander 300. Request. The other CM 100 ′ or the expander 300 that has received the reset request resets the faulty CM 100 that has requested the reset.

  In the faulty CM 100 that has received the reset, the application core 111 and the transport core 112 are restarted. At this time, the application core 111 performs fast boot activation. The fast boot activation may activate the CM 100 in a state where the data in the area on the memory 120 designated as the non-volatile target area (non-volatile flag is “1”) in the memory management table 122 remains without being initialized. it can.

  The transport firmware failure determination processing unit 161 of the power-on routine 160 confirms the transport firmware failure determination flag 121 at an early stage of activation when the useful failure information regarding the cause of the problem is not touched. If 121 is “1”, it is set to “0” and then the failure information storage routine 150 is called to indicate that the transport firmware has failed. If the transport firmware failure determination flag is “0”, the normal routine 140 is called after the normal power-on process.

  Since the transport core 112 is reset and in an operable state, the application core 111 can access the system disk 200. When the failure information data storage processing unit 154 of the failure information storage routine 150 confirms the restart of the CM 100 due to a transport firmware failure, the failure information data storage processing unit 154 refers to the memory management table 122 and holds it in the memory area whose storage flag is “1”. The stored data on the memory 120 is stored in the system disk 200 as failure information data. At this time, the data stored in the system disk 200 includes transport firmware failure information.

  If the failure occurs in the normal routine 140 of the application firmware, the failure information storage routine 150 does not save the transport firmware failure information or restart the CM 100 in the failure information storage routine 150. The data on the memory 120 held in the memory area whose storage flag is “1” in the memory management table 122 is stored in the system disk 200 as failure information data.

  Hereinafter, a flow of a series of processing when a transport farm failure occurs in the present embodiment will be described using the flowcharts of FIGS.

  FIG. 4 is a failure information storage processing flowchart (1) when a transport firmware failure occurs by the application core. The process shown in the flowchart of FIG. 4 is a preparatory process for storing failure information data in the system disk 200.

  When the application core 111 detects the occurrence of a failure in the transport firmware 170 (step S10), the application core 111 transits from the normal state until then to the failure information storage state (step S11). At this time, it notifies the other CM 100 ', the expander 300, etc. that it is in the failure information storage state (step S12).

  By referring to the memory management table 122, an area on the memory 120 that has no influence on other controls and whose storage flag is “0” is secured as a transport firmware failure information save area (step S13), and the secured area Are registered in the memory management table 122 with the storage flag “1” and the nonvolatile flag “1” (step S14). The transport firmware failure information is saved from the transport core 112 to the transport firmware failure information saving area (step S15).

  The transport firmware failure determination flag 121 is set to “1” (step S16), and the own CM 100 is restarted by fast boot (step S17).

  FIG. 5 is a failure information storage processing flowchart (2) when a transport firmware failure occurs by the application core. The process shown in the flowchart of FIG. 5 is a process in the CM 100 restart stage. In practice, various initialization processes are performed, but only the transport firmware failure determination process will be described here.

  When the fast boot activation is performed, the application core 111 checks the transport firmware failure determination flag 121 at a relatively early stage of the initialization process (step S20). If the transport firmware failure determination flag 121 is not “1” (step S21), the normal initialization process is performed, and the process proceeds to the normal routine 140. If the transport firmware failure determination flag 121 is “1” (step S21), the transport firmware failure determination flag 121 is set to “0” (step S22), other necessary initialization processing is performed, and failure information is stored. Move on to routine 150.

  FIG. 6 is a failure information storage processing flowchart (3) when a transport firmware failure occurs by the application core. The process shown in the flowchart of FIG. 6 is a process of storing failure information data including transport firmware failure information in the system disk 200.

  When the application core 111 moves to the operation of the failure information storage routine 150 after restarting the CM 100, the application core 111 checks the memory management table 122 (step S30), and stores the memory area in which the storage flag on the memory 120 is set to “1”. Data is stored in the system disk 200 (step S31).

  Although the embodiment of the present invention has been described above, the present invention is not limited to this. For example, in the present embodiment, the dual core processor configuration in which one processor core is a processor core for transport control has been described. However, three or more processors in which one processor core is a processor core for transport control. A multi-core processor configuration having a core may be used.

It is a figure which shows the structural example of the disk array apparatus by embodiment of this invention. It is a figure which shows the example of a memory management table. It is a figure which shows the function structural example of each routine of application firmware. It is a failure information storage processing flowchart (1) at the time of transport firmware failure occurrence by an application core. It is a failure information storage process flowchart (2) at the time of transport firmware failure by an application core. It is a failure information storage process flowchart (3) at the time of transport firmware failure by an application core. It is a figure for demonstrating collection of the failure information data at the time of failure occurrence. It is a figure explaining the subject of this invention.

Explanation of symbols

10 Disk array device 100, 100 'CM
110 CPU
111 Application Core 112 Transport Core 120 Memory 121 Transport Firmware Failure Determination Flag 122 Memory Management Table 130 Application Firmware 140 Normal Routine 150 Failure Information Storage Routine 151 Failure Information Storage Status Notification Processing Unit 152 Transport Firmware Failure Information Save Processing Unit 153 CM Restart processing unit 154 Fault information data storage processing unit 160 Power-on routine 161 Transport firmware fault determination processing unit 170 Transport firmware 200 System disk 300 Expander

Claims (4)

A multi-core processor in which one processor core is a processor core for transport control and at least one processor core other than the transport control processor core is a processor core having a fault information data collection function; a memory of the multi-core processor; and a memory A disk array device comprising a system disk for storing memory dump data collected from as fault information data,
The non-volatile target area of the memory whose data is not initialized when the multi-core processor is restarted is information for managing the memory for each area, and at least information indicating whether or not the non-volatile target area and data are collected when a failure occurs Memory management information having information indicating whether or not the storage target area is to be stored, and information indicating whether or not the transport control processor core is faulty,
The processor core having the failure information data collection function is:
Means for saving failure information of the transport control processor core in a non-volatile target area and a storage target area of the memory when a failure occurs in the transport control processor core;
Means for setting in the information indicating whether or not the transport control processor core is faulty, a fault of the transport control processor core, and restarting the multi-core processor;
When the information indicating whether or not the transport control processor core is faulty indicates that the transport control processor core is faulty at the time of restart, the memory management information sets the storage target area. Means for collecting data recorded in the memory area and storing the data in the system disk via the transport control processor core. The disk array device according to claim 1,
The means for saving failure information of the transport control processor core dynamically secures an area on the memory for saving the failure information of the transport control processor core, and the reserved area is the memory management information. A disk array device, wherein the failure information of the processor core for transport control is saved in a reserved area. A multi-core processor in which one processor core is a processor core for transport control and at least one processor core other than the transport control processor core is a processor core having a fault information data collection function; a memory of the multi-core processor; and a memory A system disk that stores memory dump data collected from the system as failure information data, and the data is not initialized when the multi-core processor is restarted. Memory management information having at least information indicating whether it is a non-volatile target area and information indicating whether it is a storage target area from which data is collected when a failure occurs, and whether there is a fault in the transport control processor core Display information A fault information data collection process of the transport controller processor cores in the array device,
A processor core having the failure information data collection function;
A process of saving failure information of the transport control processor core in a non-volatile target area and a storage target area of the memory when a failure occurs in the transport control processor core;
Setting the information indicating whether or not the transport control processor core is faulty as a fault of the transport control processor core and restarting the multi-core processor;
When the information indicating whether or not the transport control processor core is faulty indicates that the transport control processor core is faulty at the time of restart, the memory management information sets the storage target area. And collecting the data recorded in the memory area and storing the data in the system disk via the transport control processor core. Fault information of the transport control processor core Data collection method. In the failure information data collection method of the transport control processor core according to claim 3,
In the process of saving fault information of the transport control processor core, an area on the memory for saving the fault information of the transport control processor core is dynamically secured, and the reserved area is assigned to the memory management information A failure control data collection method for a transport control processor core, wherein the failure information of the transport control processor core is saved in a reserved area.

Images