GB2412772A

GB2412772A - Service processor providing system management functions

Info

Publication number: GB2412772A
Application number: GB0513331A
Authority: GB
Inventors: James Edward King; Rhod James Jones
Original assignee: Sun Microsystems Inc
Current assignee: Sun Microsystems Inc
Priority date: 2002-08-09
Filing date: 2003-08-11
Publication date: 2005-10-05
Anticipated expiration: 2023-08-11
Also published as: GB0513331D0; GB2412773A; GB0513332D0; GB2412773B; GB2412772B

Abstract

A computer system comprises a host processor, a service processor and a management communication device that communicates with the service processor. The service processor provides system management functions within the computer system and provides a signal indicative of normal operation to the management communication device. The management communication device sends and receives data only when it receives the signal from the service processor. The computer system may be a computer server in a network.

Description

241 2772

COMPUTER ASSEMBLY

BACKGROUND OF THE INVENTION

This invention relates to computer systems, and especially to computer systems that are employed as servers.

The systems may for instance be employed as servers for example in local area networks (LANs) or in wide area networks (WANs), telecommunications systems or other operations such as database management or as interned servers. Such servers may be used in so called "horizontally scaled" applications in which tens or hundreds of corresponding servers are employed as part of a distributed system.

A typical computer employed for such purposes will comprise a pair of processors mounted on a motherboard, together with power supply units (PSUs), and other components such as hard disc drives ( HDDs), fans, digital video disc (DVD) players, memory modules ethernet ports etc. One or more of the processors, the host processor(s), provides the main functions of the server, and may communicate with a number of peripheral components, including communication ports, optionally via peripheral component interconnect (PCI) bridges in order to provide server operation. One of those peripheral components, called the "South Bridge" further allows the host processors to communicate with internal devices via serial interfaces one of which transports the console interface of the processors.

In addition to the host processor(s), the system may include another processor, called the service processor or the remote management controller (RMC), which provides management functions for the system assembly. Such functions may include environmental monitoring, temperature monitoring of the enclosure, fan speed control, data logging and the like.

The service processor may communicate with the host processor or with one of them, and may also have one or more external communication ports so that a user or network administrator can communicate with the service processor, or can communicate with the host processor(s) via the service processor. For example, the service processor may have its own ethernet network port for direct communication to the network administrator.

Such ethernet network ports, whether communicating with the service processor or the host processor(s) will normally need a physical interface (PHY) in order to clean the signals and to provide power for driving the signals along the ethernet cabling, clock timing, line coding etc. The signals will then typically be sent to the ethernet cabling via a standard network port, for example an RJ45 port which will accept an eight line cable and is provided with a pair of light emitting diode (LED) indicators, one for indicating the existence of a link, and the other for indicating the existence of traffic on the line.

If there is any malfunction of the service processor whether due to hardware or software faults, the system is designed to continue to operate as indicated above, at least as far as the provision of services provided by the host processor are concerned, although clearly system management services will no longer be available until the service processor is replaced in the event of a hardware failure. Thus, the functioning of the server should be largely unaffected by any failure of the service processor.

However, even though the service processor has stopped functioning, power will still be sent to all the ethernet interfaces including the service processor ethernet interface. While this will not matter as far as the ethernet interfaces handled by the host processors are concerned because those interfaces will still be controlled by their associated media access controllers (MACs) and host processors, no such control is exerted on the management interface controlled by the service processor. Thus, internal lines in the system extending between the service processor and the management PHY may be susceptible to interference from any active components in the system, and in particular from the host processor(s). This interference will then be amplified and line coded by the management PHY before being sent to the ethernet lines. Visual inspection of the RJ45 management port will give the appearance that the server is functioning correctly because the LEDs will be on, indicating traffic on the line, even though this traffic is simply interference, and the server will appear to accept data from the service administrator because the RJ45 port is still operational.

SUMMARY OF THE INVENTION

According to one aspect, the present invention provides, a computer system which comprises: (i) a host processor; (ii) a service processor for providing system management functions within the computer system; and (iii) one or more external communication devices, the external communication devices including at least one management communication device that communicates with the service processor; wherein the management communication device is controlled by a signal from the service processor and is operative to send and receive data only when it receives the signal from the service processor.

The system according to the invention has the advantage that, in the event of a malfunction of the service processor, the system will still function for its intended purpose, other than to allow system management operations, but interference that is internally generated will not be put onto the external communication lines by the management communication device. For example, where the system operates as a network server, interference will not be put onto the network.

The signal may, for example, simply be a voltage level that is supplied by the service processor as described above. In such a case, a pull-up or pull-down resistor may be provided between the signal line and either a voltage rail or earth, so that, if no voltage is received from the service processor, the control voltage for the device will rise to the appropriate voltage rail or will fall to ground.

The management communication port may include a physical interface for providing line power, line coding and the like, in which case it may be provided with a reset input. The signal from the service processor may thus be sent to the reset input of the physical interface, after inverting if necessary depending on the reset input of the physical interface, in order to cause the physical interface to become inoperative if the service processor malfunctions.

The management communication port need not be the only external communication device in the computer system, and additional external communication devices, for example ethernet ports, may be provided that are controlled by the host processor(s). These devices will not be affected by a malfunction of the service processor, and will continue to operate as normal.

There will not normally be a danger of such devices placing interference on the network because they will continue to be controlled by their processors and by their associated hardware such as the media access controllers.

According to another aspect of the invention, a network may include a computer system according to the invention together with a network administrator.

The network could be a private network, or it could be a public network. Where the network is a public network, the public will only have access to the information on the lines other than the network management line supplied by the management communication device. Accordingly, the public will not be aware of any malfunction of the service processor of the network server. This will only be apparent to the network administrator, who will attend to resolving the problem.

Thus, according to yet another aspect of the invention, there is provided a method of operating such a network.

The method comprises monitoring traffic from the management communication device and effecting repair or replacement of the service processor or of the server in the event of loss of traffic from the management communication device.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will now be described in detail by way of example with reference to the accompanying drawings, in which corresponding parts are given like reference numbers. In the drawings: Figure 1 is a physical plan view of one form of computer system according to the present invention; Figure 2 is a schematic block diagram showing the system architecture of the system of figure 1; Figure 3 is a schematic diagram showing the service processor employed in the present invention together with some peripheral components; Figure 4 is a schematic diagram showing the service processor and the interconnection to certain peripheral components; Figure 5 is a schematic diagram showing connection of the computer system to a network; Figure 6 is a schematic diagram showing the service processor, the console interface, user interface and multiplexer without other peripheral components shown in figure 4; Figure 7 is a flow diagram of a system power-up; and Figure 8 is a graph of fan speed against temperature.

DESCRIPTION OF PARTICULAR EMBODIMENTS

Referring now to the drawings, in which like reference numerals are used to designate corresponding elements, figure 1 shows a physical plan view of a narrow form factor computer that is intended to provide a rack mounted server for use with the internet or as part of a local area network (LAN) or for other telecommunications purposes, and is designed to fit into, for example a nineteen inch rack electronics cabinet. Other sizes may alternatively be employed, for example to fit into 23 inch or metric racks. The assembly may be designed to be a so-called high "RAS" system, that is to say, to have high reliability, availability and serviceability. As such, it is intended that the system will be operated with the minimum amount of down time.

The computer comprises an enclosure 1 that contains a motherboard 2 in the form of a printed circuit board (PCB) designed in a custom form-factor to fit the enclosure 1 and chosen to minimise the cabling wishing the enclosure. The motherboard 2 carries the majority of circuitry within the computer. On the motherboard are mounted one or more (in this case two) host processors or central processing units (CPUs) each of which is provided with its own dedicated cooling in the form of an impingement fan that clips onto the CPU socket. Each processor 4, 6 is provided with its own dedicated block of memory 7, 8, for example provided in the form of one or two banks of dual in-line memory modules (DIMMs) with a total of 256MB to 16GB block capacity although other forms and sizes may be used.

A hardware cryptographic module (HCM) 10 may also be located on the motherboard. The HCM may be provided on a mezzanine card which plugs directly into the motherboard, and contains a co-processor providing cryptographic protocol acceleration support for security algorithms used in private community applications.

Two hard disc drives (HDDs) 12 and 14 are located at the front of the computer behind the front bezel 16.

The drives are hot-pluggable and are accessible by removal of the bezel and EMI shield 18. Two internal HDDs plug directly into the motherboard via right angled connectors located on the front edge of the motherboard 2.

Next to the HDDs is arranged a system configuration card (SCCR) reader 20 that is able to read a system configuration card (SCC) 22 inserted therein. The SCC contains all relevant information concerning the computer, so that it is possible to replace one computer with another simply by inserting the original SCC into the new computer and replacing the hard disc drives with those of the original computer.

A removable media drive bay is provided to allow optional fitting of a slimline (notebook style) digital video disc or digital versatile disc (DVD) drive 24 for reading CD and DVD media. The media transport loader is accessible through a slot in the enclosure bezel 16.

One or two 320W or 400W custom power supply units (PSUs) 26 are also provided. In addition to the dedicated CPU fans, the assembly is cooled by means of a row of fans 28 mounted between the motherboard and the media drive bays.

The computer supports input/output (I/O) expansion by means of peripheral component interconnect (PCI) cards that plug into expansion slots. These are accommodated by means of riser cards 29 that plug directly into the motherboard 2.

A number of I/O interfaces and sockets 30 are provided along the rear surface of the enclosure 1 including four ethernet ports 30, a network management ethernet port 70, and a serial port 72. The network management ethernet port 70 and the serial port 72 allow user access to the service processor and system console.

Figure 2 is a schematic representation of the system architecture of the computer system according to the invention.

Two host processors or CPUs 4 and 6 available from Sun Microsystems under the name UltraSPARCIIIi have an integer execution unit, a floating point and graphics unit, 32kB level 1 instruction cache, 64kB level 1 data cache, 1MB (256k x 32) level 2 data cache, a memory controller with error correction code (ECC) and an interface controller for the processor bus. Four DIMS sockets 7 and 8 are associated with each CPU.

The CPUs 4, 6 are connected to two PCI bridge 40, 42 which provide interfaces to independent 64 bit PCI buses leading to various peripheral components such as the riser cards 28, HDDs 12 and 14, the HCM 10 etc. The PCI bridge 40 is also connected to a PCI I/O device 44 available from Acer Labs under the code Ml535D+ also referred to as "South Bridge". This is an integrated PCI sub system which provides an integrated drive electronics (IDE) controller, a universal serial bus (USB) controller, independent universal asynchronous receiver/transmitters (UARTs), XBUS bridge and a power management controller. The PCI I/O device 44 also provides the console interface for enabling user access to the host processors 4 and 6.

A service processor or remote management controller (RMC) 50 is included for providing local and remote management services. Such services may include one or more of the following system functions: 1) power management control, 2) environmental monitoring, 3) enclosure management and event logging 4) fan control, 5) voltage rail monitoring, and 6) system status monitoring.

Other service functions may be included if desired.

The service processor is also responsible for monitoring and reporting the operational status of the system. The processor operates from the +5V rail and is capable of power cycling and resetting of the host system. It is based on an MPC850 PowerPC design with dedicated flash ROM 62 and synchronous dynamic RAM (SDRAM) 64.

Peripheral devices that are required for the management functions, include the system configuration card reader (SCCR) 20, PCI clock generator 52, general purpose IO (GPIO) devices 54 field replaceable unit identification (FRUID) devices 56, a "time-of-day" real time clock 57, and a system temperature monitor 58 provided as an Analogue Devices ADM1026 IC. These devices are provided on an inter-integrated circuit (I2C) management bus 60. As shown in figure 3, in addition to the flash ROM and SDRAM, the service controller can access electrically erasable programmable ROM (EEPROM) 66 that is provided in the temperature monitor 58 via the I2C management bus 60.

As well as monitoring the environment and managing the peripheral devices, the service processor can communicate with the PCI I/O device or console interface 44 by means of line 68. User access to the service processor 50 is available either through the 10BASE-T ethernet port 70, NET_MGT, or through the asynchronous serial port 72 (SERIAL_MGT). In this way, remote user access is available either to the service processor 50 for management functions, or to the host processor(s) 4 and 6 via the service processor 50. Remote user access, for example by the network administrator, will normally be obtained via the ethernet port 70, while local user access will normally be obtained via the serial port 72.

Figure 4 shows the service processor 50 connected to various peripheral components, and Figure 6 shows the communication between the service processor 50 and the console interface 44 and serial port 72. As shown in figure 6, communication between the serial port 72 and the service processor 50 occurs via a multiplexer or other switching device 86. The multiplexer 86 is controlled by a signal CNSL_SW from the service processor 50 along a control line 82 so that, when the voltage on the control line is low (ground) all signals to and from the serial port 72 along line 78 are routed through the service processor 50 along line 79. On receipt of the signal from the serial port 72, the service processor 50 determines whether the signal is a management mode command, in which case it is acted upon by the service processor, or whether is a console mode command, in which case the service processor routes the signal to the console interface 44 via line 95, the switching device or multiplexer 86 and line 96.

If, however, any malfunction occurs in the service processor 50, accessing the console interface 44 will not be possible. In this case, control line 82 is arranged so that its voltage will rise to the appropriate rail voltage (approximately 5V) and disconnect the lines 78 and 96 from lines 79 and 95 respectively. At the same time multiplexer 86 connects line 78 directly to line 96 so that signals are transmitted directly between the serial port 72 and the console interface 44, thereby enabling the user to access the console interface 44 on failure of the service processor.

One simple way to execute such a switch is to provide a pull-up resistor 84 between the voltage rail and the line 82, so that, if the service processor is not operational to bring the line to ground, the pull-up resistor will cause its voltage to rise to the 18! positive voltage rail value. The lines 78 and 96 may be connected by a switch, that will normally be open, but will close when the voltage on the control line 82 rises. The lines 78 and 79, and the lines 95 and 96 may also be connected by switches that open as the control line voltage rises. Such an arrangement may be realized in a number of ways, for example by means of a CMOS analogue or digital multiplexer 86 in which the control line is applied to the gates of the FET switches in the multiplexer (via inverters where necessary).

As an alternative to the pull-up resistor 84, a pull down resistor 84a as shown in Figure 4 may be employed that is connected between the control line 82 and earth. In this case the service processor 50 would hold the control line 82 at same voltage level unless it failed in which case the control line voltage would fall to earth potential.

Figure 7 is a flow diagram showing the power-up procedure of a computer system according to the invention, which may form part of the power on self test (POST) procedure. When the server cable is first plugged in to the system, the multiplexer 86 will be in an undefined condition. On plugging the server cable standby power will be applied, (step 120) whereupon the control line 82 voltage will fall to earth, and the multiplexer 82 will move to its default condition in which lines 78 and 96 are connected and the service processor is bypassed (step 122). The service processor (50) then attempts to boot up (step 124) and an interrogation (step 126) occurs as to whether the boot-up has been successful.

If this attempt fails, due to a malfunction of the service processor, the system will operate without the service processor, (and without any of its management functions), but the service processor will be bypassed, and access to the console interface will be available. If the service processor booting operation is successful, the console multiplexer 74 will be switched on (step 128), and console commands and data will be routed via the service processor, and will continue to be so until the system is powered down or the service processor fails.

Figure 4 shows the connection between various components of the system including the service processor 50 and the temperature monitor 58. The temperature monitor IC does, in fact, have its own junction internal temperature monitor, but this is not used for the purposes of temperature sensing in the system according to the invention because sensing temperature within the enclosure is extremely sensitive to the positioning of other components within the enclosure and to changes of the components.

For this reason, a separate silicon band gap temperature monitor 100 available from National Semiconductors under the product code LM75 is employed. This temperature monitor is located in the front bezel 16 so that it measures the temperature of the external air that is introduced into the enclosure rather than that of air within the enclosure. The temperature value is encoded by the monitor 100 into an eight bit word and is sent to the service processor along the I2C management bus 60 when requested by the processor.

The processor then calculates the desired fan speed in accordance with a speed table that has been input into the service processor memory, for example into the EEPROM 66 of the temperature monitor 58. Figure 8 shows an example of such a speed requirement. The fan speed required is relatively low, and constant with respect to temperature, until a first temperature T1 at which point the required fan speed increases linearly with temperature until temperature T2 is reached, when, again, a speed that is constant with respect to temperature is required. The upper constant speed above T2 may well be because the fans 28 are already running at maximum speed. The precise values of temperatures T1 and T2 will vary depending on the enclosure design, the fans and the other components.

The service processor then sends the required fan speed value to the temperature monitor 58 via the I2C management bus 60, whereupon it is converted to a pulse width modulated (PWM) signal and sent to the control input of the fan unit 28 along line 102.

The temperature monitor 58 also provides counter inputs which are used to monitor the rotational speed of all the fans within the enclosure, not only the enclosure fans 28, but also the dedicated fans for the CPUs. The fans provide tachometer output signals for this purpose, along lines 104. The signals are open drain and two pulse-per-revolution logic format. The service processor compares the measured speed values against minimum thresholds and issues alerts when required.

As described, the service processor runs the enclosure fans 28 under openloop control in accordance with the fan speed requirement given in Figure 8. The service processor could, if desired, run the fans under closedloop control, for example by reading the tachometer data supplied to the temperature monitor 58 along lines 104 and taking the difference between the tachometer readings and a demanded tachometer level.

A control line 82 (CNSL_SW) extends from the service processor 50 to the thermal reset input (THERM) 106 of the temperature monitor 58. The control line 82 is also connected to a pull-up resistor 84 connected to the positive 5V rail 85. The control line will be held down to earth voltage by the service processor 50 once the service processor is booted up, but, should the service processor fail for any reason, whether a hardware or a software failure, the control line 82 voltage will rise to the 5V rail due to its connection via the pull-up resistor. This voltage is then fed into the thermal reset input of the temperature monitor 58 which then sets the signal on the fan speed line 102 to maximum (i.e. a pulse width of 100).

As shown in figure 4, signals from the service processor to the ethernet port 70 are controlled within the service processor by the media access controller 80 which handles the open systems interconnection (OSI) level 2 (data link layer) protocols, and are sent to the ethernet port 70 via a physical lines 81 and a physical interface or PHY 71 which provides power for sending the signals along the ethernet cabling, and provides other functions such as a clock, and line coding. Manchester encoding is employed in this case, but other forms of line coding may be used, that are appropriate to the channel characteristics. A control line 82 (CNSL_SW) extends from the service processor to the reset input 83 of the PHY 71 for the management ethernet port. The control line is also connected to a pull-up resistor 84 connected to the positive 5V rail.

Management data is transmitted between the PHY 71 and the ethernet port 70 by means of an eight conductor cable. In addition, LEDS on the RJ45 socket forming the ethernet port will light up, one LED indicating that the port is operative, and the other LED turning on whenever there is traffic on the line. Other forms of cable may be employed, depending on the form of the ethernet port, and indeed other forms of port may be used.

The control line 82 is also connected to a multiplexer 86 which controls signals between the asynchronous serial port 72 and the host processor I/O device 44.

Figure 5 shows part of a network in which a server 1 communicates with a number of switches 90 by means of ethernet cabling 92 connected to the data ethernet ports 30, the switches 90 then being connected to the internet/intranet 93. In addition, the server 1 is connected to a management switch 91 and thence to the network administrator by means of the management ethernet port 70 connected to the service processor 50.

Under normal operation, data will be transmitted to and from the switches 90 by means of the data ethernet ports 30. At the same time, management data will be transmitted between the server 1 and the network administrator 94 via the management ethernet port 70 and management switch 91. In this mode, the service processor 50 holds the voltage of the control line 82 r to earth potential against the pull-up resistor 84.

However, if for any reason the service processor 50 should malfunction, whether this is caused by a hardware fault or a software problem, the control signal from the service processor will be lost and the voltage on the control line 82 will rise to the 5V rail voltage due to the pull-up resistor 84. This voltage will be led into the reset input 84 of the management ethernet port PHY 71 and switch the PHY off. Turning the PHY 71 off will cause the network administrator 94 to become aware of the malfunction since it will not be possible to send or receive management data to or from the server 1. In addition, any user who inspects the server will be able to see that the LEDs 88 and 89 are turned off. In addition, if the service processor fails, interference from the host processors 4 and 6, which are still operating, will not be picked up by the lines 81, amplified and coded by the PHY 71 and sent along the network ethernet lines 92, thereby causing interference in other parts of the network.

At the same time as the PHY 71 is turned off, the change in voltage on the control line 82 will cause the multiplexer 86 to stop sending data from the serial port 72 to the service processor 50, and instead bypass the service processor so that the data are sent directly between the UART 72 and the host processor PC I/O device 44. A user may still be able to access the host processors via the serial port 72 since the service processor will be bypassed.

As shown in Figure 4, the service processor 50 also controls the ethernet port 70 which is connected to the system administrator 94. In fact, as shown in Figure 5, the server is connected to the network administrator via a further server 91 or switch. The computer is also connected to servers 90 as part of a network 93. If the service processor 50 malfunctions, it is possible for interference on lines 31 leading to the ethernet port 70 generated by, for example, the host processor(s) 4, 6, to be sent to the network. In order to prevent this the control line 82 is connected to the reset input 83 of the physical interface 71 for the ethernet port 70 so that, on failure of the service processor 50, theethernet port 70 is quiesced and the network administrator 94 becomes aware of the fault.

In addition, the control line 82 is connected to the thermal input of the temperature monitor 58. Under normal operation the temperature monitor will send pulse width modulated fan speed signals to the fans 28 along line 102 under command of the service processor 50. When the service processor fails and the voltage on the control line 82 rises, this voltage is also fed into the thermal reset input 106 of the temperature monitor and the enclosure fans 28 are then driven at full speed.

In this way, should a malfunction of the service processor occur, the system ensures that no noise is transferred to the network, that the system is adequately cooled and that communication to the host processor(s) is still possible.

Claims

Claims A computer system which comprises: (i) a host processor; (ii) a

service processor for providing system management functions within the computer system; and (iii) one or more external communication devices, the external communication devices including at least one management communication device that communicates with the service processor; wherein the management communication device is controlled by a signal from the service processor and is operative to send and receive data only when it receives the signal from the service processor.
2. A computer system as claimed in claim 1, wherein the management communication device is controlled by a voltage level that is supplied by the service processor.
3. A system as claimed in claim 2, wherein the management control device is controlled by a voltage level that is governed by a voltage supplied by the service processor and a pull-up or pull-down resistor.
4. A system as claimed in any one of claims 1 to 3, wherein the management communication port is an ethernet port.
5. A system as claimed in any one of claims 1 to 4, wherein the management communication device includes a physical interface that provides power for signals sent from the management communication device.
6. A system as claimed in any one of claims 1 to 5, wherein the management communication device includes an indicator for indicating whether or not a communication link is established and/or whether or not traffic is being sent or received which indicator is quiesced when the management communication device is inoperative.
7. A system as claimed in claim S. wherein the signal from the service processor is sent to a reset input of the physical interface to cause the physical interface to become inoperative if the service 3 0 54 3 6711. 03 processor malfunctions.
8. A system as claimed in any one of claims 1 to 7, which includes additional external communication ports that are controlled by the or each host processor.
9. A system as claimed in claim 8, wherein the additional external communication ports are ethernet ports.
10. A system as claimed in any one of claims 1 to 9, wherein the service processor provides one or more of the following system functions: 1) power management control, 2) environmental monitoring, 3) enclosure management and event logging 4) fan control, 5) voltage rail monitoring, and 6) system status monitoring.
11. A system as claimed in any one of claims 1 to 10, which is a computer server.
12. A network, which comprises a computer system as 31 5436711.03 claimed in any one of claims 1 to 11 a network administrator which communicates with the service processor by means of the management communication device.
13. A method of operating a network which includes at least one computer server comprising: (i) a host processor; (ii) a service processor for providing system management functions within the computer system; and (iii) one or more external communication devices, the external communication devices including at least one management communication device that communicates with the service processor and with a network administrator; the management communication device being controlled by a signal from the service processor and being operative to send and receive data only when it receives the signal from the service processor, which method comprises monitoring traffic from the management communication device and effecting repair or replacement of the service processor or of the server in the event of loss of traffic from the management communication device.
14. A computer system substantially as hereinbefore described with reference to, and as shown in, the accompanying drawings.