Cluster

From Cmcl

Jump to: navigation, search

Contents

Related

IP addressing

  • Control network: 10.96.0.0/16
    • "Rack" machines should be assigned addresses in the range 10.96.64.10 -- 10.96.64.255
    • "Emulator" machines should be assigned addresses in the range 10.96.128.10 - 10.96.128.255

Network Infrastructure

The control network switch (one in CMCL lab, one in Wireless Lab) for the testbed is a cisco WS-C2960G-48TC-L (aka, a 2960g). 48 port gigabit ethernet switch.

Switch in CMCL lab

  • IP address : 10.96.0.10/255.255.0.0
  • Console: port S20 on the portmaster, telnet 10.96.0.21 6020

Temporarily moved to com1 on oink.cmcl.cs.cmu.edu 

oink$ sudo cu -l /dev/cuad0
  • Ports in use
    • port 1: To the portmaster terminal server
    • port 2: To power controller #1 (eventually)
    • port 3: To power controller #2 (eventually)
    • port 4: to boss.cmcl.cs.cmu.edu
    • port 5: to fiber
    • ...
    • port11: pc001
    • port 12: pc002
    • ...
    • port 18: pc008

Switch in Wireless lab

  • IP address : 10.96.0.11/255.255.0.0
  • Ports in use
    • port 1 to power03
    • port 2: to the portmaster terminal server 2
    • port 3: to power04
    • port 4: to pc009
    • port 5: to power05
    • port 6: to new portmaster terminal server (30 ports)
    • port 7: to power06
    • port 8: to emucontrol-1
    • port 12: to fiber
    • port 21 ... 25: to pcemu001 ... pcemu005
    • port 31: to emu-1 (rename to pcemu101?)
    • port 32 ... 40: to emu-2 ... emu-10

Machines

Please name cluster machines 'pcXXX' (e.g., pc001). For now, they should be physically located in the "Srini" area of the racks in the CMCL, expanding upwards into the "Dave" area.

  • boss.cmcl.cs.cmu.edu
    • Real IP: 128.2.222.223
    • Internal IP: 10.96.0.2 netmask 255.255.0.0
  • ops.cmcl.cs.cmu.edu
    • Real IP: 128.2.223.224
    • Internal IP: 10.96.0.3 netmask 255.255.0.0

Racked machines. Note: only pc008 has been moved to the Cemulab net:

hostname asset # KVM port control MAC experiment MAC
pc001 137424 00:13:72:FC:D3:8F 00:13:72:FC:D3:90
pc002 137425 00:13:72:FC:D3:C5 00:13:72:FC:D3:C6
pc003 137422 00:13:72:FC:D3:CB 00:13:72:FC:D3:CC
pc004 137420 00:13:72:FC:D1:70 00:13:72:FC:D1:71
pc005 137418 00:13:72:FC:D0:38 00:13:72:FC:D0:39
pc006 137421 00:13:72:FC:D4:5E 00:13:72:FC:D4:5F
pc007 137423 00:13:72:FC:D4:B2 00:13:72:FC:D4:B3
pc008 137419 1 00:13:72:FC:D4:82 00:13:72:FC:D4:83

Reserved IP addresses on the control network

  • 10.253.253.1 moo-control
  • 10.253.253.2 oink-control
  • 10.253.253.91 agami

Serial Console Server

Serial console services are provided via a Livingston Portmaster 2E. The portmaster should be configured to allow a 'telnet' connection to a particular port to be relayed directly to a serial port. The PM will then be connected via crossover to the boss node.

Serial console 1 (in CMCL) is, for the moment, connected to the Control network as 10.96.0.21/16 Serial console 2 (in Wireless lab) is on 10.96.0.23/16 Serial console 3 (the new one in Wireless lab, with 30 ports) is on 10.96.0.22/16

Eventually, this should be moved to a private connection to boss or ops and firewalled, since we have no real way of doing access control with it.

Ports: Serial console 1

  • S0: Reserved; console
  • S1...S8: pc001 ... pc008
  • S20: The control net switch

Serial console 2 (They will be moved to console 3 eventually??)

  • S2: pcemu001
  • S3: pcemu002
  • S4: pcemu003
  • S5: pcemu004
  • S6: pcemu005
  • S7: pc009

Serial console 3

  • S1 ... S10: to emu-1 ... emu-10

Power control

The six power controllers, power01 - power06, are accessible on the Cemulab control network (see below). Their IPs are 10.96.1.1 - 10.96.1.6. You can access them via HTTP using SSH port forwarding:

ssh -L8080:10.96.1.1:80 boss.cmcl.cs.cmu.edu (or 10.96.1.2)

and then direct your web browser to localhost:8080. The user name is 'root'. Once the emulabification is complete, you'll be able to use the "power" command to power cycle individual machines from the command line.

power04 is dead, and will recover soon.

hostname control MAC controller port
pc001 00:13:72:FC:D3:8F power01 5
pc002 00:13:72:FC:D3:C5 power01 6
pc003 00:13:72:FC:D3:CB power01 7
pc004 00:13:72:FC:D1:70 power01 8
pc005 00:13:72:FC:D0:38 power02 5
pc006 00:13:72:FC:D4:5E power02 6
pc007 00:13:72:FC:D4:B2 power02 7
pc008 00:13:72:FC:D4:82 power02 8
pcemu001 00:15:C5:43:ED:74 power03 1
pcemu002 00:15:C5:43:EE:52 power03 2
pcemu003 00:15:C5:43:8C:2D power03 3
pcemu004 00:1C:23:3A:76:1A power03 4
pcemu005 00:1C:23:3A:76:73 power03 5
pc009 00:13:72:57:10:D1 power03 6
pcemu101 00:11:43:43:d0:11 power05 1
pcemu102 00:15:C5:10:A0:93 power05 2
pcemu103 00:15:C5:10:A1:56 power05 3
pcemu104 00:15:C5:10:90:1C power05 4
pcemu105 00:15:C5:10:A1:92 power05 5
pcemu106 00:15:C5:10:A0:10 power05 6
pcemu107 00:11:43:53:A3:2F power05 7
pcemu108 00:11:43:51:A7:DB power05 8
pcemu109 00:14:22:B6:8F:49 power06 1
pcemu110 00:11:43:54:06:B2 power06 2

Remote KVM

The KVM is accessible as 10.253.253.31 port 443. To use it, the easiest thing is to:

ssh -L443:10.253.253.31:443 moo.cmcl.cs.cmu.edu

or some other machine that has an IP address on the CMCL control network (10.253.253.0/24). It's just on the main CMCL switch for now, so you can configure any of your own servers to have an IP there. Ask me about what IP to use in advance, please. :)

and then go to https://localhost:443/

(it runs as a java applet. It's a little slow, but it works.)

Username (administrator) and password for the kvm available in person. Ditto if you want to use the power controllers. Someone can come by and make a little lab password cheat sheet. :)

  • Ports in use
    • port 1: To pc008
    • port 28: (to some other power controller)

Storage and Computational Resources

  • The Agami box is a standalone NFS disk server. It has 3TB of usable space and four gigabit ethernet ports that client machines or switches can connect to. It has lots of disks in it and is very fast.
  • The Netezza box is a standalone SQL data mining engine. It's currently in use by the datapository, but if you have something you want to do VERY VERY FAST SQL queries on, please ask!

Useful Stuff and Equipment Links

HomeNet / CMULab

Todo List

  • Verify that all of pc01-pc08 can be power cycled; ensure that database entries are correct for their power ports
  • Get serial console access working for the power controllers (low priority)
  • In progress: Moving power controllers and terminal server to their own private VLAN with Boss
  • Get serial consoles working in wireless lab (RocketPort requires a standby PC as power?!)
  • Give boss/ops access to the wireless power controller
  • Decide what to do with the other two D620s in the development wireless emulator - OK to put them under Emulab control?
  • Setup hardware control network using vlan device on fin. Requires some reorg of the IP assignment to make a distinct space for the HW control net
  • Get power cycle + WoL working for laptops. Requires mods to Emulab power command; should be minor

Completed Todos

  • Connect pc01-pc08 to the cemulab network, disconnect from CS
  • Serial consoles for pc01-pc08
  • Console redirection enabled in BIOS for pc01-pc08
  • Move D620 laptop back to wireless lab (now renamed as emu-11)
  • Connect D620 to power controller in wireless (emu-11, connected to port 1 @ power03)

Notes on emucontrol-1 replacement

/emucontrol-1_replacement

Retrieved from "http://moo.cmcl.cs.cmu.edu/wiki/index.php/Cluster"
Personal tools