Differences between revisions 10 and 24 (spanning 14 versions)
⇤ ← Revision 10 as of 2012-03-24 14:22:48 →
Size: 3367
Editor: PieterSmit
Comment:
← Revision 24 as of 2012-12-04 13:03:20 → ⇥
Size: 4497
Editor: PieterSmit
Comment: add qos example
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
 * [[Nexus/Logging]]
 * [[Nexus/ConfigSync]]

== Jumbo frames ==
 * 201210 - Bug in Nexus design reports normal frames over trunk link as jumbo due to vlan header.
 * HowTo enable jumbo frames.
   * Done through policy. 3 Layers, qos -> queuing -> network-qos
     {{{
system qos
  service-policy type queuing output xxxxx
  service-policy type queuing input xxxxx
  service-policy type network-qos xxxxx
  service-policy type qos input xxxxx
      }}}



== Nexus 4000 ==
 * MTBF [[http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps10596/ps10642/data_sheet_c78-560881_ps10596_Products_Data_Sheet.html|187,265 hours]] = 21years.
   *
== vPC ==
 * [[http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/|Rules and diagrams]]
 * Check witch link in a port channel will be used.
 * show port-channel load-balance forwarding-path interface port-channel 2 vlan 902 dst-ip 192.168.110.212 src-ip 192.168.110.200
# sh hardware internal carmel port e 1/1
 * QOS http://alleasysetup.blogspot.com/2011/10/qos-on-cisco-nexus-5k.html
 * FlowControll to VMware http://www.boche.net/blog/index.php/2010/11/29/flow-control/
 * vPC virtual port channel, let port's on 2 vPC peers look like they are from one switch, eliminating blocked ports on ethertrunk.
 * Step1 enable features needed on switches.
   {{{
   feature lacp
   feature vpc
   }}}
 * peer-gateway - only for dumb hosts that do not use arp for gw discovery, but incoming packet.
 * pinning max-links 1
 * Before the peer link will come up, you also have to set up VPC keepalives to detect dual active peers
  {{{
  vpc domain 1
    peer-keepalive destination 10.10.10.14 source 10.10.10.13
  }}}
 * VPC Rule 101
   * VPC peers are expected to forward a frame received on a member link out any other member link that needs to be used. Only if they cannot do so due to a link failure, is forwarding across the VPC peer link and then out a member link allowed, and even then, the cross-peer-link traffic can only go out the member link that is paired with the member link that is down.

 * Restatement of VPC Rule 101
   * To put it another way, VPC peers are expected to forward a frame received on a member link out any other member link that needs to be used. Only if they cannot do so due to a link failure, is forwarding across the VPC peer link and then out a member link allowed, and even then, the cross-peer-link traffic can only go out the member link that is paired with the member link that is down.

 * With VPC, one should always put keepalives on a different link than the VPC peer link, since their purpose is to detect a situation with the peer still up but VPC peer link down. You just can't do that if they're running over the peer link. Instead, you can use the management port. Or, if you put a separate point-to-point routed link between the peers, in parallel with the VPC peer-link, you can use that for the keepalives.

  * The L3 decision determines the outbound interface. If that outbound link is a VPC member link, the same forwarding rule is applied as for L2: the frame (packet) came in the peer-link, the member interface that is VPC-paired to the chosen outbound link is up, therefore drop the packet.
 
 * There are some definite requirements for using an EtherChannel for FEXs, such as the FEX parameter pinning max-links must be set to 1.

 * Check fiber signal strength.
   {{{
   sh interface transceiver details
   }}}
Line 17: Line 73:
== List price estimates ==
 * 2012
   * '''2x CoreB Total: R831,708.00''' = 2x R415,854.00
     * R355,774.00 N3K-C3064-E-FA-L3 : Nexus 3064-E, Std Airflow (port side exhaust), LAN Ent Lic B
       * R60,080.00 8x 1GSX-MM, 4x 10Gbase-Cu, 4x 10Gbase-Sr
   * '''ADD''' 2x R147,000.00 N3K-C3048-FA-L3 + 2x10Guplink + 4 x SFP-10G-SR : 10Gbase-Sr Sfp Module
     * '''R294,000.00'''
----
   * ADD 2x N2K-C2248TP-1GE (48x100/1000-T+4x10GE) + 2x10Guplink + 4 x SFP-10G-SR : 10Gbase-Sr Sfp Module
     * '''R166,342.00'''
       * R49,970.73 N2K-C2248TP : N2K-C2248TP-1GE (48x100/1000-T+4x10GE)
       * R8,300.00 SFP-10G-SR : 10Gbase-Sr Sfp Module
   * '''2x CoreA Total: R460,160.00''' = 2x 230,080.00
     * 1x R170,000.00 2 x N5K-C5548P-FA +L3: Cisco Nexus 5548P 1Ru Chassis 2 Ps 2 Fan 32 Fixed 10Ge Ports
     * 8x R2,776.15 GLC-SX-MM : Ge Sfp Lc Connector Sx Transceiver
       * 1G Fiber uplinks - DWDM + Wan + 4Floors
     * 4x R1,166.00 SFP-H10GB-CU3M : 10Gbase-Cu Sfp+ Cable 3 Meter
       * 10G to DWDM, + Link to 2nd switch
     * 4x R8,300.00 SFP-10G-SR : 10Gbase-Sr Sfp Module
       * 10G fiber 10G links.
----
 * HQ R1.5M
   * R461k(2Core) x1pair
   * R167k(2Switch) x6pairs [2xNewServRack + 0xIBM +1xCore + 2xProduction + 1xDev]
  
 * DR R1.2M
   * R461k(2Core) x1pair N5K-C5548P-FA +L3
   * R167k(2Switch) x4pairs N2K-C2248TP-1GE [1xCore, 1xRack1 , 1xRack2, 1xSpare]
  
----
 * Option B
   * Using WS-C3750X-48P-S -> WS-C3750X-48P-S IPBase, poe + stack, 160Gbit Fabric, Max Forwarding 101.2 mpps,
     * 3750X-48P-MTBF-139,913 C3KX-PWR-715WAC-MTBF-664,055
     * Operating Temperature 40C
     * 2 x 10G MM fiber ports.
     * Problem: The airflow on the Cisco Catalyst 3750-X and 3560-X Series is "front and sides" to back airflow.
     * Spares
       * C3KX-PWR-715WAC= 715W AC power supply
       * C3KX-FAN-23CFM= Fan module spare
       * C3KX-4PT-KIT= Extension rails and brackets for four-point mounting for Cisco Catalyst 3750-X and 3560-X Series
   * HQ
     * Core 10Gig fiber concentration switch.
     * Pairs of WS-C3750X-48P-S IPBase(48x1G , 715W - PoE future, 2 can power-stack for redundant power in Datacenter, 10G uplink)
       * '''Q''' Air flow for WS-C3750X-48P-S ??? Want port side exhaust.
     
   * DR
     * Core 10Gig fiber concentration switch.

Cisco Nexus

Jumbo frames

  • 201210 - Bug in Nexus design reports normal frames over trunk link as jumbo due to vlan header.

  • HowTo enable jumbo frames.

    • Done through policy. 3 Layers, qos -> queuing -> network-qos 

      • system qos
  service-policy type queuing output xxxxx
  service-policy type queuing input xxxxx
  service-policy type network-qos xxxxx
  service-policy type qos input xxxxx

Nexus 4000

vPC

  • Rules and diagrams

  • Check witch link in a port channel will be used.
  • show port-channel load-balance forwarding-path interface port-channel 2 vlan 902 dst-ip 192.168.110.212 src-ip 192.168.110.200

# sh hardware internal carmel port e 1/1

  • QOS http://alleasysetup.blogspot.com/2011/10/qos-on-cisco-nexus-5k.html

  • FlowControll to VMware http://www.boche.net/blog/index.php/2010/11/29/flow-control/

  • vPC virtual port channel, let port's on 2 vPC peers look like they are from one switch, eliminating blocked ports on ethertrunk.
  • Step1 enable features needed on switches. 
    •    feature lacp
   feature vpc
  • peer-gateway - only for dumb hosts that do not use arp for gw discovery, but incoming packet.
  • pinning max-links 1
  • Before the peer link will come up, you also have to set up VPC keepalives to detect dual active peers 
    •   vpc domain 1
    peer-keepalive destination 10.10.10.14 source 10.10.10.13
  • VPC Rule 101
    • VPC peers are expected to forward a frame received on a member link out any other member link that needs to be used. Only if they cannot do so due to a link failure, is forwarding across the VPC peer link and then out a member link allowed, and even then, the cross-peer-link traffic can only go out the member link that is paired with the member link that is down.
  • Restatement of VPC Rule 101
    • To put it another way, VPC peers are expected to forward a frame received on a member link out any other member link that needs to be used. Only if they cannot do so due to a link failure, is forwarding across the VPC peer link and then out a member link allowed, and even then, the cross-peer-link traffic can only go out the member link that is paired with the member link that is down.
  • With VPC, one should always put keepalives on a different link than the VPC peer link, since their purpose is to detect a situation with the peer still up but VPC peer link down. You just can't do that if they're running over the peer link. Instead, you can use the management port. Or, if you put a separate point-to-point routed link between the peers, in parallel with the VPC peer-link, you can use that for the keepalives.
    • The L3 decision determines the outbound interface. If that outbound link is a VPC member link, the same forwarding rule is applied as for L2: the frame (packet) came in the peer-link, the member interface that is VPC-paired to the chosen outbound link is up, therefore drop the packet.

  • There are some definite requirements for using an EtherChannel for FEXs, such as the FEX parameter pinning max-links must be set to 1.

  • Check fiber signal strength. 
    •    sh interface transceiver details
  • Other options

  • Nexus 3000 Ultra Low Latency

    • 3064 48 x 10G + 4 x 40G - 1RU - L2 & L3 10Gbit/s on 64 ports 48-SFP+ + 4 x QSFP+ (40Gbit or 4 x 10Gbit )

    • 3016 16 x 40G quad sfp ports.

  • Cisco Nexus 5548P Switch Oct 2010

    • Need SPS license for FC.

    • L3 capable. additional hardware

    • Nexus2k Fabric extender
      • 2148T
      • 2248TP GE
      • 2231PP 10GE

  • Nexus 2000 config, upgrade, dual-home

CategoryCisco

cisco/Nexus (last edited 2017-11-08 19:13:36 by PieterSmit)