Global Knowledge Nexus7K Training Exp.

ninjaturtle

I have arrived in Washington DC for the Global Knowledge Nexus 7K one week training course. For the next week it will be nothing but Nexus for me, with the occasional diversion to something else networking. I thought it would be a good idea to log my experience, as well as share with others the knowledge I obtain.

This is the first training with Global Knowledge for me, and some of you might remember I posted awhile ago about whether or not it's worth it. Well I decided to take the plunge and pay for the entire trip out of pocket. Why? Because my current job won't pay for things like this, and simply put I want to learn it. I feel this class will not only help me in my CCNP-DC journey, but for the next job. I'm starting to find that with the CCIE-DC journey, CCNP-DC leading into it requires a lot of $$$ for lab time and materials. I'd say more than any other CCIE, respectively. Simply because you no longer have access to your own home lab. I'm certain when I get my digits, I'll be saying it was all worth it.

On another note, I know many don't have the opportunity to attend a class like this one. It's expensive!! I created this opportunity with my little friend Mr.Visa credit card. So I'd like to share the knowledge and experience with others. I see so many people here on the forums just pouring out knowledge and information, so now it's my turn to return the favor.

I'm currently sitting at Starbucks reading the Nexus NX-OS switching book, and reviewing Cisco configuration guides. Class starts tomorrow. Stay tuned...

Cheers,

lrb

Nice, looking forward to reading about your thoughts on the course Hopefully they have updated the materials to have the F3 cards!

ninjaturtle

Day 1

The Global Knowledge facility here in Washington DC is pretty impressive. Everything is high end from the classrooms, digital white boards, breakroom ect... My instructor is awesome!! She's very well versed in Nexus, as she's been involved with Nexus since it came out. They drop 4 books on your desk, which included 3 lecture volume books about 400pages each, and one lab manual about 400+ pages. Great detail in the books explaining everything form A to Z. Today was a complete breakdown of the Nexus 7K switch, from the chassis, to the fabric modules, to the SUPs. How they all work with each other, how some M1 and F1 cards work with each other, while some cannot. They did touch on the F3 cards Irb. From my understanding, the F3 cards can't hold as many entries in it's tables compared to say the M1 cards. I guess that's the price you have to pay for the speed, at least for now.

We got into FEXs in great detail. Per the instructor, the workbook had some errors where it showed N2K FEXs dual homed to a pair of Nexus 7K parent switches. This is not allowed, you can only have N2K FEX connected to one N7K. We also discussed how one of the N2K FEXs has what's called a dedicated port which is marked in yellow. The uplink ports share an ASIC in this particular model, so if you want true 10GB speed on that trunk you'll need to be plugged into that port and not sharing with the others.

Somebody in the class mentioned they were having issues with there FEXs back at the office, where they were saturating the uplinks ports. You have to be pretty careful how you lay out your N2K FEXs in your network. You can't just throw them up in a rack, and just connect a bunch of servers thinking it can handle it. Granted, with the new 23xx series you now have 40GB ports to help with the bandwidth needs, but that will soon get pushed too. Does anybody know with the new 23xx series, do you still have a dedicated port? i thought I saw on a 2348TQ marked with 2 ports in yellow.

Lab was pretty basic today, as we just worked with VDCs. The labs are interesting as you rely on others in the class for the complete topology to work. If somebody else doesn't have there VDCs working properly, with a configured interface for example, you won't be able to ping them and others. Some of the pods were running NX-OS 6.1 and some where running 6.2. With the 6.2 code, we found we couldn't do a copy run start, as it errored out. We were trying to do the rollback feature. Apparently it has something to do with VDC1, where if a change was made in VDC1, you can't make changes in other VDCs without saving it in VDC1 first. But this isn't a problem in the older 6.1 code.

We covered quite a bit in the 9.5 hours on day 1. I will definitely have to go back and review the lecture recordings.

All in all I'm enjoying the course. I don't feel bad that I spent all that money, but then again my credit cards don't feel the same. It will pay off one day. I know it will because when you invest in knowledge, the possibilities are endless.

Cheers,

ninjaturtle

Day 2

Today we got into more of the nitty gritty of the various technologies of N7K. A couple of people in the class, mainly one person learned that they had to redo some configuration on their N7K. They took advantage of the ability to create VLANs 1002, 1003, and 1004 on the Nexus, but they were going to still connect some IOS switches. As we all know, those VLANs are reserved in IOS. That could definitely lead to problems.
We discussed some of the upgrade paths, many of which can turn into nightmares, so definitely plan ahead. Try as much as possible to build for expansion, otherwise things can get rough. I will have to find the slide showing the upgrade paths, and the card types. There is also a link to an older Cisco Live presentation that goes into great detail. It had some really nice diagrams depicting the options you have and paths.
Got a chance to setup some FEXs, which is pretty straight forward. But as I discussed yesterday, you have to be careful how you setup your FEXs. You can also make FEX interfaces L3, but not really sure why and when you would need to do that, but it's possible. You can also have subinterfaces on the FEXs too. I learned that the FEXs get there OS from the parent, which takes about 10min. I found that really odd that it takes so long, on this high end performance switch. Same thing with a reboot of a N7K, it's about 30min or more apparently which to me is crazy.
We also spent some time today talking about Port Profiles and getting those setup. If anybody wants to see those notes, I'd be happy to throw those up. I left those notes back in the classroom, so I'll have to add those at a later time.
We spent the remainder of the afternoon going over STP and STP Extensions, as we'll be hitting on FabricPath tomorrow morning. It's interesting that the person who invented STP, is helping with TRILL which will put out STP one day. I thought INE mentioned that she helped with FabricPath too, but not 100% on that, I'll have to go back and listen to that lecture again.
And the final topic today was 802.1 Q-in-Q Tunnel Ports. It was really interesting to hear about the service provider end, and how they take advantage of Q-in-Q.
We did more labs today which included: AAA, STP, MST, PortProfiles, VDC, FEX.
Lots of details today and a ton of notes that will take forever to upload those on the forums. I really didn't expect to cover so much in such a short period of time, but I'm loving it. I'll definitely have to review my notes and lecture videos when I get back home.

Doyen

What a surprisingly small world! I'm the administrator that sets up their data center labs each week for North America. I'm glad you are enjoying your Nexus 7K course . I know that their courses are expensive and I hope you getting your money's worth out of it. As of late, I've been busy with/at work to take advantage of their training or visit this site as often as I used to. However, it is nice that when I had a chance to stop by here for a quick look around that I ran across your topic. I definitely enjoyed reading your posts. I'm also looking forward to what you will post for the rest of the week about your experience.

ninjaturtle

Doyen wrote: »

What a surprisingly small world! I'm the administrator that sets up their data center labs each week for North America. I'm glad you are enjoying your Nexus 7K course . I know that their courses are expensive and I hope you getting your money's worth out of it. As of late, I've been busy with/at work to take advantage of their training or visit this site as often as I used to. However, it is nice that when I had a chance to stop by here for a quick look around that I ran across your topic. I definitely enjoyed reading your posts. I'm also looking forward to what you will post for the rest of the week about your experience.

Heya Doyen! Wow, that is a surprisingly small world! I copied some of the data center lab pics of MyGK account. That place is awesome!! I had to add those to my picture archive.
I'm glad you're are enjoying the post. Looks like a couple people are reading it!

My instructor actually got on the phone last night with GK support, as the AAA server isn't working properly with what appears to be affecting only Pod2. The student and NewAdmin accounts aren't authenticating on the N7Ks. The issue is still outstanding at the moment.

Doyen

Yep, Annette was actually speaking with me about that the other night. Even after verifying each step of the lab, the infrastructure team member and I could not find anything wrong with the configs entered or how radius was setup on the server side other than both accounts failing authentication at .22. It's good that the further labs were dependent on it, but we are going to have to look at that pod over the weekend and see if we can recreate the issue again & then troubleshoot it. It was one of the reasons why I stayed late last night. Still, I'm sorry that those students had to deal with that.

Data center is amazing and may be the next certification path that I may explore since it is fun to work with. The DCUCS. DCUFI, and DCUCI are intriguing- interesting to troubleshoot & fun to learn. I wish you well on your CCNP Data Center journey and whatever you determine lies beyond that.

ninjaturtle

Day 3

Today we spent most of the day on vPC and then got into FabricPath. I learned quite a bit about vPC, vPC+, EvPC and double sided vPC. We had a good lab on double sided vPCs. We also got into the details of FHRP and best practices in Nexus 7K, followed by a lab. We had various scenarios where we would break things and verify FHRP was indeed working. The way some of these labs are designed, you have to work with the other pod groups and make sure you work together. We started some very good discussions and started question various portions of the lab. An excellent way to learn and dig deeper into the technology. I find many times when I lab at home, I start just going through the motions. But in the classroom setting you take a step outside of the labbing box if you will, and really analyze what you're doing and trying to accomplish.

The following are very good documentation about vPC best practices on the N7K and troubleshooting:
http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf
Cisco Nexus 7000 Series NX-OS Troubleshooting Guide -- Troubleshooting vPCs - DocWiki

We got into discussing the vPC Peer-Switch command and how exactly it works. Basically, when peer-switch is enabled on each of the N7Ks switches they share a virtual bridge ID which allows both of the switches to act as root for the VLAN. I spent some time Googling back at the hotel, and found I still have a lot of reading to do on this topic. I found some hybrid setups with peer-switch enabled that I'm interested in reading more about. This will definitely be on the To-Do list when I return home.

Next we got into the details on Bridge Assurance and how it protects against certain problems that can cause bridging loops in the network. Apparently upon further investigation this is supported in IOS version 12.2(33) and higher. But it's only supported with Rapid PVST+ and MST which also explains why it's used on the Nexus. We learned in the lab today that BA has to be turned on both ends, otherwise you'll get a BKN status with a type BA_Inc which shows an inconsistent state.
To enable BA on an interface you would enter 'spanning-tree port type network'. The best practice now per Cisco is to enable this port type across the entire switch with 'spanning-tree port type network default'. This will set the default port type. If you set the default port type, you have to be careful that all ports are connected to a BA enabled switch, otherwise as I mentioned earlier you'll get the BKN status with type BA_Inc. So when you see type 'network' you know it's running Bridge Assurance.

We got into the theory of FabricPath and how it came about. I find it very interesting how they developed FabricPath and how they took advantage of IS-IS. And also how it relieved many shortcoming of STP. I need to go over the FabricPath introduction recording again, as our instructor gave an excellent intro to it. I'll have to try and write it all out here on the forum.

ninjaturtle

Doyen wrote: »

Yep, Annette was actually speaking with me about that the other night. Even after verifying each step of the lab, the infrastructure team member and I could not find anything wrong with the configs entered or how radius was setup on the server side other than both accounts failing authentication at .22. It's good that the further labs were dependent on it, but we are going to have to look at that pod over the weekend and see if we can recreate the issue again & then troubleshoot it. It was one of the reasons why I stayed late last night. Still, I'm sorry that those students had to deal with that.

Data center is amazing and may be the next certification path that I may explore since it is fun to work with. The DCUCS. DCUFI, and DCUCI are intriguing- interesting to troubleshoot & fun to learn. I wish you well on your CCNP Data Center journey and whatever you determine lies beyond that.

Now that's an even smaller world that you actually worked with Annette. She's an awesome instructor! I'm really enjoying her lectures. I'm actually in Pod 2, and it hasn't stopped us from moving forward in the labs. You'll have to let me know what you find, as I'm interested now. It looks like only pod 2 is having the issue, for both the remote and on-site groups. The other pods are okay. That should narrow it down a bit, as to what could be the problem, respectively. Thanks for helping to troubleshoot the issue Doyen!

I'd have to strongly agree with you, the data center track is an awesome track! The only trouble with the data center track is finding the open time slots to lab, as resources are limited and expensive! The CCIE-DC definitely sits in front of me and is my ultimate goal.

Cheers mate,

ninjaturtle

Day 4

We started really speeding it up today and started cranking through the slides. We spent most of the day today going over FabricPath and OTV. MPLS was also on the list, but we didn't have time. I asked if we could briefly go over MPLS, but she took the time to go into detail about the MPLS features on the N7K. I do wish we could lab more, and really get more hands on experience. We do have access to the labs at night, but we are still dependent on each other as a group. I'm definitely going to have to come back to MPLS later, as I found that stuff pretty interesting. Especially the traffic engineering stuff, but that looks like it gets deep real quick.

At this point I can definitely say I have a lot of review and reading to do when I get home. I'll definitely read through all 3 volumes again, and crank out some notes. The lab manual will be nice to carry around and review lab commands at lunch time at work.

We did more but I'm pooped today and need some sleep, after I finish transcribing some more notes. I will say this though, everybody who has a blog and takes times to write articles almost daily ...I commend you. It's not easy and it can be tedious at times.

Doyen

Ninjaturtle,

Read the PM that I sent you.