CSR1000v

cpartin

I got this working based on instant000's link (thanks!), and noticed the same thing with only one core being pegged. I tried passing the "-smp X" option to QEMU but that didn't help. Found a link which states QEMU doesn't support SMP on windows hosts, only on linux via KVM. Don't have a linux VM handy at the moment or I'd test that.

Computer Networking Studies with GNS3: 1. Junos with GNS3 (ver 0.8.3.1) on Win7 Host, qemu vs VirtualBox

Jackace

Anyone else able to try this on VirtualBox and see if they can get it to work? I'm mostly curious now if it is virtualbox or something on my system that won't let the VM run.

bermovick

I got onepk running in virtualbox but it was pretty terribly slow. I haven't been able to get the qemu ... thing to work.

Jackace

bermovick wrote: »

I got onepk running in virtualbox but it was pretty terribly slow. I haven't been able to get the qemu ... thing to work.

I got the onePK image to run in virtualbox as well, but when I tried to pull out the ova file for the router it uses that would never run in virtualbox and if I created a new VM using the vmdk file for the router that vm would boot, but crash after the grub menu.

bermovick

Ahh. I accidentally ruined my ova image by trying to get rid of all the stupid Ubuntu shite included (gnome-sudoku anyone?) and install a less resource-intensive WM without making a backup of the original image first. I don't want to download another 5G image so more or less just stopped trying.

cwestmac

Hi guys,

Has anyone tried running ESXi under VMWare Workstation on their own PC and setting up the CSR1000v within ESXi? Does it work almost as well as having a dedicated ESXi box? Are there any major problems? My personal PC has 32GB of RAM and I have a SSD I can use for swapping, so it would save me close to $1000 it would cost me to get a server w/ 64 or 72GB of RAM.

I'll be testing this config myself soon, but am deep in studying for my CCDP exam right now for the end of May. Once that's knocked off I'll be going for the CCIE R&S so I'm trying to gather as much information as I can.

Thanks!

cwestmac

And if I went that route... I guess I'd need a NIC in my computer with two ports. One for my regular network, and one that plugs into a switch for my CCIE lab network? Similar to this topology, except the "hypervisor" in this case would be the second NIC on my PC:

CCIE RSv5 Equipment Build - IEOC - INE's Online Community

Please let me know if this is correct or not.

gorebrush

I think nesting is a bad idea myself.

But running ESXi on bare metal is awesome. Do you have another machine? You could dual boot your main machine with ESXi the manage the routers from your other one

cstambaugh

Ok, so I'm in the "finding what works best" for me boat right now. I have a few CSR1k's in ESX, also tried GNS3 QinQ trunking to a breakout switch yesterday. They both work well up to a point, and its a stupid point but maybe I'm not thinking correctly on it.

In GNS3, to QinQ truck, you have to assign vlans on the vswitch side that correspond with your physical breakout switch side. The dilemma I'm running into is in a lab book when an interface on a router is say needing to be assigned to a different vlan for lab purposes. How do you do this? Do you just run sub interfaces on the routers?

Example:
R1 f0/0 - vlan 101 QinQ to breakout switch. Breakout switch assigned vlan 101 to int f0/1 to plug into my switch stack.
(using vlan 101 for port mapping for R1's f0/0)
Lab book says assign f0/0 to vlan 34. Should I use a sub interface for 34, or should I redo my QinQ mapping so vlan 34 is my assigned vswitch to physical switch map?

How are you guys doing it?

I'm going to try both the CSR's and the vIOS (onePK) today to see which is best.

edit:
Using the CSR 1k's is so much easier. Its not as visual, but its less complication than what was involved with GNS3. I'm definitely going this route.
I just create the sub-interface on the CSR, bind that to the vlan the lab says the physical is supposed to be on. Bind that vlan to the physical switch interface to emulate it plugging in there. Everything works.

Trewq2014

Hello fellows!

Someone have expierence boot vIOS from OnePK under ESXi? I got some issue, when I boot vIOS and configure interface (Gi0/0 for example) and spell some symbol like ; ' or \ - vIOS crash...
Maybe I need some voodoo magic to solve it? ))

My general idea - build under ESXi topology like in INE v5 workbook, where for switch I wanna use ESXi vSwitch.

Another bad expierence: I tried boot vIOS under VMware Workstation, but my vIOSes doesn't want communicate..I have some suggestion about Ethernet driver which VMware use for Virtual Ethernet Controller - it's doesn't support Gigabit ethernet...

gorebrush

The CSR is great! Brian McGahan says mostly anything he has tried works so if he thinks its good enough - that's endorsement enough.

I tried IOSv on ESX and while it worked fine for me, I'm wary in case the bugs divert my attention away from actual learning.

cstambaugh

How's the performance on the CSR1000v when using an SSD drive as swap in esx?

gorebrush

I'm not swapping at all with 10 routers now. I do have 32GB of RAM though. I also bought a 480GB SSD for the server and i've moved all the CSR instances over to that, along with my Wireshark/general purpose server too.

There is a tweak you can make on the ESX server to enable memory sharing, and some people have reported that this certainly helps with memory usage.


Here is the tweak: -

Large Pages can be disabled on the ESXi Host under Advanced Settings -> Mem -> Mem.AllocGuestLargePage and change it to 0

use this image : csr1000v-universalk9.03.10.02.S.153-3.S2-ext-C1-M2560-N3-DS8.ova

and allocate 1 vCPU and 2.5G to the csr1000v VM (C1 : 1vcPU and M2560 = 2.56G RAM)

this should improve your memory usage..

networkjutsu

gorebrush wrote: »

Large Pages can be disabled on the ESXi Host under Advanced Settings -> Mem -> Mem.AllocGuestLargePage and change it to 0

use this image : csr1000v-universalk9.03.10.02.S.153-3.S2-ext-C1-M2560-N3-DS8.ova

and allocate 1 vCPU and 2.5G to the csr1000v VM (C1 : 1vcPU and M2560 = 2.56G RAM)

this should improve your memory usage..

I saw this at IEOC. Enabled it and my RAM usage went down from 30GB to 23.5GB. Not using that version though. I am using the IOS XE 15.4(2) since this says to use 15.4.

gorebrush

I copied it from there

R1#sh ver
Cisco IOS XE Software, Version 03.12.00.S - Standard Support Release
Cisco IOS Software, CSR1000V Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.4(2)S, RELEASE SOFTWARE (fc2)

I'm using the correct IOS too. The CSR is awesome.

networkjutsu

networkjutsu wrote: »

I saw this at IEOC. Enabled it and my RAM usage went down from 30GB to 23.5GB. Not using that version though. I am using the IOS XE 15.4(2) since this says to use 15.4.

UPDATE: So from 30GB to 11.5GB being used. Holy crap, this TPS (Transparent Page Sharing) is awesome! I tried to spin up 10 more CSRs and jumped to around 30GB and after a while it started to come down to around 15GB! I guess my Samsung Evo 120GB that I just bought for $70 went to waste. I may just have to use it as a boot drive for my main PC.

gorebrush

Heh, I bought a 480GB SSD for my server and it primarily hosts my CSR's - though I'd been itching for an excuse for more SSD storage anyway because I do mess around with a lot of VM's and doing that on mechanical disks is OK but a bit slow.

zener_1987

Hi dears
can i run 20 csr 1000v with the below pc's hardware

cpu core i7
ram 32g

please share with us your experience

gorebrush

Yup, should run fine. I ran 20 of them with an i5 and 32GB RAM.

You should enable TPS - Transparent Page Sharing - this enables ESX to create one copy of parts of memory for each machine - IIRC use of 20 routers was like 10-12GB if I recall correctly.

How to Modify Large Memory Page Settings on ESXi

OfWolfAndMan

gorebrush wrote: »

Yup, should run fine. I ran 20 of them with an i5 and 32GB RAM.

You should enable TPS - Transparent Page Sharing - this enables ESX to create one copy of parts of memory for each machine - IIRC use of 20 routers was like 10-12GB if I recall correctly.

How to Modify Large Memory Page Settings on ESXi

I am assuming you used the ISO image and not the .ova for this? I am using an ova file with small deployment right now which runs me 2.5 Gb per VM. I have 32, but was unsure how far I could cut back until things started becoming unstable.

gorebrush

.OVA I used too - small deployments. I still had 2GB per VM so it was overprovisioned, (20x2=40) but with TPS it doesn't actually use all of the physical memory.

OfWolfAndMan

Awesome. Worked like a champ. Running 10 CSRs in image below:

advanex1

Can I ask what hardware you guys are running? Doubt I could run very many instances on the MacBook, but just figured I'd check it out. Have to download the IOS from cisco when I get back to the room.

gorebrush

I ran mine on: -

i5-4570S
32GB RAM
Asrock Z87 Extreme4
500GB SSD
Loads of NIC's

d4nz1g

These guys support openflow API as well.. A couple of CSR1000v and a Centos running OpenStack neutron could be a nice combination

theodoxa

advanex1 wrote: »

Can I ask what hardware you guys are running? Doubt I could run very many instances on the MacBook, but just figured I'd check it out. Have to download the IOS from cisco when I get back to the room.

Custom Built Server:

MSI H97 PC MATE Motherboard
Quad Core Intel Core i5
24 GB 1833 MHz DDR3*
Dual Gigabit PCIe NIC**
120 GB SSD
1 TB HDD

*I tried to upgrade it from 16 GB to 32 GB, but ESXi wouldn't load with 32 GB. The motherboard and ESXi both claim to support 32 GB of RAM and I switched the sticks around to confirm one simply wasn't bad and moved them around in the slots to confirm that one of the slots wasn't bad. I ended up leaving 3 x 8 GB (24 GB) in.

**The Realtek NIC required me to add VIBs to the ESXi image. I also had to add VIBs for the onboard SATA controller.

I also have a Dell PowerEdge 1950 III with Dual Quad Core Xeon Processors and 32 GB or RAM, but it is ridiculously loud and my server room (spare bedroom) becomes a sauna when I turn it on. The custom built server cost more, but it is reasonably quiet and doesn't produce as much heat, though the room is still a bit warm from the server and Cat. 3560E breakout switch.

OfWolfAndMan

advanex1 wrote: »

Can I ask what hardware you guys are running? Doubt I could run very many instances on the MacBook, but just figured I'd check it out. Have to download the IOS from cisco when I get back to the room.

-Mobo: Sabertooth X79
-Memory: 32 GB Quad channel 1866 Mhz RAM
-Intel i7 3820 Quad Core (3.6 GHz)
-Storage: 1x 128 GB SSD, 1 TB HDD, 1 TB iSCSI LUN (Off of my NAS)
-Storage (Win 7): 2 x 7200 RPM 500 GB HDD (Probably needs to be replaced with something newer. I think one is about to die).

Keep in mind this is also a dual boot of a Win 7 Desktop so I can play my PC games without any potential downsizing of my hardware . I use Grub4DOS to do so.

If you are attempting to do this on VMWare fusion, you're not going to be able to do TPS like gorebrush mentioned above.

d4nz1g wrote: »

These guys support openflow API as well.. A couple of CSR1000v and a Centos running OpenStack neutron could be a nice combination

Did not know that! Will have to check it out.

Levithan

Anyone know the current recommendation for the amount of routers and switches needed for the CCIE Lab, as well as the IOS required? Is it still 15.2?

I've got an old 2950 that's been sitting in the closet with 8GB of ram. Looking at ebay, I can upgrade it to 32GB for about $60.
I already have ESXi 5.1 installed and 2 CSR1000v's installed (15.1) I believe.

Has anyone been labbing this way, and what have their experiences been? I've been studying steadily for the written, but am itching to lab with something larger, as all my labbing has been with GNS3 and the 7200 router thus far....

theodoxa

Levithan wrote: »

Anyone know the current recommendation for the amount of routers and switches needed for the CCIE Lab, as well as the IOS required? Is it still 15.2? I've got an old 2950 that's been sitting in the closet with 8GB of ram. Looking at ebay, I can upgrade it to 32GB for about $60. I already have ESXi 5.1 installed and 2 CSR1000v's installed (15.1) I believe. Has anyone been labbing this way, and what have their experiences been? I've been studying steadily for the written, but am itching to lab with something larger, as all my labbing has been with GNS3 and the 7200 router thus far....

The INE Workbooks use 10 CSR-1000V Routers for the individual technology labs and 20 CSR-1000V Routers for Full Scale (multiple technologies) labs.

For Physical Routers, they call for IOS 15.3T or 15.4S with Data and Security Licenses (For CSR-1000V, the AX License should be enabled).

But, the "eigrp upgrade-ecli" command requires IOS 15.4(1)S, IOS 15.4(2)T, or IOS XE 3.11S, so I would go with at least that. I am running IOS XE 3.14S (Equivalent of IOS 15.5S).

R1#show ver 
Cisco IOS XE Software, Version 03.14.00.S - Standard Support Release 
Cisco IOS Software, CSR1000V Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.5(1)S, RELEASE SOFTWARE (fc5) 
Technical Support: http://www.cisco.com/techsupport 
Copyright (c) 1986-2014 by Cisco Systems, Inc. 
Compiled Thu 20-Nov-14 17:11 by mcpre 


Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc. All rights reserved.  Certain components of Cisco IOS-XE software are licensed under the GNU General Public License ("GPL") Version 2.0.  The software code licensed under GPL Version 2.0 is free software that comes with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such GPL code under the terms of GPL Version 2.0.  For more details, see the documentation or "License Notice" file accompanying the IOS-XE software, or the applicable URL provided on the flyer accompanying the IOS-XE software. 


ROM: IOS-XE ROMMON 

R1 uptime is 0 minutes 
Uptime for this control processor is 1 minute 
System returned to ROM by reload 
System image file is "bootflash:packages.conf" 
Last reload reason:  



This product contains cryptographic features and is subject to United States and local country laws governing import, export, transfer and use. Delivery of Cisco cryptographic products does not imply third-party authority to import, export, distribute or use encryption. Importers, exporters, distributors and users are responsible for compliance with U.S. and local country laws. By using this product you agree to comply with applicable laws and regulations. If you are unable to comply with U.S. and local laws, return this product immediately. 

A summary of U.S. laws governing Cisco cryptographic products may be found at: http://www.cisco.com/wwl/export/crypto/tool/stqrg.html 

If you require further assistance please contact us by sending email to export@cisco.com. 

License Level: ax 
License Type: Default. No valid license found. 
Next reload license Level: ax 

cisco CSR1000V (VXE) processor (revision VXE) with 785444K/6147K bytes of memory. 
Processor board ID 93DTHKWQPN9 
3 Gigabit Ethernet interfaces 
32768K bytes of non-volatile configuration memory. 
2506280K bytes of physical memory. 
7774207K bytes of virtual hard disk at bootflash:. 

Configuration register is 0x2102