There is an unbelievable amount of junk on the net that can send one in all the wrong directions. When it comes to dynamips it is no different and it seems everyone has a different opinion on what works and what does not. It probably depends on the OS dynamips runs on and the alignment of the moon and the stars. For me the best solution thus far was to run Dynamips on Ubuntu Linux server edition. The reasons for this is due to the 100Mb memory Ubuntu server consumes to run the OS as well as the stability and responsiveness of Dynamips on linux. It has worked well.
To break out to real switches I started with the multiple USB adapter approach. Basically you map each USB to Ethernet adapter to a router ethernet port in the dynagen net file. Sounds simple, and while it is, the USB approach for me turned out to be unreliable. It would work, then it would not. This plus the hassle with using USB adapters from different vendors and drivers just becomes messy.
After more research I tried the “switch in the middle” method. This is by far the easiest, quickest, cleanest and most reliable route:
Overview:
So you are probably already using dynamips to run the routers. To break out the first thing is to create a trunk between your linux server and a breakout switch. Then create a bunch of vlans on the linux server. One for every port that will connect to real switches e.g. if R2 needs to connect to the real switches with Fa0/0 and Fa0/1, create two vlans for this router. In the dynagen net file map each ethernet port to the vlan ’sub interfaces’ which were created when the vlans were created. Create the same vlans on the breakout switch that were created on the server. Assign each access port that will link to the real switches into a different vlan. Each access port on the breakout switch now represents a router port e.g. fa0/6 in vlan 106 on the breakout switch represents router R6 fa0/0 interface which connects to Sw2 fa0/6.
Requirements:
1) A second network card
2) The vconfig utility.
apt-get install vlan
3) Kernel that will support Dot1q
modprobe 8021q
4) A breakout switch. Try to stick with a Cisco switch.
5) If you want to support QinQ / dot1q-tunnel, the breakout switch needs to support QinQ tunneling
Steps:
1) Create a trunk between your linux server and another switch that will become the switch in the middle or breakout switch.
2) Create the required vlans on the server. Or use the following script to do steps 1 and 2.
#!/bin/bash
#############
# vlansetup.sh
# http://21500.net
################# Enable Dot1q support
modprobe 8021q#### Set the ethernet MTU
ifconfig eth1 mtu 1536#### Create the Vlans
vconfig add eth1 101
vconfig add eth1 102
vconfig add eth1 103
vconfig add eth1 104
vconfig add eth1 105
vconfig add eth1 106
vconfig add eth1 107
vconfig add eth1 108
vconfig add eth1 109
vconfig add eth1 110
vconfig add eth1 111
vconfig add eth1 112
vconfig add eth1 113
vconfig add eth1 114
vconfig add eth1 115
vconfig add eth1 116
vconfig add eth1 117
vconfig add eth1 118
vconfig add eth1 119
vconfig add eth1 120
vconfig add eth1 121
vconfig add eth1 122
vconfig add eth1 123#### Bounce the interface
ifconfig eth1 down
ifconfig eth1 up
3) Configure the trunk on the breakout switch
interface fastethernet 0/24
switchport trunk encapsulation dot1q
switchport mode trunk
4) Assign the Access ports their vlans
!
interface fastethernet 0/1
switchport access vlan 101
switchport mode access
!
interface fastethernet 0/2
switchport access vlan 102
switchport mode access
!
interface fastethernet 0/3
switchport access vlan 103
switchport mode access
!
interface fastethernet 0/4
switchport access vlan 104
switchport mode access
! …
! …
! …
interface fastethernet 0/23
switchport access vlan 123
switchport mode access
!
5) If you want to support QinQ and CDP between the virtual routers and the real switchs, your breakout switch needs to support QinQ. Enable a dot1q tunnel on each interface.
interface range fastethernet 0/1 – 23
switchport mode dot1q-tunnel
l2protocol-tunnel cdp
6) Map the ports in the dynagen .net file
[[ROUTER R1]]
f0/0 = NIO_linux_eth:eth1.101
f0/1 = NIO_linux_eth:eth1.103
[[ROUTER R2]]
f1/0 = NIO_linux_eth:eth1.102
7) Connect the switches to the breakout switch with crossover cables. In this example Port fa0/1 on the breakout switch represents Fa0/0 on R1 and connects to Sw1 Fa0/1.
R1 Fa0/0 -> BSw Fa0/1 -> SW1 Fa0/1
R2 Fa1/0 -> BSw Fa0/2 -> SW1 Fa0/2
R1 Fa0/1 -> BSw Fa0/3 -> SW2 Fa0/1
Once I got this working I found that someone already did the hard work. Wish I saw this much earlier. The only difference between the two methods is that I used one breakout switch while MrPaul’s method uses the Dynamips switches and the breakout switch.
HOWTO Connect Real Switches Using One NIC & QinQ
This is the image MrPaul made:
Hi, I have implemented yours and MrPaul setup, I can ping from virtualrouters to realswitches, but I have an issue, the cdps from virtual routers dont reach the real switches, they get via the trunk only to the BOsw, but not passed to the qinq interfaces, any idea?? thanks for your time
Comment by gedeon — May 26, 2009 @ 5:55 am
l2pt is only supported on dot1q-tunnel and PAgP interfaces, not on “normal” trunks.
Comment by Greg — June 3, 2009 @ 5:06 pm
Hi Gedeon, you need to enable:
l2protocol-tunnel cdp
Comment by 21500 — June 3, 2009 @ 8:48 pm
Greg, not sure whether I follow you there, but if I understand correctly you mean it is best to have a breakout switch that supports dot1q tunnels. Agree 100%
Comment by 21500 — June 3, 2009 @ 8:51 pm
Thank you for it. But I want to ask you something.
1.) Second ethernet card is a mandatory ? I can use one ethernet card where in my Notebook ?
2.) When We can connect to a router and L3 switch’s router port ( no switchport is entered in a interface of L3 Switch) , this system ;Qinq works correctly through tunneling ? (in other ways i.e experts RS labs Routed between a router and Port of switch: ip routing)
if you clear my doubt I am very grateful.
Comment by ret2ww — June 11, 2009 @ 9:04 pm
1) yes, one is enough. In your case you use the notebook so you dont need a management ip. There is another way to work around one ethernet card, is to assign one of the vlan ’subinterface’ with an ip address. Plug the workstation you will work from into the breakout switch on the port that maps to that ’subinterface’ on the dynamips server. This will allow you to communicate with the dynamips server through the breakout switch from your workstation.
2)Umm, if I follow you correctly, yes, even if your breakout switch does not support qinq, you can still use the ‘no switchport’ on the real switches, but qinq and trunking to virtual routers wont work. To make full use of the real switches I would suggest using a breakout switch that does support qinq. This will allow you to trunk to a virtual router e,g fa0/0.6 encap dot1q 6 and QinQ between virtual routers.
Comment by 21500 — June 12, 2009 @ 2:42 pm
THANK YOU for your answer.
1.) you say that if I have a ethernet card on my pc, I will do a config like the below on my ubuntu ? this process is assign one of the vlan ’subinterface’ with an ip address ?
vconfig add eth0 101
vconfig add eth0 102
vconfig add eth0 103
vconfig add eth0 104
vconfig add eth0 105
if ı am wrong could you give us an example of this?
2.) actually I didn’t talk about break out switch’s qinq support. we think that. Our switches support qing and L3 functions. In our topologies, one router through switch connection is L3, not trunk
ı posted a picture about my question
according to a part of topology in the picturei it is work without problem in our ubuntu dynamips with real switch and virtual routers ?
http://img193.imageshack.us/i/routerandswitch.jpg/
Comment by ret2ww — June 14, 2009 @ 10:41 am
1) Yes that config will work. Since you use one PC as both your workstation and to run the dynamips server, you will ‘console’ to the virtual routers via the dynamips/dynagen console, you dont need an ip address on the subinterface. You only need the ip address on one subinterface if you use a dedicated dynamips server with one network card and a different pc as a workstation. Somehow you need to connect to the virtual routers right? The only issue here is how will you manage the real switches? Via console cable?
2) Yes it will work, whether the breakout switch supports qinq or not.
Comment by 21500 — June 14, 2009 @ 11:07 am
Hi thank you again
Yes I manage the switches with console cable because my equipments is only on my job station and I don’t pay more money for them so I setup my topologies in my job place with my notebook and my companies switches in my spare time. I am working as the shift
after the right configuration on my topologies,Can I connect with telnet or ssh my switches over the virtual routers without problem?
thank you for interesting.
Comment by ret2ww — June 14, 2009 @ 9:58 pm
Does cdp work both ways with this setup?
Comment by buddy — July 15, 2009 @ 4:01 am
Hey, nice writeup and thanks for the mention.
You have any clever solutions to doing 5 ATM OC3 ports on a 7200 for the IP Expert Vol 2 Section 4 lab? I need this particular setup for the Cell Mode MPLS.
Comment by MrPaul — July 21, 2009 @ 12:23 am
Hi, thnx for writeup, I am just curious can i design full INE topology working on this scenario, though it seems workable but dont have idea either it would work. would appreciate if anybody could give me insight or issues involve ….some ppl put quad nics to integrate dynamips with switchs…..is this scenario would work fine with Internetwork epxert topologies ?
thnx in advance.
Comment by napolo — March 14, 2010 @ 10:55 am
Hi napolo, yes you can build a full topology. For RS I would suggest the breakout switch method and for SP only dynamips. This setup is overkill of SP. Even though I have the switches, I never use them.
Comment by 21500 — March 15, 2010 @ 7:25 pm
this topology working perfectly so far, only problem i found that wen dot1q-tunneling enabled on Breakout switch ports, CDP neighbor(Routers running on dynamips) learnt on switches correctly but Routers are not learning switches as cdp neighbor.
any thing missing here…..though no problem with working observed.
Comment by napolo — April 14, 2010 @ 9:15 pm
luckily for me, i have a spare 7200 router here in my office and
I have implemented a breakout solution using EoMPLS pseudowires.
In dynamips, my routers connect to ethernet ports on PA-8E cards on a 7200.
On the outside of the lab the switches connect to a real 7200 router with PA-8E cards. I connect the internal 7200 router to the external router using a cloud and run ldp between the two. No extra PC interfaces, dot1q enabled PC or fake switches are required
I then xconnect each router interface to the appropriate switch interface across the MPLS enabled link.
All of this is transparent to CDP etc, so it is really like a bunch of cables connecting inside to outside. downside is you need a router or metro switch on the outside (which you could probably sell to fund some real 2600xms
)
anyway, just another idea thrown into the hat
Comment by hroyd — June 7, 2010 @ 1:21 pm
“metro switch on the outside”
All you need is that switch, read the post
Comment by 21500 — June 7, 2010 @ 2:05 pm
yes i did read the post. i said no fake switches. i know you use a real switch on the outside, but thats no diff from me using a real router or metro switch.
i see people have issues with CDP with your solution. i don’t know if thats true as i haven’t tried it myself, but there is no issue with cdp for me.
i was only pointing out a simpler alternative, which is all it is. as i have a free 7200, its a good one for me. may not be a good idea for anyone else.
Comment by hroyd — June 7, 2010 @ 3:32 pm
dude, you dont get it, there are no fake switches in the method above. In fact your virtual 7200 router is exactly the same concept as the ‘fake’ switches you are referring to and the break out 7200 is the same as the break out switch. Simple method, it is not.
Comment by 21500 — June 7, 2010 @ 8:11 pm
m8.. i do get it
did i mention that you use fake switches… no
do you also use multiple interfaces like i also mentioned…no (well thats how you started out)
i was referring to the other breakout methods touted out there like the ones yours is ‘better’ than
you use unix with a kernal that supports 802.1q. you have the vconfig untility installed and a load of 802.1q config and then have to rely on dot1q tuneling to fix your cdp issues… what do you do if you run an xp machine btw
i simple run some eompls xconnects to give me a transparent ethernet circuit from inside to outisde.. is yours transparent like that?
i know which is simpler, but like i said its not for everyone. thus for me this is the best solution
no moons and stars required
ttfn
Comment by hroyd — June 7, 2010 @ 9:39 pm
Guys,
I found that the only reason that some people are having issues with CDP is that the breakout switch should be another 3560!! If you use a 3550 the CDP protocol cannot travel across the Layer 2 tunnel. I’ve managed to get this working using MACOSX, now i’m just thinking maybe it would be ok to buy an 8pc 3560 as the breakout switch … that should be OK.
Comment by GungaJim — June 9, 2010 @ 2:54 pm
Hello everybody,
How do we telnet to both real switches and virtual routers running in Dynamips (Ubuntu)? How can we use serial2net program with Break out switch to telnet to all real and virtual devices? Any suggestions would be appreciated.
Thanks
Comment by Oscar — June 25, 2010 @ 10:38 am
Hello GungaJim,
Yes i was using 3560 n observed cdp issues. I will try to replace it with 3560.
thnx.
Comment by napolo — June 27, 2010 @ 4:46 am
Hi,
I have problem doing router on stick config, trying to route intervlan. Could any body suggest how to ?
Comment by marao — July 24, 2010 @ 7:28 am
Oscar,
I configured a VRF for switch management. This allows me to telnet to the switches from my PC and still maintain an accurate switched lab. The only difference is the VRF VLAN interface and the port I assign to the VLAN.
mp
Comment by mprofitt — August 3, 2010 @ 3:20 am