Sunday, December 17, 2017

TIWA: "Today I was Asked..."

"Today I was asked..." is a new blog series where I go over networking questions that were asked to me by co-workers or associates.

In my current role, I am a senior Network Engineer. I am also very public about my readiness for the CCIE exam. Sometimes my co-workers treat me as if I already have a CCIE and know "everything" about networking.

This blog series is about times when I was asked a networking question and didn't have the answer or I found myself second guessing my answer which causes me to go home and build labs around these topics and make sure I understand them.

As a result of building a lab and doing research I thought it would be great to document the efforts as it may be useful to someone else. For a selfish reason it also helps me to write out information about topics to better memorize them.

I hope you enjoy my first post in the series: "How can you tell if Policy Based Routing is routing packets? Can you see it in the routing table or somewhere?"

TIWA: How can you tell if PBR is routing packets?

Today I was asked: "How can you tell if Policy Based Routing is routing packets? Can you see it in the routing table or somewhere?"

I didn't have the correct answer for that at the time, so I looked it up, whipped up a lab and here we are :)



SPOILER ALERT: debug ip policy FTW!!!!

My co-worker was correct to assume, there is nothing in the routing table to tell you there is a policy based route installed or that packets are being forwarded in contrary to the routing table. Checking the routing table alone is not enough to determine the flow of packets. You must also check the ingress interface for policies.


To  build Policy Based Routing (PBR) you need 3 basic ingredients:
  1. access-list (standard or extended)
  2. route-map
  3. reachable next-hop

In the graphic above we are sourcing our traffic from the Loopback0 interface: 1.1.1.1 on R1 (on the right). In the middle is our PBR router. Using the default route traffic will traverse the PBR router using the next-hop of 10.10.10.6. We want to use Policy Based Routing to route traffic sourced from: 1.1.1.1 to the next-hop 20.20.20.6.


PBR: (configuration)


access-list 1 permit host 1.1.1.1

route-map PBR permit 10
 match ip address 1
 set ip next-hop 20.20.20.6

interface FastEthernet0/0
 ip policy route-map PBR


IOS: (verification)


To see if PBR is turned on or is in-use:


show ip interface fa0/0
show ip interface fa0/0 | i Policy

show ip policy

show cef interface fa0/0
show cef interface fa0/0 | i Policy|polic


To verify the configuration elements of PBR:


show route-map PBR

show access-list PBR

NOTE: Hit counts are good indicators your policy is getting hit.


To verify the traffic is hitting the Policy as it ingresses and interface:


debug ip policy

NOTE: Using the "debug" command on a policy which see's alot of traffic can blow up your console. Best practice to use ACL's to filter the "debug" output to a particular host or hosts.

Thursday, December 14, 2017

Pi-hole: Day 1 (First 5 minutes)

Wow! This is so simple why haven't I been using this for years?

Pi-hole is a DNS blackhole server. It is so light weight it can run well on meager hardware such as a RasberryPi.

By default it pulls from various blacklist DNS sources, to help block the dns requests for many common and popular malware domains as well as ad-campaigns.

I had this up and running in about 5 minutes as a VM. I changed my DNS offering from my router's DHCP server to point to my new Pi-hole server. Using my phone I dis-associated and re-associated with my WiFi so it would received the new DNS offering. I immediately went to 2 sites that are notorious for banner and popup ads.

The first site's banner ads we gone!!! And the site seemed to load a bit quicker.

The second site I tested is notorious for pop-ups as well as banner ads. The banner ads were gone and when I invoked a popup it quickly disappeared off screen. (This isn't a feature of Pi-hole but a result of the DNS being blackholed)

The Pi-hole dashboard immediately registered the requests. The below screenshot came directly after my two connectivity tests from above within about 5 minutes of being up and running.



Also, I went ahead and turned DNS forwarding on and I chose the Quad9 DNS (9.9.9.9). I've heard good things about it. I've tested reachability to it and it tends to be a bit quicker than Google's 8.8.8.8. Quad9 also has the added benefit of being security and privacy focused. I used OpenDNS in the past but was turned off when Cisco bought it (personal preference).

Pi-hole is Free and Open Source but please consider making a donation: https://pi-hole.net/donate/

There is a pretty large Pi-hole user base as the project is already a few years old. There are many things to tune if you choose to do so. You can also find curated DNS blacklists out there from the users groups, that might be worth adding a resource.

Personally I'm interested in the log retention of Pi-hole and are there any ways to forward the logs to log collector or database to allow the investigation of DNS queries outside of Pi-hole.

I'll do a follow-up post in a few weeks after I let this run on the network for a while. So check back.

Sunday, December 3, 2017

TIL: as-path prepending

Today I learned: You can prepend any AS numbers in the prepended string.


They typical method of as-path prepending is to prepend or add your autonomous system number to the AS_PATH attribute to influence inbound traffic patterns.

You can technically add any autonomous system to the AS_PATH even AS's that don't belong to you.

NOTE: This is frowned upon in production. "Just because you can doesn't mean you should!"

See the example below:

Without context or a topology this seems a little bland but the results are there. You can see from the BGP table below we have prepended a bunch of AS's that do not belong to us.

Prepeding configured out-bound from R3 --> R1:


R3#sho run | s as-path|route-map|router bgp
router bgp 200

 neighbor 155.1.13.1 remote-as 100
 neighbor 155.1.13.1 route-map AS_254 out

ip as-path access-list 254 permit ^254$

route-map AS_254 permit 10
 match as-path 254
 set as-path prepend 254 250 123

route-map AS_254 permit 20


Showing the R1 partial BGP table:


R1#sho ip bgp neighbors 155.1.13.3 routes

[ ... OUTPUT OMITTED ... ]

     Network          Next Hop            Metric LocPrf Weight Path
 *>  28.119.16.0/24   155.1.13.3                             0 200 54 i
 *>  28.119.17.0/24   155.1.13.3                             0 200 54 i
 *   51.51.51.51/32   155.1.13.3                             0 200 254 250 123 254 ?
 *   205.90.31.0      155.1.13.3                             0 200 254 250 123 254 ?
 *   220.20.3.0       155.1.13.3                             0 200 254 250 123 254 ?
 *   222.22.2.0       155.1.13.3                             0 200 254 250 123 254 ?


Credit: This was influenced by a lab from the INE workbook.

Friday, December 1, 2017

Do you think LogZilla is better than Kiwi?

tl;dr
On LinkedIn, I was asked the question "Do you think LogZilla is better that Kiwi?" and my response(below) was a few thousand characters more than LinkedIn allows in a "comment". See comment here.

Before trying LogZilla I did a quick comparison of a few centralized log management products(LogZilla included). This included research on compatibility, how to videos, usability, ease of install, and also "What do i need it for?" and "how am I going to use it?".

I did like Kiwi for its simplicity. I was happy to see they have a web-interface. I liked their one-time purchase price model. This would be perfect for a small scale install on a budget.

Kiwi is a small product offering from Solarwinds. Solarwinds' product focus is not centralized logging their product focus is compliance/configuration management and performance analytics. Kiwi is not their most profitable business unit.(If I'm wrong... tell me.)

What turned me off about Kiwi was it runs on a windows platform. I don't have spare Windows VM's or licenses lying around and for that reason I had to move on because I didn't have a platform to run it on.

The second product I looked at was LogZilla. Right out of the box, it had additional features and integrations that Kiwi didn't offer. I watched a few of the videos from their YouTube channel and decided I should give this product a try. They do centralized log management and they do it well. This isn't part of a larger suite of products, this is their product. What that means to me is, I don't have to worry about getting an inferior product because its not part of the most profitable business unit within the company instead, it is the business unit of the company.

They offer a free trial download and getting LogZilla installed can be completed with a single command. It can't get any easier right? If you read my blog then you know I decided to use the prebuilt VM which got me up and running in less than 30 minutes. I personally really like the dashboards/widgets and the layout LogZilla has. One thing I really like about it is, you can use it right out of the box or you can customize it to any level the suits you or your businesses needs. Almost everything is customizable. I'm piloting this at my house so I don't need much but, I am exploring building some automation scripts. This product fits my use case at home, and hopefully I can leverage it to fit business cases at work.

One of the last reasons I prefer LogZilla over Kiwi, isn't necessarily a technical or business reason it's more of a human reason. Shortly after getting LogZilla up and running I reached out to their sales department to get my trial period extended. I had a few back and fourths with members of their team and even the CEO reached out to me after seeing my blog post. That was important to me. I got to know them a little bit and understand that they too are a small business. I currently work for a small business, and before this company I worked for an even smaller business. Supporting small business is something I like to do, because I had a small business once and I know what its like. I enjoyed making every customer a personal experience and that's what LogZilla has done for me so far.

Some of the other products that were up for consideration were, ELKSplunk and Nagios Log Server.

Although I don't work with Splunk directly, it's in most environments I work in. I know it as one of the super giants in the industry like ArcSight. Splunk does have a "free" version (with data cap) you can run, but I was a bit intimidated because I associate big names with big complicated systems. So until someone gives me a reason I 'have to run Splunk', it can live at the bottom of my list.

One product that I haven't tried and maybe I'll try it a bit down the road is Nagios Log Server. I didn't even know they had a log management product. I know Nagios from a few years ago, I had to work setup Nagios to monitor availability and performance for some forward facing services and back end services too. Looking into it, looks like it runs ELK in the background. I'm pretty excited about this product. Nagios Log Server you can run with a data cap of 500MB/day.

ELK is the new hip thing in town. It's trendy. Everywhere I work, organizations are standing up ELK stacks. Some big installs, some small installs, some in production, some just for testing, it's everywhere.

To be clear, I'm not doing a bake-off here. I just want to work with great products and push the limits of what I know to learn new things everyday.

Here's a summary if your considering a syslog for your home or business. Give them all a try and find the right product for you.

Hopefully I can continue to offer some valuable feedback from my experiences with the tools I choose to use.

Kiwi offered a fully features system free for 14-days.
LogZilla offers their system free for 7-days.
Splunk  - 500MB/per day
Nagios Log Server - 500MB/per day
ELK  ???

Sunday, November 26, 2017

LogZilla: Day 1

TL;DR - It was China!!!

I wanted to increase visibility of my network at home through the use of a central syslog server. I decided on trying a COTS product instead of rolling my own. I chose LogZilla as my product to try. I had it downloaded, running and receiving its first logs in less than 30 minutes.

I downloaded the *.ova as I already have a small ESXi server running a few VM's with some spare resources.


I setup the VM's hardisk as thin provisioned and gave the VM 2 GB memory. The VM booted but gave an error that it needed a minimum of 4 GB to run LogZilla, I stepped it up to 4 GB and it booted fine. I know the website suggests 8GB but I'm cheap :) Upon initial boot the console asked you log in, and begin the 'first boot sequence'. I assume this is downloading the latest version and updating the VM before launching the LogZilla service.



Once LogZilla is up and running you should go around to your devices and configure the IP address of LogZilla as your remote syslog server or configure your current syslog server to forward events to LogZilla.

Upon first log in to the Web GUI of LogZilla you'll be presented with a Generic Dashboard full of helpful widgets.

The first of which displays the overall statistic of LogZilla's log ingestion. This shows the Events Per Day in max and average. This is helpful to understand what scale license you'll require for your environment.


Another widget shows a pie chart depicting all of the hosts that have sent logs to LogZilla and shows a comparison by volume.


Under the top widgets are 2 pre-configured "Live Stream" widgets in table format. These update in real-time and provide the live stream view. These are very helpful, one table contains all the logs that contain "Error" and the other contains all the logs which contain "Failed". This is great for quick look. I have this widget set to show the logs for the whole day. Because I have a small home network all of the events that contain "Error" or "Failed" in a day isn't very many.

All the widgets are customizable and you can build your own widgets. You can also utilize the provided or custom widgets to create or modify the dashboards. Being the first day I wasn't too interested in making custom dashboards or widgets, I really just wanted to get it working and pumping some logs through.

The bottom widget table is also a "Live Stream" widget. I have it set to show the logs from the 'last minute'.


After having LogZilla running for less than 12 hours I had enough data to start looking into some of the investigative features. I noticed a couple of TLS errors coming from openvpn for a time of day I wasn't connecting to my own VPN.


I selected an event and right-clicked, there are a bunch of helpful context options. 


Using the "Display Geo IP Information" tool I was able to located the source:


From the event I am able to right-click and create a Trigger based on that event.


A trigger allows me to specify some criteria to match in a log message and make an action. This can be sending an email alert or assigning an actionable item to another LogZilla user. For my purposes I simply want a notification and the item to be marked as "Actionable":


The "Name" is arbitrary but should be something that means something to you. When using the "Create Trigger" feature from the main dashboard LogZilla will pre-fill all this information based on the event you chose from the beginning. I edited out the specifics of the "Event match" because I didn't want it to on trigger on the single IP address. I want it to trigger anytime there is a log message that contains the "Event match" phrase from the screenshot.

Now, if LogZilla see's a log containing the message we specified it will create a notification and put the log entry in the "Actionable" widget as configured by my Trigger.



LogZilla has also helped me notice some anomalies that would otherwise gone unnoticed. After the first day of collecting logs I noticed I have nearly 10K log events coming from my router. If I watch the "Live Stream" I see most of them are duplicates. I was seeing this log almost every 5 seconds.

Note: "Host" column removed to better fit in screenshot.

I looked it up and found this reference. An update and changing the log level from the router would make this go away.

So, for my first day of watching logs I have found China trying to connect to my VPN and an anomalous log that shows up every 5 seconds. 

Awareness is King!

This concludes Day 1 of my Thanksgiving holiday. 


Sunday, November 19, 2017

Future = Application Layer Networking

I was having a conversation with a peer about the future of networking. The foundation of the conversation revolved around SDN and the changes that SDN brings to network operators and engineers. The point was raised to me that 'engineers and operators of future networks won't need to have the granular low level understanding of bits, bytes and protocols.' As control of the network becomes more and more software driven the engineer/operator needs only high level understanding. My response to that is: Nothing could be further from the truth! My prediction for the future is: true application layer networking. Have a predictable and deterministic path through the network based on application only.

I think in the future, of application layer routing, we will need to incorporate some level of routing intelligence on each host/end device. I'm not sure exactly what that will look like yet but, I know it is not along the lines of OSPF or EIGRP.

In our current model, for most networks (home networks and small business networks) there is a single egress point where all traffic leaves your LAN to destinations on the internet.

In mid-sized business/enterprise you'll have redundant links a backups. You may have site-to-site tunnels with IPSEC connected remote sites but anything not on your LAN or part of your remote sites still egresses a single point to destinations on the internet.

Large businesses/Enterprises may have multiple egress points to the internet, all managed by lead engineers and operators with oversight from the senior engineers, involving multiple AS's and public IPv4 subnets that span the globe. This is expensive and the bottom line is, even with all the sophistication, the workstations and end devices are still taking the shortest path out of the network based on destination IP address and not on application specific characteristics.

Routers are doing destination based forwarding all over the globe. They are not making routing decisions based on the the type of traffic in the payload of the packet.

One thing I foresee SDN doing for us is bringing dynamic intelligence into routing. Having your controller understand the link requirements of protocols and identify those protocols as they are passing through the routers and forward them based on the application traffic they are carrying not just their destination IP address.

Another thing I believe the future holds for us is true multi-path routing, where end devices, even a common smart phone, can have multiple gateways and not just redundant default gateways, instead they would be application specific gateways. For example I could be connected to my cellular network, wifi and maybe a bunch of ad-hoc networks all at the same time. Perhaps those ad-hoc networks have gateways of their own and we can use them to egress to the internet essentially giving a device like our phone, multiple egress points. Letting our devices participate in the decision making process for routing and forwarding and how to best utilize the links available to it on a per-application basis.

Sorry I went off on a minor futuristic sci-fi routing tangent for a moment.

To bring this full circle, I feel like the engineers and operators of the future will actually need to know more about the inter-workings of each protocol more than just Layer 4. If the future is anything close to application layer networking, we will actually need to be closer to the bits and bytes to understand the protocol of the applications themselves in-order to programmatically and deterministically route them to their destinations.

P.S. - I'm not talking about getting rid of IP addresses but instead introduce more to forwarding than just the destination. I'm sure all the "every packet should be treated equal" people out there are going to have a fit with this.

Comments are welcomed.

Monday, July 31, 2017

Unicast Reverse Path Forwarding (uRPF)

There was a misunderstanding recently with Unicast Reverse Path Forwarding (uRPF). This was an organizational network requirement that had to be met. After the initial configuration changes were applied it brought the network down for that site. Changes needed to be backed out and the configuration changes reviewed again. This is where I come in.
As a consultant I asked to see the requirements document, the draft configuration changes and the post configuration. The problems were 3 fold:
1) the requirements were misunderstood
2) how to apply the configs to meet the requirement was misunderstood
3) the configs were incorrect

The requirement reads:

"The network device must not accept any outbound IP packets that contain an illegitimate address in the source address field by enabling Unicast Reverse Path Forwarding (uRPF) Strict Mode or via egress ACL."
There is a keyword in the requirements which helps to explain the purpose and how to enable uRPF. The keyword is "accept". Generally, anytime you see the word "accept" it indicates this needs to be applied 'inbound' to the router.

Another key element from the requirement is "Strict Mode". There is a difference in how we need to approach this problem if we are using "Strict" or "Loose" modes and how our network is laid out.


This is a security concern because routers by default are doing 'destination-routing' meaning when a router decapsulates a frame it only looks at the destination IP address of the packet, does a route lookup and forwards a new frame. In this simplified explanation you can see the router never inspects the source IP address in the Layer-3 header to verify if it knows the origin of the packet.

This is exactly what Unicast Reverse Path Forwarding(uRPF) does. When a router decapsulates a frame it checks the routing table to see if it has route for the SOURCE IP ADDRESS. If the router does NOT have a route for the source address it drops the packet and does NOT forward it.

What's the big deal? If a router doesn't perform a source IP address lookup what can happen?

In the topology pictured here someone could spoof a source IP address from within our LAN and the upstream routers would still forward the packet to its destination since they are only doing a destination IP lookup. In this scenario after the packet reaches its destination, a reply packet will be sent to the spoofed IP address which will never come back to our LAN Network.

This type of spoofing is used in many DoS and DDoS attacks.




A basic stub topology using R2 as our perimeter edge router. South of R2 is our 2 networks 10.10.10.0/24 and 20.20.20.0/24. R2 has a default route pointed upstream to our public-wan provider.

Inside each LAN Network we have 1 workstation: 10.10.10.254 & 20.20.20.254, from each of these workstations we will be spoofing "222.222.222.222".

Verify reachability from our 'LAN Networks' out to the "Internet" by pinging 30.30.30.1 from each workstation in the LAN Network. Then spoof "222.222.222.222" from the workstation in the Vlan 20 segment.

Below are 2 screenshots of Wireshark PCAPs. I have captured from in-between the LAN Network & R2, and R2 & the "Internet".


Verification Test:

PCAP #1
PCAP #2

You can see from the above verification test, the pings from the workstations were successfully routed and they received a reply. The spoofed "222.222.222.222" was also successfully routed but, as expected, it did not receive a reply.

To meet our requirement we must enable uRPF in "Strict Mode". You should consult the command reference for your OS version.

Below we are only applying uRPF Strict Mode to the sub-interface 10. This is so we can see side-by-side the effect of uRPF.

Apply uPRF:


R2(config-if)#int fa1/0.10

R2(config-subif)#ip verify ?
  unicast  Enable per packet validation for unicast

R2(config-subif)#ip verify unicast ?
  notification  drop-rate notify
  reverse-path  Reverse path validation of source address (old command format)
  source        Validation of source address

R2(config-subif)#ip verify unicast source ?
  reachable-via  Specify reachability check to apply to the source address

R2(config-subif)#ip verify unicast source reachable-via ?
  any  Source is reachable via any interface
  rx   Source is reachable via interface on which packet was received

R2(config-subif)#ip verify unicast source reachable-via rx ?
  <1-199>          IP access list (standard or extended)
  <1300-2699>      IP expanded access list (standard or extended)
  allow-default    Allow default route to match when checking source address
  allow-self-ping  Allow router to ping itself (opens vulnerability in
                   verification)
  l2-src           Check packets arrive with correct L2 source address
  <cr>

R2(config-subif)#ip verify unicast source reachable-via rx


Now that the configuration changes have been applied let's test.

Ping from Workstation in Vlan 10:


WRKSTN#ping 30.30.30.1 source 10.10.10.254 repeat 1
Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 30.30.30.1, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.254
!
Success rate is 100 percent (1/1), round-trip min/avg/max = 80/80/80 ms

WRKSTN#ping 30.30.30.1 source 222.222.222.222 repeat 1
Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 30.30.30.1, timeout is 2 seconds:
Packet sent with a source address of 222.222.222.222
.
Success rate is 0 percent (0/1)


You can see from the above CLI output our pings sourcing from our workstation IP's are still successful. This is because R2(our perimeter edge) has a route in its routing table for the source 10.10.10.254 and it forwards the packet on-ward. The ping's sourced from 222.222.222.222 are not successful because R2 does NOT have a route for 222.222.222.222 so it does NOT forward the packet.

This can be seen in the PCAP screenshots below.



Below are 2 screenshots of Wireshark PCAPs. I have captured from in-between the LAN Network and R2 (PCAP #1), and R2 and the "Internet"(PCAP #2). You can see the packet is forwarded to R2 but R2 never forwards the packet upstream. This is because upon receiving the packet sourced from 222.222.222.222, R2 does a route lookup and see's that it DOES NOT have a route for the source address and drops the packet.

PCAP #1

PCAP #2
In the above example we are using the 10.10.10.0/24(dot1q: 10) sub-interface between our LAN Network and R2.

Remember, we only applied uRPF to the 10.10.10.0/24(dot1q: 10) sub-interface on R2. What do you think will happen when R2 receives a spoofed packet sourced from 222.222.222.222 on the 20.20.20.0/24 (dot1q: 20) sub-interface?


R3#ping 30.30.30.1 source 20.20.20.254 repeat 1
Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 30.30.30.1, timeout is 2 seconds:
Packet sent with a source address of 20.20.20.254
!
Success rate is 100 percent (1/1), round-trip min/avg/max = 20/20/20 ms

R3#ping 30.30.30.1 source 222.222.222.222 repeat 1
Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 30.30.30.1, timeout is 2 seconds:
Packet sent with a source address of 222.222.222.222
.
Success rate is 0 percent (0/1)


You can see from the above CLI output the pings sourced from our out-bound interface 20.20.20.254 were successful pings and the pings sourced from 222.222.222.222 were not. This is to be expected because the upstream device doesn't actually have a route back to 222.222.222.222. Let check the PCAP's and see if the router is actually forwarding those echo-requests.

PCAP #1

PCAP #2
You can see the spoofed IP echo-request gets forwarded through R2 upstream to the Internet router.


Now if we apply the same config we did from earlier and perform the pings again we should NOT see them forwarded up stream to the Internet.


R2(config)#int fa1/0.20

R2(config-subif)#ip verify ?
  unicast  Enable per packet validation for unicast


R2(config-subif)#ip verify unicast ?
  notification  drop-rate notify
  reverse-path  Reverse path validation of source address (old command format)
  source        Validation of source address


R2(config-subif)#ip verify unicast source ?
  reachable-via  Specify reachability check to apply to the source address

R2(config-subif)#ip verify unicast source reachable-via ?
  any  Source is reachable via any interface
  rx   Source is reachable via interface on which packet was received

R2(config-subif)#ip verify unicast source reachable-via rx ?
  <1-199>          IP access list (standard or extended)
  <1300-2699>      IP expanded access list (standard or extended)
  allow-default    Allow default route to match when checking source address
  allow-self-ping  Allow router to ping itself (opens vulnerability in
                   verification)
  l2-src           Check packets arrive with correct L2 source address
  <cr>

R2(config-subif)#ip verify unicast source reachable-via rx
R2(config-subif)#


With the configuration now applied to the .20 sub-interface we can now test again and look to the PCAPs for verification.

PCAP #1

PCAP #2

Verify:

Feature Enabled?

R2#sho cef interface fastEthernet 1/0.10
FastEthernet1/0.10 is up (if_number 10)
  Corresponding hwidb fast_if_number 3
  Corresponding hwidb firstsw->if_number 3
  Internet address is 10.10.10.1/24
  ICMP redirects are always sent
  Per packet load-sharing is disabled
  IP unicast RPF check is enabled
  Input features: uRPF
  IP policy routing is disabled
  BGP based policy accounting on input is disabled
  BGP based policy accounting on output is disabled
  Hardware idb is FastEthernet1/0
  Fast switching type 1, interface type 18
  IP CEF switching enabled
  IP CEF switching turbo vector
  IP CEF turbo switching turbo vector
  IP prefix lookup IPv4 mtrie 8-8-8-8 optimized
  Input fast flags 0x4000, Output fast flags 0x0
  ifindex 3(3)
  Slot  Slot unit 0 VC -1
  IP MTU 1500

R2#sho cef interface fastEthernet 1/0.20
FastEthernet1/0.20 is up (if_number 11)
  Corresponding hwidb fast_if_number 3
  Corresponding hwidb firstsw->if_number 3
  Internet address is 20.20.20.1/24
  ICMP redirects are always sent
  Per packet load-sharing is disabled
  IP unicast RPF check is enabled
  Input features: uRPF
  IP policy routing is disabled
  BGP based policy accounting on input is disabled
  BGP based policy accounting on output is disabled
  Hardware idb is FastEthernet1/0
  Fast switching type 1, interface type 18
  IP CEF switching enabled
  IP CEF switching turbo vector
  IP CEF turbo switching turbo vector
  IP prefix lookup IPv4 mtrie 8-8-8-8 optimized
  Input fast flags 0x4000, Output fast flags 0x0
  ifindex 3(3)
  Slot  Slot unit 0 VC -1
  IP MTU 1500


Verify Configuration:

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#int fa1/0.10
R2(config-subif)#do sho run int fa1/0.10
Building configuration...

Current configuration : 140 bytes
!
interface FastEthernet1/0.10
 encapsulation dot1Q 10
 ip address 10.10.10.1 255.255.255.0
 ip verify unicast source reachable-via rx
end

R2(config-subif)#no ip verify unicast source reachable-via rx
R2(config-subif)#do sho cef inter fa1/0.10
FastEthernet1/0.10 is up (if_number 10)
  Corresponding hwidb fast_if_number 3
  Corresponding hwidb firstsw->if_number 3
  Internet address is 10.10.10.1/24
  ICMP redirects are always sent
  Per packet load-sharing is disabled
  IP unicast RPF check is disabled
  IP policy routing is disabled
  BGP based policy accounting on input is disabled
  BGP based policy accounting on output is disabled
  Hardware idb is FastEthernet1/0
  Fast switching type 1, interface type 18
  IP CEF switching enabled
  IP CEF switching turbo vector
  IP CEF turbo switching turbo vector
  IP prefix lookup IPv4 mtrie 8-8-8-8 optimized
  Input fast flags 0x4000, Output fast flags 0x0
  ifindex 3(3)
  Slot  Slot unit 0 VC -1
  IP MTU 1500


Show uRPF DROP statistics:

R2#sho ip traffic
IP statistics:
  Rcvd:  0 total, 0 local destination
         0 format errors, 0 checksum errors, 0 bad hop count
         0 unknown protocol, 0 not a gateway
         0 security failures, 0 bad options, 0 with options
  Opts:  0 end, 0 nop, 0 basic security, 0 loose source route
         0 timestamp, 0 extended security, 0 record route
         0 stream ID, 0 strict source route, 0 alert, 0 cipso, 0 ump
         0 other
  Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble
         0 fragmented, 0 fragments, 0 couldn't fragment
  Bcast: 0 received, 0 sent
  Mcast: 0 received, 0 sent
  Sent:  0 generated, 10 forwarded
  Drop:  0 encapsulation failed, 0 unresolved, 0 no adjacency
         0 no route, 5 unicast RPF, 0 forced drop, 0 unsupported-addr
         0 options denied, 0 source IP address zero

[ ... OUTPUT OMITTED ... }


R2#  show ip interface fa1/0.20
FastEthernet1/0.20 is up, line protocol is up
  Internet address is 20.20.20.1/24
  Broadcast address is 255.255.255.255
  Address determined by non-volatile memory
  MTU is 1500 bytes
  Helper address is not set
  Directed broadcast forwarding is disabled
  Outgoing access list is not set
  Inbound  access list is not set
  Proxy ARP is enabled
  Local Proxy ARP is disabled
  Security level is default
  Split horizon is enabled
  ICMP redirects are always sent
  ICMP unreachables are always sent
  ICMP mask replies are never sent
  IP fast switching is enabled
  IP Flow switching is disabled
  IP CEF switching is enabled
  IP CEF switching turbo vector
  IP CEF turbo switching turbo vector
  Associated unicast routing topologies:
        Topology "base", operation state is UP
  IP multicast fast switching is enabled
  IP multicast distributed fast switching is disabled
  IP route-cache flags are Fast, CEF
  Router Discovery is disabled
  IP output packet accounting is disabled
  IP access violation accounting is disabled
  TCP/IP header compression is disabled
  RTP/IP header compression is disabled
  Probe proxy name replies are disabled
  Policy routing is disabled
  Network address translation is disabled
  BGP Policy Mapping is disabled
  Input features: uRPF, MCI Check
  WCCP Redirect outbound is disabled
  WCCP Redirect inbound is disabled
  WCCP Redirect exclude is disabled
  IP verify source reachable-via RX
   5 verification drops
   0 suppressed verification drops
   0 verification drop-rate

Monday, March 27, 2017

Multi-VRF EIGRP with Authentication

Tasks

  1. Establish EIGRP adjacencies using both Named Mode and traditional EIGRP
  2. Each EIGRP Autonomous System must be in it's own VRF on the Hub Router (R2)
  3. Use authentication between a pair of neighbors
  4. BONUS: Leak routes from 1 VRF to another
Using the topology below configure the tasks from above.

Notes:

R1

  • Traditional EIGRP configuration
  • NO VRF's

R2

  • Named EIGRP configuration
  • 2 VRF's
  • Authentication to R3
  • BONUS: Leak routes from 1 VRF to another

R3

  • Traditional EIGRP configuration
  • NO VRF's
  • Authentication to R2

Approx. Lab Time: 10 - 15 Minutes


Configs - Phase 1: Interfaces and Loopbacks

R1:

interface Loopback1
 description "Loopback for EIGRP advertisments"
 ip address 1.1.1.1 255.255.255.255

interface FastEthernet0/0
 description "Connection to R2"
 ip address 10.0.12.1 255.255.255.0
no shut

R2:

! Establish VRF's
! R1 -> R2 = VRF 12
! R2 -> R3 = VRF 23

ip vrf 12
 rd 12:12

ip vrf 23
 rd 23:23


! Configure interfaces & Loopbacks

interface Loopback12
 description "Loopback for EIGRP advertisement"
 ip vrf forwarding 12
 ip address 12.12.12.12 255.255.255.255

interface Loopback23
 description "Loopback for EIGRP advertisement"
 ip vrf forwarding 23
 ip address 23.23.23.23 255.255.255.255

interface FastEthernet0/0
 description "Connection to R1"
 ip vrf forwarding 12
 ip address 10.0.12.2 255.255.255.0
no shut

interface FastEthernet1/0
 description "Connection to R2"
 ip vrf forwarding 23
 ip address 10.0.23.2 255.255.255.0
no shut

R3:

interface Loopback3
 description "Loopback for EIGRP advertisement"
 ip address 3.3.3.3 255.255.255.255


interface FastEthernet1/0
 description "Connection to R2"
 ip address 10.0.23.3 255.255.255.0
no shut

Verify:

R1:

R1#ping 10.0.12.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.12.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 24/31/40 ms

R2:

R2#ping vrf 12 10.0.12.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.12.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/28/40 ms

R2#ping vrf 23 10.0.23.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.23.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/25/28 ms

R3:

R3#ping 10.0.23.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.23.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 24/31/40 ms



Configs - Phase 2: EIGRP 


EIGRP between R1 --> R2 = AS 12
EIGRP between R2 --> R3 = AS 23

R1: Traditional EIGRP Configuration

router eigrp 12
 network 0.0.0.0

R2: EIGRP Named Configuration

router eigrp central
 !
 address-family ipv4 unicast vrf 12 autonomous-system 12
  !
  topology base
  exit-af-topology
  network 0.0.0.0
 exit-address-family
 !
 address-family ipv4 unicast vrf 23 autonomous-system 23
  !
  topology base
  exit-af-topology
  network 0.0.0.0
 exit-address-family
!

R3: Traditional EIGRP Configuration

router eigrp 23
 network 0.0.0.0

Verify:

R1:

R1#sho ip eigrp neighbors
EIGRP-IPv4 Neighbors for AS(12)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
0   10.0.12.2               Fa0/0                    10 01:14:52   56   336  0  3

R1#sho ip eigrp topology
EIGRP-IPv4 Topology Table for AS(12)/ID(1.1.1.1)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status

P 12.12.12.12/32, 1 successors, FD is 28160
        via 10.0.12.2 (28160/256), FastEthernet0/0
P 1.1.1.1/32, 1 successors, FD is 128256
        via Connected, Loopback1
P 10.0.12.0/24, 1 successors, FD is 28160
        via Connected, FastEthernet0/0

R1#ping 12.12.12.12 so lo 1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 12.12.12.12, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/27/40 ms

R2:

R2#sho eigrp address-family ipv4 vrf 12 neighbors
EIGRP-IPv4 VR(central) Address-Family Neighbors for AS(12)
           VRF(12)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
0   10.0.12.1               Fa0/0                    13 01:07:53 1406  5000  0  3

R2#sho eigrp address-family ipv4 vrf 12 topology
EIGRP-IPv4 VR(central) Topology Table for AS(12)/ID(12.12.12.12)
           Topology(base) TID(0) VRF(12)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status

P 12.12.12.12/32, 1 successors, FD is 163840
        via Connected, Loopback12
P 1.1.1.1/32, 1 successors, FD is 340787200
        via 10.0.12.1 (340787200/327761920), FastEthernet0/0
P 10.0.12.0/24, 1 successors, FD is 13107200
        via Connected, FastEthernet0/0

R2#sho eigrp address-family ipv4 vrf 23 neighbors
EIGRP-IPv4 VR(central) Address-Family Neighbors for AS(23)
           VRF(23)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
0   10.0.23.3               Fa1/0                    14 01:08:07 1379  5000  0  3

R2#sho eigrp address-family ipv4 vrf 23 topology
EIGRP-IPv4 VR(central) Topology Table for AS(23)/ID(23.23.23.23)
           Topology(base) TID(0) VRF(23)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status

P 23.23.23.23/32, 1 successors, FD is 163840
        via Connected, Loopback23
P 10.0.23.0/24, 1 successors, FD is 13107200
        via Connected, FastEthernet1/0
P 3.3.3.3/32, 1 successors, FD is 340787200
        via 10.0.23.3 (340787200/327761920), FastEthernet1/0

R2#ping vrf 12 1.1.1.1 so lo 12
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
Packet sent with a source address of 12.12.12.12
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/28/36 ms

R2#ping vrf 23 3.3.3.3 so lo 23
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
Packet sent with a source address of 23.23.23.23
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/28/36 ms

R3:

R3#sho ip eigrp neighbors
EIGRP-IPv4 Neighbors for AS(23)
H   Address                 Interface              Hold Uptime   SRTT   RTO  Q  Seq
                                                   (sec)         (ms)       Cnt Num
0   10.0.23.2               Fa1/0                    13 01:12:58   47   282  0  3

R3#sho ip eigrp topology
EIGRP-IPv4 Topology Table for AS(23)/ID(3.3.3.3)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status

P 23.23.23.23/32, 1 successors, FD is 28160
        via 10.0.23.2 (28160/256), FastEthernet1/0
P 10.0.23.0/24, 1 successors, FD is 28160
        via Connected, FastEthernet1/0
P 3.3.3.3/32, 1 successors, FD is 128256
        via Connected, Loopback3

R3#ping 23.23.23.23 so lo 3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 23.23.23.23, timeout is 2 seconds:
Packet sent with a source address of 3.3.3.3
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/25/32 ms


Configs - Phase 3: EIGRP Authentication


NOTE: When applying the authentication configs your neighbors WILL bounce.

R2: Named EIGRP Address Family Authentication

! Establish a key chain and shared key string
key chain to-r3
 key 23
   key-string p@ssw0rd

! EIGRP Authentication commands go under the address-family NOT the interface!!!

router eigrp central
 !
 address-family ipv4 unicast vrf 23 autonomous-system 23
  !
  af-interface FastEthernet1/0
   authentication mode md5
   authentication key-chain to-r3
  exit-af-interface
 !
 exit-address-family

R3: Traditional EIGRP Authentication

! Establish and key chain and shared key string
key chain to-r2
 key 23
   key-string p@ssw0rd


! EIGRP authentication commands go under the interface config

interface FastEthernet1/0
 description "Connection to R2"
 ip address 10.0.23.3 255.255.255.0
no shut
 ip authentication mode eigrp 23 md5
 ip authentication key-chain eigrp 23 to-r2

Verify:

R2 & R3:

  1. Use the same commands and technique from above to verify your adjacency is still up.
  2. Check your Key Strings:

R3#sho key chain to-r2
Key-chain to-central:
    key 23 -- text "p@ssw0rd"
        accept lifetime (always valid) - (always valid) [valid now]
        send lifetime (always valid) - (always valid) [valid now]
  1. debug to verify authentication
Successful Authentication:

R3#debug eigrp packets

*Mar 21 12:07:33.367: EIGRP: received packet with MD5 authentication, key id = 23
*Mar 21 12:07:33.367: EIGRP: Received HELLO on Fa1/0 - paklen 60 nbr 10.0.23.2

Password Mis-match:

R3#debug eigrp packets

*Mar 21 12:10:42.363: EIGRP: pkt key id = 23, authentication mismatch
*Mar 21 12:10:42.367: EIGRP: Fa1/0: ignored packet from 10.0.23.2, opcode = 5 (invalid authentication)

Mis-configured Authentication: (1 side configured, the other side NOT configured)

R3#debug eigrp packets

*Mar 21 12:16:30.263: EIGRP: Fa1/0: ignored packet from 10.0.23.2, opcode = 5 (missing authentication)


BONUS: Leak EIGRP routes from 1 VRF to another


TASK: Ping from R1 to R3

We must:

  • Configure import and export route-targets for both VRF's
  • Configure BGP and redistribute EIGRP into BGP
  • Redistribute BGP back into EIGRP

Configs - Phase 1: Route-targets and Redistribution


R2:

! Configure import and export route-targets for both VRF's

ip vrf 12
route-target import 23:23
route-target export 12:12
!
ip vrf 23
route-target import 12:12
route-target export 23:23

exit


! Configure BGP and redistribute EIGRP into BGP

router bgp 65535

address-family ipv4 vrf 12
redistribute eigrp 12
exit

address-family ipv4 vrf 23
redistribute eigrp 23
exit

Verify - Phase 1:


R2:

R2#sho bgp vpnv4 unicast all

BGP table version is 37, local router ID is 123.123.123.123
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 12:12 (default for vrf 12)
 *>  1.1.1.1/32       10.0.12.1          2662400         32768 ?
 *>  3.3.3.3/32       10.0.23.3          2662400         32768 ?
 *>  10.0.12.0/24     0.0.0.0                  0         32768 ?
 *>  10.0.23.0/24     0.0.0.0                  0         32768 ?
 *>  12.12.12.12/32   0.0.0.0                  0         32768 ?
 *>  23.23.23.23/32   0.0.0.0                  0         32768 ?
Route Distinguisher: 23:23 (default for vrf 23)
 *>  1.1.1.1/32       10.0.12.1          2662400         32768 ?
 *>  3.3.3.3/32       10.0.23.3          2662400         32768 ?
 *>  10.0.12.0/24     0.0.0.0                  0         32768 ?
 *>  10.0.23.0/24     0.0.0.0                  0         32768 ?
 *>  12.12.12.12/32   0.0.0.0                  0         32768 ?
 *>  23.23.23.23/32   0.0.0.0                  0         32768 ?
R2#


At this point the routes for both VRF's are in both VRF's local routing tables. We still cannot ping from R1 to R3 because they aren't aware of each other's networks. The routes from R2 are not being propagated out to R1 & R3. We need to redistribute the routes from BGP-->EIGRP.


Configs - Phase 2: Redistribute BGP into EIGRP


R2:

router eigrp central
 address-family ipv4 unicast vrf 12 autonomous-system 12
  topology base
   redistribute bgp 65535 metric 1000 100 255 1 1500
  exit
 exit
 address-family ipv4 unicast vrf 23 autonomous-system 23
  topology base
   redistribute bgp 65535 metric 1000 100 255 1 1500
  exit
 exit

Verify - Phase 2:


R1:

R1#sho ip route | b Gate
Gateway of last resort is not set

      1.0.0.0/32 is subnetted, 1 subnets
C        1.1.1.1 is directly connected, Loopback1
      3.0.0.0/32 is subnetted, 1 subnets
D EX     3.3.3.3 [170/2588160] via 10.0.12.2, 00:08:52, FastEthernet0/0
      10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C        10.0.12.0/24 is directly connected, FastEthernet0/0
L        10.0.12.1/32 is directly connected, FastEthernet0/0
D EX     10.0.23.0/24 [170/2588160] via 10.0.12.2, 00:08:52, FastEthernet0/0
      12.0.0.0/32 is subnetted, 1 subnets
D        12.12.12.12 [90/28160] via 10.0.12.2, 01:45:05, FastEthernet0/0
      23.0.0.0/32 is subnetted, 1 subnets
D EX     23.23.23.23 [170/2588160] via 10.0.12.2, 00:08:52, FastEthernet0/0

R1#ping 3.3.3.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 52/60/72 ms

R3:

R3#sho ip route | b Gate

Gateway of last resort is not set

      1.0.0.0/32 is subnetted, 1 subnets
D EX     1.1.1.1 [170/2588160] via 10.0.23.2, 00:00:10, FastEthernet1/0
      3.0.0.0/32 is subnetted, 1 subnets
C        3.3.3.3 is directly connected, Loopback3
      10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
D EX     10.0.12.0/24 [170/2588160] via 10.0.23.2, 00:00:10, FastEthernet1/0
C        10.0.23.0/24 is directly connected, FastEthernet1/0
L        10.0.23.3/32 is directly connected, FastEthernet1/0
      12.0.0.0/32 is subnetted, 1 subnets
D EX     12.12.12.12 [170/2588160] via 10.0.23.2, 00:00:10, FastEthernet1/0
      23.0.0.0/32 is subnetted, 1 subnets
D        23.23.23.23 [90/28160] via 10.0.23.2, 01:37:00, FastEthernet1/0

R3#ping 1.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 60/71/104 ms