Network Field Day 22 - DriveNets

DriveNets gave a great presentation. As stated in previous blog posts, this was my first NFD and DriveNets was the first presentation for the entire 3-days. These guys really set the bar high and lead the way with a lot of great technical content. DriveNets found a way to create a horizontally scalable routing platform by completely disaggregating the software from the hardware. While there are a few common buzzwords in there that are getting a lot of air-time these days, what these guys are doing is very different from other vendors.

If you want to learn more about containers, orchestration, distributed systems and whitebox hardware go watch the DriveNets presentations from Network Field Day 22.

Traditional Routing platforms are appliances or chassis based. You can’t just add more ports to an appliance. You can only grow a chassis to the full-capacity of its line cards.

Scalable Operations

Horizontally scalable means to add nodes (or remove nodes) as you want to increase (or decrease) your scale. This goes hand-in-hand is distributed systems. One of the best ways to build distributed systems is to separate the application from the operating system by way of containers. Containers make applications portable and can then live on any compute resource. Understanding the maintenance and management of containers is important for this model. Each container should contain a single process. For example, you would have a separate container for each protocol you’re running. This is important because if you aren’t running a particular protocol you don’t have to deploy that container. Also, in looking down the road toward upgrades, you typically don’t ‘upgrade’ and container, instead you redeploy the container with the updated application code inside.

Disaggregated Hardware and Software

DriveNets are using standardized whitebox hardware running a flavor of Linux (Ubuntu) in order install Docker and run their containers. If you standardize the hardware, you can streamline the base OS installation and maintenance leaving only an orchestration layer to spin up and distribute the containers onto each nodes as necessary. DriveNets has their own orchestrator called DNOR (DriveNets Orchestrator). This all creates a model that each node can be thought of like Lego block. If you want to add more compute and more ports you add another Lego block. Compared to a typical hierarchical network model of Edge-Aggregate-and-Core this all gets distilled into a single platform.

This architecture really hit close to home because my company and some of my direct work has been engineering horizontally scalable distributed systems for Defensive Cyber Operations (DCO) purposes. I understand engineering efforts behind this design. What DriveNets has accomplished is very cool.

Who's Using It?

Now, most businesses or enterprises don’t need horizontally scalable networking. This seems to be a great fit for service providers and cloud scalers where growth is rapid and seemingly never ending. There are cost savings and operational benefits to doing it this way. Say goodbye to the days of changing out chassis to increase capacity, in this model of building networks modularly like DriveNets, you can add resources as you need in the form of adding nodes. All of the work that DriveNets is doing in software development, ensures this process of scaling up or adding nodes happens seamlessly and that your network can continue to grow.

I really enjoyed their presentation.

If you want to learn more about containers, orchestration, distributed systems and whitebox hardware go watch the DriveNets presentations from Network Field Day 22.

For the full list of presenters, all the videos, links to all the delegates and more check out the NFD22 website: https://techfieldday.com/event/nfd22/

Disclaimer: I was invited to Networking Field Day 22 with GestaltIT covering travel and accommodation. There is no requirement to blog, promote, or produce any content from the event. This post is my opinion and my opinion alone.

Network Field Day 22 - #NFD22

Network Field Day (NFD) was an absolutely magical time. This was my first NFD and I didn’t know what to expect. Upon meeting everyone at a kickoff dinner the night before the presentations it became clear to me, most of these people were veterans of NFD and some of the most influential people in the networking space. I was definitely the outsider looking in, I found myself questioning why am I even here? I’m not equal to these people. What can I bring to the table that these brilliant minds haven’t already thought? This is where the nervousness of meeting new people and imposter syndrome started setting in.

This is for you @fryguy_pa

Although I felt like an outsider in the beginning, everyone welcomed myself and the couple other new delegates with open arms. Before dinner was over, I felt very welcomed and accepted as a peer. The next 4-days just felt like old friends hanging out. It was like a 4-day nerdy summer camp, where you get to participate in cool activities all week, make new friends, enjoy each other’s company and learn from each other and the vendors we got to meet.

Tom & Ben

First, I want to talk about the Network Field Day team. Tom Hollingsworth and Ben Gage were boots on the ground and with us the entire time. They shepherded us along throughout the 4-days we were together. They have this down to a science and operated like a well-oiled machine. Also, thanks to all the back-office support keeping Tom and Ben in line and helping with all the scheduling, confirmation emails and social media coordination.

What is Network Field Day from a delegate’s perspective?

NFD is an exclusive invite only event when 12-delegates get to meet with various companies and presenters to learn more about new products, new software, recent acquisitions and more. This is typically and engineer to engineer style discussion, the sales and fluff is kept to a minimum. This helps us as engineers, operators, integrators and administrators to make educated informed decisions for our companies, our customers and our partners. Being able to meet the presenters in person, ask questions and get answers face-to-face goes along way.

Being part of NFD you’re really on the front lines of what’s happening in the industry. When Arista purchased Big Switch they officially made the announcement to us and the people joining remotely via live stream. We got to hear from Avi Networks as part of the VMware presentation because of their recent acquisition. And about 2 months after we visited CloudGenix they were acquired by Palo-Alto Networks.

All of the content is live streamed as it’s happening and all the video feeds are captured edited and posted to YouTube, Vimeo and the Tech Field Day website.

For the full list of presenters, all the videos, links to all the delegates and more check out the #NFD22 website: https://techfieldday.com/event/nfd22/

My Top Picks from #NFD22:

• Most Innovative: DriveNets (My Blog Post)
• Most Needed (and Entertaining): Forward Networks
• Most Educational: Broadcom
• Back to Basics: RiverBed

DriveNets

I really enjoyed the DriveNets presentation I think what they are doing is really innovative in our field. Disaggregating the software from the hardware, using white box hardware, creating a horizontally scalable platform through use of containers and their own orchestrator. There's no doubt lots of engineering when into this product and I'm excited to see where this goes next. You can read more on my blog post here, or on the Network Field Day 22 website to watch all their videos.

Forward Networks

Their presentations were packed with Grumpy Cat (R.I.P.) Memes, Seinfeld references, and more. It was memorable and entertaining. Through a mock work-week the presenters showed 5 different use cases for their product. When you see their product and videos there's no question why they are successful. You should go watch the videos right meow! 

Broadcom

I learned alot about the chips and the silicon that is in the equipment we touch everyday. I learned about the difference in the chipset families and why we have different architectures and their benefits. If you want to be educated on the reasons Broadcom makes different chipset families go watch the videos. Jeff Fry wrote an excellent post about their NFD22 presentation at: https://fryguy.net/2020/05/06/broadcom-nfd22/

RiverBed

Romain Jourdan from Riverbed gets the "Back to Basics" award for giving a good ol’fashioned whiteboard session about BDP (Bandwidth Delay Product). I’m not gonna lie, it definitely had me checking my TCP facts/knowledge at the door. Great Job Romain!

When Attendees Become the Content

It was really funny to see tweets from former and current NFD delegates in the presentations of the vendors. RiverBed had a tweet from Phil Gervasi (seen in this video) and Kentik had a tweet from Kevin Myers (seen in this video). Being an NFD delegate you are sometimes unknowingly part of the presentation.

So Long, and Thanks for All the Fish

Months after attending Networking Field Day and as I was putting together this series of posts, I found something that reminded me of Network Field Day in my travel pack. This is a banker’s pouch that was given to each attendee. This small pouch of goodies is a well thought out collection of most requested items. This is a testament to how prepared the TechFieldDay crew is to most efficiently cater the needs of the delegate throughout the experience. Contained in the pouch were: Advil, Apsrin, Pepto, Emergen-C, Shout Wipe & Go, throat lozenges, jolly ranchers, TechFieldDay stickers and much more. Today, this pouch stays in my travel pack and goes with me where ever I go. It’s perfectly convenient.

Disclaimer: I was invited to Networking Field Day 22 with GestaltIT covering travel and accommodation. There is no requirement to blog, promote, or produce any content from the event. This post is my opinion and my opinion alone.

I'll show you mine if you show me your's, investigating OSPF's "B" bit

My dedication to delivering accurate content is what has postponed this post for more than 2 months. Every time I set out to document this behavior I'm double checking everything and always learning something new. Finally I have enough stable material to present...

[1940's Victory at Sea announcer voice...]

Investigating OSPF's "B" bit! The case of "I'll show you mine if you show me your's".

As I study for my CCIE R/S, I was working a lab recently and found myself confused by a lab task. This caused me to spend 2-days deep diving the technology through repetitive labbing and reading RFC’s to understand what was going on and to answer the question: "How can you exchange prefixes accross multiple areas in an OSPF domain when there is not an Area 0?"

Most seasoned engineers would utilize a virtual-link to extend area 0 across normal areas, but what if there is not area 0 to begin with?

Let’s use a simple topology(the control):

The Control Topology

R1: int lo1111 ip add 1.1.1.1 255.255.255.255 ip ospf 1 area 0 int eth0/0 ip add 10.1.12.1 255.255.255.0 ip ospf 1 area 12 router ospf 1 area 12 virtual-link 2.2.2.2

R2: int lo2222 ip add 2.2.2.2 255.255.255.255 ip ospf 1 area 2 int eth0/0 ip add 10.1.12.2 255.255.255.0 ip ospf 1 area 12 router ospf 1 area 12 virtual-link 1.1.1.1

In this topology you would build a virtual-link between Routers 1 & 2 across area 12 to extend area 0, so that R2 can share the networks of area 2 with the rest of the OSPF domain.

Now, lets change area 0 to area 1.

• How can we establish full connectivity?
• Create a virtual-link…?

Try it.

R1: int lo1111 ip add 1.1.1.1 255.255.255.255 ip ospf 1 area 1 int eth0/0 ip add 10.1.12.1 255.255.255.0 ip ospf 1 area 12 router ospf 1 area 12 virtual-link 2.2.2.2

R2: int lo2222 ip add 2.2.2.2 255.255.255.255 ip ospf 1 area 2 int eth0/0 ip add 10.1.12.2 255.255.255.0 ip ospf 1 area 12 router ospf 1 area 12 virtual-link 1.1.1.1

The Test Topology (no area 0)

You’ve configured a virtual-link but it won’t come up? Why not?

You can troubleshoot and run debugs ‘debug ip ospf adj’ but you won’t get any reason why this isn’t working.

When you’re completely lost and have nowhere else to turn the last place you look is the RFC. (I’m still trying to train myself to look at RFCs earlier on in the troubleshooting process)

From RFC2328 Section “15. Virtual Links”

...Virtual links serve to connect physically separate components of the backbone. The two endpoints of a virtual link are area border routers. The virtual link must be configured in both routers. The configuration information in each router consists of the other virtual endpoint (the other area border router), and the non-backbone area the two routers have in common (called the Transit area). Virtual links cannot be configured through stub areas (see Section 3.6).

So, the question is, how do 2 remote routers know if they are “area border routers”? In most Cisco documentation an “area border router” is defined as having at-least one interface in area 0 and other interfaces in one or more areas. How does the RFC suggest the routers identify as Area Border Routers?

In RFC2328, Section “12.4.1. Router-LSAs”

A router also indicates whether it is an area border router, or an AS boundary router, by setting the appropriate bits (bit B and bit E, respectively) in its router-LSAs. This enables paths to those types of routers to be saved in the routing table, for later processing of summary-LSAs and AS-external-LSAs. Bit B should be set whenever the router is actively attached to two or more areas, even if the router is not currently attached to the OSPF backbone area. Bit E should never be set in a router-LSA for a stub area (stub areas cannot contain AS boundary routers).

Ok, now we have something we can check. Let perform an experiment against a control and grab a PCAP between the 2 routers to see if the “B” bit is set coming from either direction in the Type-1 Router LSA's.

First let’s observe our first topology(the control) with an area 0, that does NOT YET have virtual-links configured.

The Control Topology

In the Type-1 Router LAS's, you see the “B” bit is set always coming from R1 first, then R2 will respond:

WireShark Filter: ospf.v2.router.lsa.flags.b == 0 || ospf.v2.router.lsa.flags.b == 1

It seems as if R1 (the router with area 0 configured) has to first tell the adjacent routers that it is an area border router before R2 will respond saying it too is an area border routers. This is just like the "I'll show you mine if you show me yours" game. R2 is being a little shy about exposing it's "B" bit until R1 does first.

R1: router ospf 1 area 12 virtual-link 2.2.2.2

R2: router ospf 1 area 12 virtual-link 1.1.1.1

Now, configure the virtual-links like you normally would and observe the PCAP again. After the virtual-links are fully adjacent, you will see both R1 and R2 are now setting the “B” and “V” bits. Which makes sense now that both routers have an interface in area 0, albeit because of the virtual-link itself. The "V" bit means that the source is a virtual-link end-points, and now that the virtual-link is fully adjacent they are both exchanging Type-1 Router LSA's with the "B" & "V" bit's set.

Moving over to the 'test' topology that DOES NOT have an area 0 or virtual-links configured, observe a PCAP while the routers form an adjacency. You'll see neither R1 or R2 have their “B” set.

The Test Topology (no area 0)

But wait I thought the RFC said:

Bit B should be set whenever the router is actively attached to two or more areas, even if the router is not currently attached to the OSPF backbone area.

This is exactly what we have; a router, with interfaces in 2 areas without a backbone area. The keyword in the RFC is “should”. In RFC parlance this means it’s optional for the vendors to implement. In this case Cisco does NOT initially set the “B” bit unless the router has at least 1 interface in area 0 and if neither router initiates setting the “B” bit then, the remote router doesn’t know the other is an Area Border Router and won’t form a virtual-link.

This is where the phrase "I'll show you mine if you show me your's" comes in. Both routers are being a little shy about their "B" bits and won't identify themselves as area border routers until a router with an interface in area 0 does so first.

There is a scenario when this is not the case.

Let’s take our topology, the one without an area 0, and configure its interfaces to be in a VRF and run OSPF in the VRF and observe a PCAP. You’ll see the “B” is now set. Wait… but why? There isn’t an area 0 configured? I thought you needed an 'area 0' in-order for a router to initiate setting the "B" bit?

The Testing Topology (no area 0)

R1: ip vrf OSPF_TEST int lo1111 ip vrf forwarding OSPF_TEST ip add 1.1.1.1 255.255.255.255 ip ospf 1 area 1 int eth0/0 ip vrf forwarding OSPF_TEST ip add 10.1.12.1 255.255.255.0 ip ospf 1 area 12

R2: int lo2222 ip add 2.2.2.2 255.255.255.255 ip ospf 1 area 2 int eth0/0 ip add 10.1.12.2 255.255.255.0 ip ospf 1 area 12

Now, configure a virtual-link between the 2 routers, juts as you normally would.

The Test Topology (no area 0)

R1: router ospf 1 vrf OSPF_TEST area 12 virtual-link 2.2.2.2

R2: router ospf 1 area 12 virtual-link 1.1.1.1

It comes up!!! Wait… but why? There isn’t an area 0 configured?

The answer can be found in Cisco documentation regarding the OSPF “capability vrf-lite” command: (https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_ospf/command/iro-cr-book/ospf-a1.html#wp2582896905)

The OSPF VRF process acts as an Area Border Router (ABR) when you configure an OSPF process that is associated with a VRF without the capability vrf-lite.

Simply put, the Master Peter Paluch(@Peter_Paluch) so eloquently stated on the Cisco forums: (https://community.cisco.com/t5/routing/where-to-configure-the-quot-capability-vrf-lite-quot-on-ce-or-pe/m-p/2812308/highlight/true#M260868)

...if an OSPF process is run in a VRF then it automatically and unconditionally considers itself to be an ABR - it believes to be connected to a so-called MPLS Superbackbone (even though there may be no BGP/MPLS configured on the router at all).

In conclusion, it is possible to to have fully-adjacent virtual links across an OSPF domain where area 0 does NOT exist. By putting an interface in a VRF, a router will advertises to it's adjacent routers that it is an area border router ABR.

Through my testing I have not found an instance where a router who DOES NOT have an interface in area 0 has initiated setting the "B" bit unless it is configured as part of a VRF.

Thanks to all who helped me through this 2-day deep dive: Nick Russo(@nickrusso42518) and the folks over at The Network Collective.

CCIE R/S: I'm the best engineer I have ever been.

Walking out of the lab exam I can confidently say: “I am the best engineer I have ever been.”

I had originally scheduled my exam for Feb 14 and I am so glad I moved my date to April 2^nd. I am much better engineer than I was Feb 14. I’ve put in nearly an additional 100 hours labbing, I have learned quite a bit more and solved some complex topologies.

During the exam I remember reminding myself to smile, to just take a moment and take it all in. Soaking in the fact that this day has finally come and I’m really here, I’m really doing this. I smiled a few times during the exam, I looked around, nobody else was smiling.

I stayed at the Hampton Inn. It was nice, clean and I enjoyed it. They have a shuttle that will run you anywhere within 3 miles. I used it to grab dinner the night before. It also was the shuttle that took me to Cisco building 5 the morning of.

I arrived at Cisco a few minutes past 8:15am and was one of the last people to arrive. There was around 16 of us of varying tracts. Our ID’s we checked, we placed our lunch orders, received name tags and were escorted as a group to the testing room. It was the last room down a long hallway. Once in the room, our ID’s were checked again and we were assigned seats. There was no babying anyone here. Turn off your phones put all your personal items on the shelving in the back and sit down. Within 1 or two minutes we were told to begin.

Troubleshooting

This first section is “Troubleshooting”. It is 2-hours long with an option to extend 30 minutes. (This 30-minutes gets subtracted from your configuration section) I didn’t feel like I had enough points at the end of my 2-hours so, I chose to extend the additional 30-minutes. What I will say is, it is an amazing feeling when you can get your output to match the expected output. You really feel like you’ve won.

Diagnostics

Next is ‘Diagnostics’. I found this section to be the hardest of all. Out of all the labbing, reading and rehearsing I didn’t not prepare for the diagnostics portion, I don’t know if anyone does.

Configuration

I got started the configuration portion before lunch. I didn’t get too far before the proctor announced lunch time. It was that moment, I remember hearing other’s stories about their CCIE lab experience and talking about “the golden moment”. The golden moment is when everything is pingable(I think), you should achieve this before lunch. Well, I was way off from achieving ‘the golden moment’, but I wasn’t sweating it.

Lunch

When it was lunch time we were lead into the next room where our lunch bags were lined up. I remember seeing most people scarfing down their food and I couldn’t understand why. I was thinking to myself while watching them inhale, ‘slow down, this is your break, for some of us this might be the best part of our day.’ It was Jason’s Deli and well, Jason’s isn’t bad. I tried to take my time and enjoy myself because after all I’ve spent way too much time preparing for this day, for it just to become a smear of a memory. I wanted to experience all it had to offer… I want to savor every cent’s worth of my $1600 sandwich. I remember asking myself while enjoying my ‘deli club’, ‘now that I’ve seen the test, how could I have prepared better?’ and well, I guess I don’t have an answer for that. I’ve already committed all my free time, my weekends and my evenings where am I going to get more time from? Maybe that’s the key, it’s not the amount of time but the quality of studying during that time.

The Highs

Here’s where I was successful, for my first attempt:

• I had a good pace. While I didn’t complete all the sections I never felt panicked.
• I really used my templates. I templated anything that went on more than 1 router/device. I’m comfortable templating things. This comes from doing lots of labs, over and over. I had found some pretty quick ways to get through some large configs leveraging templating.
• The main protocol implementation and deployments I am very comfortable with. I didn’t struggle to implement any protocols (for the sections I completed). This confidence and experience comes from doing lots of labs, and implementing protocols many different ways.

The Lows (Face Palm Moments )

There were 2 or 3 items that I had to do twice. This didn’t take an extremely long time but looking back that was time I’ll never get back.

You might be asking: How could you do items twice?

I would read the requirements and template items out in notepad changing the part that needed to be changed per device and pasted them in. Only to find you forgot to change something like a router-id along the way so all your devices have the same router-id. 🤦

Or one time I was pasting in configs from a template I created and realized I made a mistake. So, to clear-out the entire configs section so I can re-paste it all in, I did a ‘no router [protocol]‘ and removed the whole protocol, only to realize later this protocol was preconfigured on the router so I accidentally removed all the pre-configurations along with my changes. I was able to recover from this mistake, but it took a while to rebuild. 🤦

The Interface is Terrible

I have only 1 complaint about the CCIE R&S lab exam: The interface which has the tickets/tasks, topologies and files is terrible. Over the past year I’ve used a combination of INE and Cisco 360 lab materials. I’ve labbed my own topologies as well as Narbiks material using a variety of EVE-NG, GNS3 or whatever is provided form vendors. All of which have better interactivity with the devices and clearer diagrams to understand.

Even with this being extremely frustrating this will not be the reason I fail (if I failed). I would’ve failed because I didn’t understand some of the requirements. Most of the items were clear but I had a hard time distilling down what was expected in the ‘end-state’.

I felt the difficulty level was below what I was expecting.

What's life all about?

After it was all over I find myself contemplating life. I was in the Uber ride heading to the airport and saw a nice bit of thick green grass in between an exit-ramp and overpass. There were varying patches of wild-flowers coloring the green backdrop. I wanted to get out a roll around like a dog scratching his back.

 I’m wondering why would anyone put as much time into something as I have, just to fail? I mean all the time I spent away from my family. All the event’s I had to declined… and for what? I mean what is networking? Networking is nothing more than pushing around theoretical ones and zeros. What’s the point?

Enough already! Did you pass or not?

People have been asking “did you pass or not?”, to which I’m replying, “does it matter?”. I’m not checking my results for 1-week. Instead I’m going to celebrate with my family and friends and live life. I don’t feel like I passed but at least for 1 week I’m going to walk around feeling like a champion.

You see, if I check my results and find-out I failed, my whole week of celebration I’ll have in the back of my head the stress of labbing more, reading more, watching more videos, missing my family and declining more events. So, I’m putting it off for 1-week so I can celebrate stress free. Then, after 1-week I’ll take a look and see if Cisco invites me back to take the test again 😉

I'm not looking, nope, nope, nope.

CloudShark Holiday PCAP Challenge: Grinchident 2018

This was a challenge I participated in around Christmas time. Below is a copy and paste of the write-up I supplied Cloudshark.

Today, myself and other participants along with the folks from Cloudshark did a webinar to discuss what there is to gain from doing challenges like this and how to build them.

If you would like to try the challenge use this link: https://enterprise.cloudshark.org/blog/holiday-capture-challenge-2018/

BELOW CONTAINS SPOILERS, ALOT OF THEM!!!!

After completing the Cloudshark Halloween challenge, which was a lot of fun (I blogged about it here: https://sealingtech.org/2018/11/14/trick-or-treat-halloween-pcap-challenge-from-cloudshark/ ), sparked my thirst for PCAP challenges, I started the Holiday PCAP challenge: https://enterprise.cloudshark.org/blog/holiday-capture-challenge-2018/ . I really loved how this one had a PCAP inside of a PCAP inside of a PCAP style. I worked through this one pretty quick but I did reach a hard stop. I’m stuck and can’t go any further. Below are my findings.

The challenge starts with 2 PCAP’s: (more will be found—We have 5 so far)

• grinch_activity.pcapng
• holiday_chunk_0.pcapng

Before I start any PCAP challenge or begin to look at any PCAP I always perform these few steps:

• StatisticsàProtocol Hierarchy
• StatisticsàConversations

This will give us a breakdown of the protocols and conversations we’re seeing before we even dive in to any of the packets. This allows us to treat the PCAP like an onion and slowly peeling off one layer at a time.

Going back to the packet list… as, soon as you open the “grinch_activity.pcapng” file you’re immediately presented with some UDP traffic and an obvious email.

The UDP traffic looks eerily similar to the RTP Stream that was embedded in the Halloween PCAP Challenge. Because the UDP Data made up about 18% of the overall traffic with in the PCAP I thought it would be a good idea to peel off that traffic and get it out of the way.

3 Songs

Using the same procedure for all, I was able to extract 3 songs:

• Christmas is Coming by Vince Guaraldi Trio (udp && ip.addr == 10.0.0.5 && ip.addr == 10.0.0.15) 
• Skating by Vince Guaraldi Trio (udp && ip.addr == 192.168.93.173 && ip.addr == 192.168.253.82) 
• You're a Mean One, Mr. Grinch (udp && ip.addr == 10.163.173.167 && ip.addr == 172.20.192.31)

I identified these conversations from the Conversations Statistics window from above.

Step 1:

Right click the first packet in the Packet List and navigate to “Decode As…”. Under the “Current” table heading double click to activate the drop down menu and choose “RTP”.

This instructs Wireshark to use its RTP protocol analyzer against this traffic.

Step 2:

Navigate to: TelephonyàRTPàRTP Streams

Step 3:

In the RTP Streams window click “Analyze”.

Step 4:

In the RTP Stream Anaylsis window click “Play Streams”.

Step 5:

Enjoy!

Export Objects

One of the next things I do almost every time is export all the objects Wireshark can identify.

Navigate to: FileàExport Objectsà HTTP

This will dump only a few files, but very important files.

I quickly grabbed the *.torrent file and threw it in a torrent program. This yielded another PCAP file: “btc.pcapng”.

Now, we have a total of 3 PCAP’s to work with.

I lightly investigated the “mad-holiday-antler.jpg” image but it hasn’t yet yielded me a flag.

Merge PCAPs

Since we now have 3 PCAPs to work with I took the strategy of removing the 3 song streams from the PCAP and merging all 3 into 1 PCAP file.

Export Objects: Part 2

Cloudshark also gave us the Pre-Master-Secret key file to be used with these PCAPs. I applied that file to the PCAP and did the export object procedure again, this time yielding many more objects.

There are some interesting finds to come out of this export:

• Visits to Cloudshark’s website to specific blog posts(the may give clues to solving some of the challenges 😉)
• Bittorrent 
• Memcached 
• VPN leaks 
• IMDB for The Grinch Who Stole Christmas 
• A few visits to Anti-Christmas websites 
• Bing 
• CyberChef on Github

Some of these items are clues to solving the Mystery…

Emails

 

Before doing any other deeper investigation lets finish looking at all of the clear text protocol: SMTP

You can navigate to: FileàExport ObjectsàIMF

This will export the *.eml files cleanly and then open them with notepad or notepad++.

Or because the protocols are plain text you can read them in line in the Packet Details in Wireshark.

DNS

Looking back at our protocol statistic we have many obvious DNS requests to lets take a look.

I often like to give special attention to DNS because it’s clear text and you can tell a lot about a network by observing DNS. Also, for the sake of PCAP challenges you can hide a lot in DNS if you wanted.

Out of all the packets there were only a few DNS requests and responses. They didn’t yield any flags but I feel it’s important to document the findings.

IRC (or GRC: Grinch Relay Chat)

Identified in the Protocol Hierarchy(our first step), we can see a small amount of IRC traffic.

IRC traffic typically uses TCP over port 6667 by default. We can identify this traffic using the filter: “irc”

In the packet details pane I expand the protocol and right click on “Trailer” and choose “Apply as column”. This allows me to quickly see the text conversation.

It’s a small Christmas poem with the cadence of Dr. Suess How the Grinch Stole Christmas or ‘Twas the Night Before Christmas.

In this poem are a few clues:

• “in an old protocol where short messages are cached.”
• “peer-obsessed”, “seeds”, “abhorrent”(rhymes with torrent)
• “VPN is configured”
• “sound!”

Memcached (my favorite part)

(Feb 7, 2019 UPDATE: Sake from the Webinar showed a much cleaner and wuicker way to solve this all within Wireshark)

This is chaining together many items and was my favorite to find so far.

To quickly identify the Memcache protocol use the filter: “memcache”

This will yield 2 packets. There are many many packets that make up this entire converstation but all your attention can be brought to one packet.

A single session that once reassembled equals 336848 bytes.

I expanded these details and right-clicked the “Value” and chose “Copy as Printable Text”.

I pasted it into a notepad and couldn’t figure out what to do with it.

I saw it had a trailing “==” sign which often indicated Base64 but I couldn’t figure out what I was supposed to do with it.

Keeping this in the back of my head as I browsed around trying to identify different parts of this story I remember seeing CyberChef, and also it seemed like this PCAP was leaving clues within itself. So, I followed the link to CyberChef (https://gchq.github.io/CyberChef).

NOTE: I recently used CyberChef in another CTF and was familiar with it’s capabilities.

I browsed to CyberChef and pasted in what I thought was Base64 and started to build a recipe. I added the “From Base64” ingredient and viola! I had something.

Right there in the first few lines of the Output I saw “mergecap” and also a string of repeating “HAPPYHOLIDAYS”. I knew mergecap was used to merge multiple PCAPs or also split them. So, I figured this must be another PCAP. I clicked the save icon and saved the file as a PCAP and opened it… it worked!!! I had my fourth PCAP!!!.

I merged this PCAP into my larger PCAP I was already working because it looked like it had part of an existing conversation.

UDP

Upon merging the newly found PCAP, from above, I now had 2 strange UDP packets at the top of my Packet List.

Changing the packet view to ASCII yielded the “HAPPYHOLIDAYS” phrase I saw earlier… but something wasn’t quite right. 

I copied the 2 phrases out of Wireshark and into Notepad for a closer look.

Looking through this we can extract something but what is it? If you subtract all the UpperCase letters and leave only the lowercase something emerges. (I feel like I’m using the decoder ring from ‘A Christmas Story’)

I don’t have the full URI so this makes me think this is incomplete. I’m hoping this points me to another PCAP to grab but at this time it is un-confirmed.

Ascii Art

I also found some ascii art which is similar to what was hidden in the Halloween PCAP Challenge, but with a Grinch/Christmas theme.

User Agent Strings

• GrinchBrowse/1.0 
• Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:64.0) Gecko/20100101 Firefox/64.0 
• curl/7.55.0

IP Address in HTTP Header?

I found an IP address in an HTTP header that I’m wondering is this something I should hunt or not. I did some preliminary investigation, and have it pinned to Comcast ISP in the North East. I’m betting this is something that didn’t get scrubbed when using trace wrangler to sanitize the PCAP. I assume this is a real IP address used by Cloudshark offices or one of their employees.

OpenVPN

This I have yet to solve.

NordVPN, seems fitting as "Nord" is French for North, as in the NorthPole, maybe it’s a stretch.

NordVPN CA20

us349.nordvpn.com

I changed the HMAC bytes length to 64 and the packets started lining up properly. You can use the filter “openvpn”, you’ll notice all the “[Malformed Packet]” message removed and expand the header find the “Packet-ID” value and scroll down through the packets you should see them all increment: 1, 1, 2, 2, 3, 3, 4, 4…..

FROM THIS...

...TO THIS

ROT13

Hints were being dropped throughout the PCAP. I found someone browsing to the wiki article for ROT13, which is a type of encoding.

I haven’t yet found a string that I can use this on so I’m still looking.

CyberChef supports ROT13 so I’m poised and ready.

The 5th PCAP: 

https://fs-hpcc.qaggle.net

A co-worker (Scott) pointed me to this site which I had previously seen and dismissed. He shared the screenshot which made me perk up:

I immediately did a view source and discovered a hidden message:

I completely ignored the username/password credential entry because of the source looks like no-matter what you enter it will always return: “Password incorrect. Please try again.”

Now, I have the 5^th PCAP:

It, like the other PCAPs, has a UDP packet as the first packet, then my quick glance shows only one other conversation.

The single UDP packet had the same messaging as the other I’ve found adding to my collection: (The middle section)

I’m hoping that this leads me to another PCAP but I can’t seem to get there. I’m missing something.

EVE-NG in the Cloud

EDIT: (Feb 28, 2021) THIS IS OUTDATED CONTENT

Please see this New Video Series: https://youtube.com/playlist?list=PLGs-v2ysJb5bUwWbXFypwr2BmPonTTOER

Please see this new blog post here: https://showipintbri.github.io/blog/eve-ng-in-gcp

Everything You Need to Know About OSPF

"Everything you'll ever need to know about OSPF" is a bold statement but is goes back to "Mastering the Foundations" something I've been saying for the past 25 days. If you can master these OSPF foundations the rest is icing on the cake.

PRO-TIP: Don't treat this as a long list of items to memorize but rather a checklist of topics to lab up.

Default Behavior

• Won't start unless OSPF can determine a router-id
• Router-id determined:
1. Configured router-id
2. Highest loopback IP address
3. Highest interface IP address
• Every router within an area must have the same OSPF database
• Filtering/Summaries happen at the ABR
• Default Priority = 1, highest priority becomes the DR (where applicable)
• A Priority of "0" means router will NOT participate in DR/BDR elections.

OSPF Network Types

Broadcast

• Default for Ethernet interfaces
• Elects DR/BDR
• Uses Multicast
• Allows more than 2 routers on a link
• Timers: Hello - 10, Dead  - 40

Non-Broadcast

• Elects a DR/BDR
• Uses Unicast (neighbor statements)
• Allows more the 2 routers on a link
• Timers: Hello - 30, Dead - 120

Point-to-Point

• Default for Serial and Tunnel interfaces
• Does NOT elect DR/BDR
• Uses Multicast
• Only 2 routers allowed on a link
• Timers: Hello - 10, Dead - 40

Point-to-MultiPoint

• Does NOT elect DR/BDR
• Uses Multicast
• Allows more than 2 routers on a link
• Installs /32 host routes per neighbor
• Timers: Hello - 30, Dead - 120

Point-to-MultiPoint Non-Broadcast

• Does NOT elect DR/BDR
• Uses Unicast (neighbor statements)
• Allows more than 2 routers on a link
• Installs /32 host routes per neighbor
• Timers: Hello - 30, Dead - 120

Loopback

• Default for Loopback interfaces
• When included in OSPF, uses a /32
• To advertise with mask other-than /32, manually set network type to "point-to-point"

LSA Types

LSA Type-1: Router LSA's

• Originated from each router
• Flooded within an area
• Tells the area about all the links participating in OSPF and are associated with that area

LSA Type-2: Network LSA's

• Originated by the DR
• Only DR can originate Type-2 LSA's (If there is no DR their aren't any Type-2's)
• This LSA tells all the routers in an area about all the routers on a shared medium like Ethernet

LSA Type-3: Summary LSA's

• Originated by an ABR
• Carry the destination network prefixes from one area into another

From nonbackbone > backbone

• Connected Routes
• Intra-Area Routes

From backbone > nonbackbone

• Connected Routes
• Intra-Area Routes
• Inter-Area Routes

LSA Type-4: ASBR-Summary LSA's

• Originated by an ABR
• Tells all the other areas about the ASBR
• Tells all the other areas "to get to this Router-ID(ASBR) go through Me(ABR)!"

LSA Type-5: External LSA's

• Originated by an ASBR
• Flooded through out the OSPF domain, except into stubby areas
• Contains the Network prefix and subnet-mask for the external network

LSA Type-7: NSSA External LSA's

• Originated by ASBR
• Exist only in a Not-So-Stubby Area (NSSA)
• Are NOT flooded outside the area they were originated

Area Types

Backbone Area

• Area 0
• Act's as the HUB for all other areas
• Accepts all LSA Types

Normal Area

• All non-stub areas
• Allows LSA's Type: 1, 2, 3, 4, 5 & External Default

Stub Area

• Allows LSA Types: 1, 2, 3 & Summary Default Route ( No External Type-5's )

Totally Stubby Area

• Allows LSA Types: 1, 2 & Summary Default Route

Not-So-Stubby Area (NSSA)

• Allows LSA Types: 1, 2, 3, 7 ( No External Type-5's )
• NSSA's allow redistributing into an area but still maintain it's 'stub area' properties (not allowing External Type-5's)
• Redistributed routes are converted to Type-7 LSA's and advertised throughout the area by the ASBR
• The ABR converts Type-7 LSA's into Type-5's before advertising them into the backbone area.
• LSA Type-7's are only flooded within the area they originate

Totally Not-So-Stubby Area (NSSA)

• Allows LSA Types: 1, 2, 7 & Summary Default Route ( No External Type-5's )
• NO LSA Type-3's

Route Types

• O = Intra-area Route
• O IA = Inter-area Route (Generated by Type-3 LSA's)
• E1 = External Metric Type-1 (Generated by Type-5 LSA's)
• E2 = External Metric Type-2 (Generated by Type-5 LSA's)
• N1 = NSSA Metric Type-1
• N2 = NSSA Metric Type-2