Sunday, August 19, 2018

EVE-NG in the Cloud

EDIT: (Feb 28, 2021) THIS IS OUTDATED CONTENT


Please see this New Video Series: https://youtube.com/playlist?list=PLGs-v2ysJb5bUwWbXFypwr2BmPonTTOER


Please see this new blog post here: https://showipintbri.github.io/blog/eve-ng-in-gcp




I've been running EVE-NG (http://www.eve-ng.net/) locally as a VM on an old PC I use as an ESXi server for about 6 months. It works great and I've really been loving it lately.

In preparing for CCIE Lab exam I've needed to build and run very large topologies. The amount of resources you need will very greatly based on the virtual images you require. I have found the Cisco IOL images to use the fewest resources and run the most reliably. If your working towards Data Center, Service Provider or other exam tracks you'll likely need more than Cisco IOL and will need to run IOS-XRv, Nexus images, CSR1000v's or others which will consume many more resources.

The topologies I'm working from are the Foundation Labs, Troubleshooting and Full-Scale labs from INE's CCIE v5 Routing and Switching workbook. These labs vary from 14 virtual devices to 24 virtual devices. Specifically the version I'm working with uses the IOSv images which consumes many resources during boot and while running compared to Cisco IOL images.

The resources I have at home in my old PC I'm using as an ESXi server is limited and I needed more resources available to run the larger topologies. This brought me to seeking a way to run EVE-NG on scalable and expandable resources.

All initial credit goes to Arwin Reprakash from https://ithitman.blogspot.com/2018/04/configuring-eve-ng-on-google-compute.html, documenting the process and sharing.




If you have a Gmail account you can activate Google Cloud for your account and get $300 FREE, from Google to spend on their resources.

You might be asking yourself: "How much will $300 get me?"

The cost varies based on the resources you consume.

If you're running IOL images you can get away with one of the lower tiers and leave your VM on for 24 hours/day for nearly a YEAR without paying a dime!!!

If you need to consume more resources obviously it will decrease your free $300 at a faster rate. 8vCPU's and 30GB memory you can run for 24 hours/day for about 40 days straight.

Juts for comparison I took a quick glance at what other vendors offer, here's a cost breakdown:

FULL DISCLOSURE: I have not vetted or tested all of these solutions, I'm listing them based on price comparison only. They each offer different solutions. Choose which ever is best for you!

Google Cloud: FREE $300 

(https://cloud.google.com)

  • 1vCPU 3.75GB Memory = ~$26/Month (About 11 Months of continuous running)
  • 4vCPU 4 GB Memory = ~$78/Month (About 3.8 Months of continuous running)
  • 8vCPU 30 GB Memory = ~$195/Month (About 1.5 Months of continuous running)



Packet.net (Bare-Metal) NOT FREE

t1.small.x86: ($0.07/hr) [730 hours = $51.10] (https://www.packet.net/bare-metal/servers/t1-small/)

  • 8 GB of DDR3 RAM
  • 80 GB of SSD
  • 4 Physical Cores @ 2.4 GHz (1 × Atom C2550)

c1.small.x86 ($0.40/hr) [730 hours = $292.00] (https://www.packet.net/bare-metal/servers/c1-small/)

  • 32 GB of DDR3 ECC RAM
  • 120 GB of SSD (2 × 120 GB in RAID 1)
  • 4 Physical Cores @ 3.5 GHz (1 × E3-1240 v5)

m2.xlarge.x86 ($2.00/hr) [730 hours = $1460.00] (https://www.packet.net/bare-metal/servers/m2-xlarge/)

  • 384 GB of DDR4 ECC RAM
  • 120 GB of Redundant SSD (2 × 120 GB in RAID 1)
  • 3.8 TB of NVMe Flash
  • 28 Physical Cores @ 2.2 GHz (2 x Xeon Gold 5120)


Cloud My Lab: Free Trial

https://cloudmylab.com/cciers/#price
https://cloudmylab.com/eve-ng/#cycle-monthly


Configs:


Create the nested virtualization supported image based on Ubuntu 16.04 LTS

gcloud compute images create nested-virt-ubuntu --source-image-project=ubuntu-os-cloud --source-image-family=ubuntu-1604-lts --licenses="https://www.google.com/compute/v1/projects/vm-options/global/licenses/enable-vmx"



Edit sshd_config to allow "root" user to login

nano /etc/ssh/sshd_config

!-change:
PermitRootLogin yes

PasswordAuthentication yes



Change interface name to "eth0"

nano /etc/udev/rules.d/70-persistent-net.rules



Reboot

shutdown -r now



Download the gpg.key, install the new repository, install eve-ng

wget http://www.eve-ng.net/repo/eczema@ecze.com.gpg.key

apt-key add eczema@ecze.com.gpg.key

apt update

add-apt-repository "deb [arch=amd64] http://www.eve-ng.net/repo xenial main"

apt update

apt-get install eve-ng

apt-get install eve-ng



Remove the 4.15 Kernel, use only 4.9 eve-ng Kernel

cd /boot/

mkdir ./old/

mv *4.15* ./old/



Edit grub

sed -i -e  's/GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="net.ifnames=0 noquiet"/' /etc/default/grub

update-grub



Create a new non-root user:

sudo adduser showipintbri

sudo usermod -a -G sudo shoipintbri



Disable root from sshing

nano /etc/ssh/sshd_config

!-change:
PermitRootLogin no

9 comments:

  1. Hi, excellent guide, thanks.
    Whenever I'm trying to start a new node I get the error message: "Failed to create network(11)". After 4 or 5 retrials the image starts. Any idea to fix this?
    Thanks

    ReplyDelete
    Replies
    1. I have the same issue as well. It seems that on first boot you will always get that error but, subsequent boots will not get that error.

      I run an instance of EVE-NG at home and I never turn it off, I never get that error message anymore beyond the very first time I booted it.

      In the Google Cloud because everytime I use it, I boot it up for the first time I get the error message every time.

      The Solution: Start the failed image a second time, I have found it starts without any issues on the second time.

      Delete
    2. Connect using glcoud console. The eve-ng would trigger its installation process and it will automatically install the needed networks.

      Thanks for the tutorial by the way.

      Delete
    3. Same happened to me but I found that Ubuntu had updated and the 4.15 kernel files had returned.

      Delete
    4. I have issue I found new file in /boot/ after that i will move file no relate eve-ng to old folder use this command mv *4.15* ./old/ after that work fine

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Hello Tony, Thanks for your walk-through guide. I would like to know "can we extend CPU, Memory after setup", if "yes" may i know references. If "No" can i migrate this setup to more resources one.

    Thanks in advance.

    ReplyDelete
  4. can anyone tell how to upload router images to eve-ng google cloud please ?

    ReplyDelete
  5. Very nice... Thanks a lot for this. I was able to setup mine however I have some bit of pain. I am unable to connect my devices to the internet using the "Management Cloud". Do you have a fix for this please?

    ReplyDelete