Senior Development, Operations and Infrastructure Engineer
Founder / Consultant, January 2015 - Present
Providing infrastructure, networking, cloud, technology and devops solutions for startups.
- Designed, built and managed 'all of the things' for AWS environments devexi.com, execvision.io and my-pregnancy.org.
- Technologies include VPC arcitecture, subnets and routing, security groups, load balancing, RDS and Redshift, as well as IAM.
- Integration with third-party solutions including pagerduty, pingdom, datadog, github, bitbucket, slack, deadmanssnitch and travis.ci.
- Built continuious integration systems using technologies including jenkins and travis.ci, custom apt repositories using aptly, and deployments with ansible.
- Automated all things infrastructure and configuration with ansible.
Lookingglass Cyber Solutions
DevOps Engineer, Sept. 2014 - January 2015
Primarily focused on building out a new data center to meet the growing needs of our company. Designed a highly-available, redundant network architecture utilizing Cisco hardware including ASR 1001-x, ASA 5585 pair and Nexus 5672UP and 2232 switches.
- Built a highly-available Cisco-based 10Gb network providing complete redundancy to the edge of our network.
- Built a highly-available infrastructure network segment, providing core services to our development teams (DHCP, DNS, DDNS, etc).
- Worked with resellers to procure hardware.
- Worked with data center to procure rack space, power and internet connectivity.
DevOps Engineer, March 2014 - Sept. 2014
Brought on board to help grow our team, support existing services and infrastructure, as well as scale, automate and improve our AWS-based race tracking / time keeping service.
- Built a scalable, redundant network with per-service network segments inside of Amazon's AWS VPC.
- Automated deployments of VoltDB into the AWS VPC.
- Developed a solution to automate and scale our Jenkins-based CI testing server, utilizing Docker and AWS.
- Built an automated FreeBSD ZFS-based system, allowing individual developers to have up-to-date copies of live MySQL data to test and develop against.
DevOps Engineer, August 2013 – March 2014
Hired to lead a new department responsible for the production, customer-facing environments, as well as to build out new data center environments. My team built a hot-failover disaster recovery environment in the Amazon Cloud utilizing AWS, VPC, EC2, RDS and a lot of Chef. Day-to-day, I mentored junior team members on GNU/Linux administration, configuration management and production environment support.
- Built a hot-failover environment in an AWS VPC, to allow for live migrations in the event of outages to our production environment.
ZeniMax Online Studios
Senior Systems Engineer, July 2011 – August 2013
Hired as the first systems administrator, I was responsible for the early stages of the studio, as well as building the production datacenter environments. My team evaluated, settled on, built and implemented the technology to support millions of concurrent users. Day-to-day responsibilities included building redundant and scalable architecture on which large-scale, production GNU/Linux deployments would support millions of concurrent users from around the globe. To make our jobs easier, we utilized chef and fabric to automate all of the things, and built automation tools utilizing ansible, bash and perl.
- Deployed large-scale production systems on which millions of players log into and play in the Elder Scrolls Online world.
- Built redundant and scalable architecture with Cisco and Juniper networking equipment, HP blade servers and an ESXi cloud utilizing vCenter for management.
- Worked with the development teams to package and automate their code for deployment into the various environments.
- Troubleshoot and support networking and systems issues during live beta test sessions.
Support Engineer II / Lab Manager, June 2010 – July 2011
For a world leader in intelligent Intrusion Prevention Systems and the creator of Snort, provided second-tier support for entire range of software and appliances. Also responsible for administration and direction of lab including forecasting, provisioning and delegation of work.
- Built frontend and backend automation tools in the AWS cloud which allowed customers to upload troubleshooting data, and for engineers to analyze and archive that data in real time.
- Supported production IDS / IPS systems for public and private sector clients, including complex MySQL, perl, GNU/Linux, and network connectivity and performance issues.
- Participated in an on-call rotation in order to provide all-day, every-day support to customers.
Systems Administrator, June 2009 – June 2010
For an automated red light and speed enforcement vendor, managed the on-site and in-the-field servers and networks in support of realtime traffic enforcement systems.
- Built and maintained an 80-site VPN which allowed remote traffic enforcement systems to communicate with our backend in realtime.
- Developed monitoring and alerting for all remote sites, which allowed us to proactively respond to field issues.
- Lead the migration from analog to IP cameras, which reduced our field support costs by allowing for remote maintenance and resolution of issues.
- Deployed puppet to manage the state of our remote systems.
Systems Administrator, Sept. 2007 - April 2009
For an international supplier of military and civilian emergency first responder products, managed the data center, network, and remote-office connectivity.
- Administered the firewalls and VPN appliances in support of road-warrior and remote office connectivity and security.
- Managed a small network consisting of Windows and GNU/Linux servers and desktop.
- Ensured backup system was functional and that the integrity of our backups
- Developed backup strategy and ensured that our backups were reliable.
Hanover Uniform Company
Systems Administrator, April 2005 – Sept. 2007
Administered a mixed Windows and GNU/Linux network for a small 30-personal office. Responsible for public-facing websites and mail servers, as well as development of eCommerce sites and customer support.
- Moved code repository from a private SVN server to GitHub.
- Implemented automated Travis CI tests which fire off automatically during each commit. These tests allow us to ensure that our code compiles and runs, which helps us catch bugs early.
- Brought together a small team of the core developers and gave them commit / admin access to the GitHub code repository.
- Currently working on a completely new frontend. Building a backend API with CakePHP, and a frontend with AngularJS.
- (some) Python and Ruby
Skills and Technologies
- Virtualization including VMware (ESXi, vCenter), KVM, LXC, Docker, OpenVZ, SmartOS
- GNU/Linux ecosystem - bind, dhcp, squid, postfix, mysql, MariaDB, keepalived, apache, nginx, graphite, SMTP, LDAP, nagios, munin, SNMP.
- Networking technologies including VPN, 802.3ad, TCP/IP, Routing, VLAN and security