Kyle Johnson

Senior Development, Operations, Cloud and Infrastructure Engineer

Job History

DomainMarket

DevOps / Infrastructure Engineer, April 2019 - Present

Responsible for bringing legacy infrastructure in to the modern world, and for implementing DevOps best-practices along the way.

Migrated self-hosted single-point-of-failure infrastructure, including DNS, MySQL, Redis, and Linux servers, to redundant, highly-available AWS-hosted infrastructure.
Implemented infrastructure-as-code utilizing Terraform and ansible.
Deployed GitLab and implemented CI-pipelines, and automated no-downtime deployments.
Migrated self-hosted DNS for 350k domains from tinydns server to redundant, highly-available PowerDNS servers.
Built system utilizing python, redis queues, Vault, Consul, Let’s Encrypt, and PowerDNS to automate issuance and deployment of SSL certificates for 350k domains.

ExecVision

DevOps Manager, July 2017 - February 2019

Responsible for designing and deploying from the ground-up all infrastructure, network, security, and monitoring systems, as well as development and production AWS environments in support of highly-distributed remote team.

Created highly-available AWS architecture utilizing EC2, autoscaling groups, RDS, lambda, ELB, EFS, ansible, and terraform.
Created systems to optimize cost savings by efficiently scaling EC2 spot instances between 0 and 300 servers daily.
Built CI/CD processes with gitlab, jenkins, and ansible.
Participated in SOC 2 audits and certification.

Digitally Imported, Inc.

DevOps Manager, May 2015 - July 2017

Responsible for growing a legacy network which supported a number of internet raido streaming sites, including di.fm, rockradio.com, jazzradio.com, classicradio.com and radiotunes.com

Migrated legacy single-point-of-failure varnish load balancer to redundant nginx Plus solution.
Built and managed multi-server GlusterFS FreeBSD ZFS SAN for ingestion, analysis, and storage of audio files.
Backported configuration management of servers and services in to ansible

Lookingglass Cyber Solutions

DevOps Engineer, Sept. 2014 - January 2015

Primarily focused on building out a new data center. Designed a highly-available, redundant network architecture utilizing Cisco hardware.

Built a highly-available Cisco-based 10Gb network providing complete redundancy to the edge of our network.
Built a highly-available infrastructure network segment, providing core services including DHCP, DNS, and DDNS.
Worked with resellers to procure hardware and with data center to procure rack space, power, and internet connectivity.

ChronoTrack

DevOps Engineer, March 2014 - Sept. 2014

Brought on board to help grow team, support existing services and infrastructure, as well as scale, automate and improve AWS-hosted race tracking / time keeping service.

Built scalable, redundant AWS environments for development and production.
Created Jenkins CI pipelines and automated deployments of internal services.
Developed a solution to automate and scale our Jenkins-based CI testing suite, utilizing Docker and AWS.
Built an automated FreeBSD ZFS-based system, allowing individual developers to have up-to-date copies of live MySQL data to test and develop against.

Millennial Media

DevOps Engineer, August 2013 – March 2014

Hired to lead a new department which was responsible for the customer-facing environments, as well as to build out new data center environments. My team built a hot-failover disaster recovery environment in the Amazon Cloud (AWS) utilizing VPC, EC2, RDS, and a lot of Chef.

Built a hot-failover environment in AWS to allow for live migrations in the event of outages to our production environment.
Mentored junior team members on Linux administration and DevOps best practices.
Migrated deployment automation and systems configuration from CFEngine to Chef.

ZeniMax Online Studios

Senior Systems Engineer, July 2011 – August 2013

Hired as the first systems administrator, I was responsible for the early stages of the studio, as well as building the development and early production data center environments. My team evaluated, settled on, built and implemented the technology to support millions of users.

Architecting and building redundant and scalable infrastructure with Cisco and Juniper networking equipment, HP blade servers, and an ESXi private cloud.
Deployed large-scale production systems on which millions of players log into and play in the Elder Scrolls Online world.
Used Puppet and Chef to package and automate code, configure systems, and ensure infrastructure was in a consistent, expected state.
Troubleshoot and support networking and systems issues during live beta test sessions.
Participated in the hiring process, and mentored junior engineers.

SourceFire, Inc.

Support Engineer II / Lab Manager, June 2010 – July 2011

For a world leader in intelligent Intrusion Prevention Systems and the creator of Snort, provided second-tier support for entire range of software and appliances. Also responsible for administration and direction of lab including forecasting, provisioning and delegation of work.

Built frontend and backend automation tools in the AWS cloud which allowed customers to upload troubleshooting data, and for engineers to analyze and archive that data in real time.
Supported production IDS / IPS systems for public and private sector clients, including complex MySQL, perl, GNU/Linux, and network connectivity and performance issues.
Participated in an on-call rotation in order to provide all-day, every-day support to customers.

Traffipax, Inc.

Systems Administrator, June 2009 – June 2010

For an automated red light and speed enforcement vendor, managed the on-site and in-the-field servers and networks in support of a traffic enforcement systems.

Built and maintained an 80-site VPN which allowed remote traffic enforcement systems to securely communicate with our data center.
Developed monitoring and alerting for all remote sites, which allowed us to proactively respond to field issues.
Lead the migration from analog to IP cameras, which reduced our field support costs by allowing for remote maintenance and resolution of issues.
Utilized puppet to manage the state of remote systems.

TVI Corporation

Systems Administrator, Sept. 2007 - April 2009

For an international supplier of military and civilian emergency first responder products, managed the data center, network, and remote-office connectivity.

Administered the firewalls and VPN appliances in support of road-warrior and remote office connectivity and security.
Managed a small network consisting of Windows and Linux servers and desktops.

Hanover Uniform Company

Systems Administrator, April 2005 – Sept. 2007

Administered a mixed Windows and GNU/Linux network for a small 30-personal office. Responsible for public-facing websites and mail servers, as well as development of eCommerce sites and customer support.

Open Source

ZoneMinder

ZoneMinder is an open-source CCTV / video surveillance software package, allowing for monitoring, recording and motion detection of both hardware and analog cameras. The daemons are written in C++, the management scripts in perl, and the frontend in PHP and JavaScript. I took over the project after two years of inactivity from the then sole developer, with the goal of opening up the project to the community to ensure its long-term success.

Moved code repository from a private SVN server to GitHub.
Implemented automated Travis CI tests.
Brought together the community and enabled a small team of the core developers to contribute to the project.
Wrote and implemented a RESTful API using CakePHP.

sewer

Sewer is a Let’s Encrypt (ACME) client for python. I contributed the PowerDNS DNS provider, which allows sewer to use PowerDNS as an authentication method for ACME.

Patiobar

A NodeJS web frontend for pianobar, which is a CLI frontend for Pandora. Provides a simple way for controlling what is playing on the radio. I use this to allow guests (and myself) to control the music playing outside on my patio with their phones.

Displays the currently playing song from pianobar, a CLI client for pandora.com, as a widget on a Dashing dashboard.