Site Reliability Engineer, GNC (Falcon)

SITE RELIABILITY ENGINEER, GNC (FALCON)

SpaceX is looking for a Site Reliability Engineer to operate and scale custom-built mission-critical products for Guidance, Navigational, and Control (GNC). The GNC team performs trajectory design and vehicle simulation and participates in recurring mission-critical launch operations. This position will work with the GNC team to maintain and improve a set of GNC-focused tools. Examples of these products include Monte Carlo simulations on a high-performance computing cluster, automated data analysis systems, continuous integration systems for rocket and simulation software, GNC analysis infrastructure, and vehicle configuration verification tools. The ideal candidate will be flexible, possess broad skills across product operations and software development, and flourish in a fast-paced and challenging environment.

RESPONSIBILITIES:

• Deploy, upgrade, operate/maintain, and scale a suite of mission-critical GNC products and services
• Provision and maintain virtual and physical servers
• Work with SpaceX HPC team to monitor and maintain a 4000+ thread HPC cluster
• Closely collaborate with GNC software engineers to create highly operable and maintainable products
• Add monitoring for web apps and respond to outages
• Manage the underlying computational infrastructure of GNC in collaboration with IT
• Engage in and improve the whole lifecycle of services: from inception and design, through deployment, operation and refinement
• Make recommendations for future hardware purchases
• Practice sustainable incident response and postmortems
• Provide end-user support to GNC engineering for products by becoming an expert on analysis applications and support users in troubleshooting and pointing to features
• Configure automated deployment pipelines for web apps
• Develop or improve GNC web apps and tools for better usability, maintainability, and robustness
• Demo and document new software changes such as operating system upgrades, shared filesystem changes, or major tool rollouts
• Focus on performance bottlenecks and performance improvement techniques

BASIC QUALIFICATIONS:

• Bachelor’s degree in computer science, information systems/IT, engineering, math, or scientific discipline and 2+ years of software development experience OR 4+ years of professional experience building software with site reliability or DevOps in lieu of a degree
• Experience with Linux operating systems
• Experience with Python and Python based development frameworks

PREFERRED SKILLS AND EXPERIENCE:

• 2+ years of systems administration, site reliability engineering, or DevOps experience
• 2+ years of experience with Python and Python-based development frameworks
• 2+ years of Linux experience
• Expertise with Docker, Vagrant, and Kubernetes or similar technologies
• Extensive Experience with configuration management tools such as Ansible, Puppet, Terraform
• Experience with build systems (Make, Bazel / Pants / Buck, Gradle) and package management tools (pip, npm)
• Strong understanding of virtualization and hypervisor technologies
• Understanding of databases and data modeling
• Experience with automatically managing dozens or hundreds of servers
• Strong networking knowledge of TCP/IP
• Experience scaling web applications and optimizing applications for performance
• Professional experience with standard front-end technologies like modern HTML, CSS, JavaScript (we use AngularJS, Polymer, Backbone.js, React, and more), REST, JSON
• Solid understanding of UI/UX design to provide intuitive applications
• Experience with high-performance computing systems or large-scale data analysis systems
• Must be comfortable working with mission-critical and sensitive systems, with a sense o