The Video on Demand Content Management team is looking to add an additional Engineer SA's to their team as they just converted one of our contractors perm.
This is a team of 12 SA's (6 Engineering, 6 Content).
Split between remote and Reston/Denver/Philadelphia. They are responsible for every movie that comes through the "pipe" to make sure that it comes up on the setup box menu. They are responsible for quality and access of these movies. They are in a virtual and
physical environment with about 150 boxes they're supporting.
They are looking for a well rounded Sr. SA with a strong scripting background, Linux/Unix, network protocol. Nice to have's would be web dashboards, vitalization, Splunk, Python, and Nagios.
This individual will be a 24x7x365 on-call member of an Agile, geographically dispersed Product Engineering Operations team within the Technology and Product Organization. This team operates Comcast’s cross-platform Video On Demand Infrastructure including
back-end systems supporting ingest & entitlement databases delivering video content to over 25 million customers.
This job involves supporting a full range of activities necessary to deliver services on an end-to-end basis supporting the national Video On Demand infrastructure, including but not limited to; design, implement, troubleshoot, analyze and maintain the VOD
infrastructure of internal and external customer environments; including network connectivity and utilization, Windows / Linux server administration, virtualization, performance tuning, architectural design, software & hardware deployments, process development
& analytics development.
Identifies, researches, and resolves technical problems while maintaining documentation of said networks, systems and processes. Relies on experience and judgment to plan and accomplish goals while providing support, root cause analysis and remediation strategies
for identified local and systemic issues. Requires the development of creative technical solutions aimed at improving performance, workflows, methods and procedures.
• Function as a technical leader within an Agile team responsible for the day-to-day, 24x7x365 on-call engineering, system administration and operation of Comcast’s VOD (Video On Demand) ingest and delivery infrastructure
• Lead process improvement strategies, build a strong collaborative environment for all stakeholders, possess and display deep and broad technical knowledge while applying a broad scope of influence. Provide input into system design while composing and publishing
technical requirements and architectural designs. Demonstrate deep understanding of end-to-end operational process life cycles and system designs
• Innovate and evaluate at a senior level; drive solutions through other teams; think strategically across one or more technologies demonstrating a vertical depth of knowledge. Stay current on cutting-edge technology while educating peers and encouraging knowledge
growth and collaboration. Drives and champions change; innovatively solves problems; is willing to take risks and contribute innovative ideas
• Support, & maintain the VOD infrastructure via Windows / Linux operating system installations, upgrades, patching, tuning, log parsing, construct data serving methods, system configurations. While guiding and ensuring holistic operational monitoring and alarming
practices across Comcast VOD devices
• Perform application software installations, upgrades, and troubleshooting across products including but not limited to; NetApp, Isilon, Oracle, MySQL, Apache, Aspera, Puppet, Elemental, RGB Networks, thePlatform, Nagios, Harmonic, MediaInfo, Sencore, and
other Comcast VOD related devices. Lead the planning, testing and execution of aforementioned system and application level upgrades
• Autonomously lead projects, including architecture and design reviews. Apply expert engineering methodologies in Agile environments. Serve as a team lead on a regular basis across multiple projects while providing peer & team program or product leadership
• Present and defend market ideas and concepts to peers and management. Lead efforts to forge relationships across teams and or mentor large groups, foster technical solutions amongst peers, build ties with partner organizations, vendors and professional organizations
• Perform troubleshooting, problem resolution activities and root cause analysis with other support teams (e.g. network support, service desk, vendors). Produce root cause analysis findings that focus on recommendations to learn from, and avoid problem recurrence
mitigating future systemic system issues
• Participation in an on-call rotation for after-hours coverage as needed. Occasional travel required, other duties as assigned
Required / Desired Skills & Experience:
• At least 7 years of Linux systems administration, performance tuning, and troubleshooting experience in a large-scale (100+ servers), customer-facing, highly available environments
• In-depth experience with IP networking (TCP/IP packet analysis, DNS, DHCP, NAT, SNMP etc.)
• Experience supporting large-scale database applications with a clear understanding of databases and the ability to write / parse SQL queries and databases
• Tools development in Perl, Python, Ruby, PHP, ASP, CGI or similar languages. Scripting with Perl, Bash, Python, Groovy. A focus on automation to reduce time on repeatable tasks and data aggregation / presentation
• Experienced in managing highly available customer facing applications while defining and designing applications based on LB and GSLB
• Experience with large-scale storage infrastructures (NetApp, Isilon) deploying, configuring and supporting said devices
• Experience managing servers remotely in a distributed environment along with digital data transfers through Aspera or similar products
• Design and implementation of monitoring and notification approaches maximizing system stability, function and performance. Utilizing tools such as Nagios, Cacti, Midas, Icinga, OpsView, NetCrunch, Splunk, etc.
• At least 1 year experience with Agile / Lean environments utilizing work management tooling (Rally), experience with Scrum constructs in team environments.