Senior Manager, System Test

General Overview

Job Title:  Senior Manager, System Test
Functional Area:  Engineering (ENG)
Career Stream:  Test Engineering Development (TED)

Summary

The Senior Manager, System Test, works with cross functional teams of design engineers, customers, supply chain and project managers to rigorously test and validate rack-level, system solutions and ensure new product release readiness. This position will manage engineers with responsibilities for System Integration Testing and Reliability. In this position you will own the test system setup, function, and validation for complex, high-end servers and hyperscale compute & network racks, including applications and operating systems. 

The ideal candidate will have a rack-level, system integration test background in storage/server, GPU and networking products. This person will develop and provide guidance on Celestica Storage/GPU/network system integration and reliability verification plans, test cases and test coverage, such as: functional test, hardware/software integration test, system and rack integration verification, and product reliability analysis and validation. 

Detailed Description

• Design & Execute the comprehensive validation and integration of all components within advanced AI compute racks, ensuring hardware and software elements function cohesively and optimally, ensuring compatibility, connectivity, performance, security, reliability, cost effectiveness and resilience. 
• Lead the development and implementation of system integration test solutions and participate in project planning, scheduling, and reporting. Review and interpret product/system requirements/specifications and define system integration and validation plans.
• Lead the reliability analysis and validation for new products/systems and components. Participate in project planning, scheduling, and reporting. Review and interpret product/system requirements/specifications and define validation plans.
• Provide technical leadership and direction to engineers; be a recognized expert (go to person) in system integration testing and reliability analysis/testing. 
• System Test Setup & Execution: Oversee the set up and rigorously test product and system solutions to guarantee they meet or exceed stringent performance, reliability, and efficiency standards. Own the test system setup and function for complex, high-end servers and hyperscale compute & network racks, including applications and operating systems. Oversee the qualification and seamless integration of rack components such as servers, accelerators, switches, power shelves, and cooling solutions.
• Power Distribution Management: Oversee the integration and configuration of smart PDUs (Power Distribution Units) to  efficiently manage power distribution within the rack. Ensure seamless compatibility with rack-level  power management functions and overall system optimization. 
• Rack-Level Management: Maintain a comprehensive understanding of and proficiency in rack-level  management functions, including real-time monitoring, control, and automation of critical rack  components. 
• Hands-On Racking and Stacking: Oversee the hands-on lab racking and stacking procedures, including the  assembly and setup of intricate, high-touch rack configurations. Ensure meticulous installation and  integration of all components. 
• Documentation and Reporting: Oversee the creation and maintenance of thorough and detailed documentation for setup, testing/analysis and the summarization of results. This includes documenting installation procedures, test plans, analysis/testing results,  performance metrics, and any issues identified during the process. 
• Cross-Functional Collaboration: Own the team collaborations with a diverse range of cross functional teams, including design engineers, program managers, customers, and vendors. The goal is  to ensure seamless integration, optimal & reliable operation, and the successful delivery of cutting-edge rack level solutions. 
• Test Automation & AI: Identify & automate procedures and testing to reduce development time, increase quality and reduce errors. Identify and use AI tools to streamline and improve efficiency of planning, executing, analyzing & reporting.
• Troubleshooting & Root-Cause: Oversee troubleshooting, debugging and root-cause analysis on hardware, network, and software related issue

Knowledge/Skills/Competencies

• Proficiency in managing teams of 8+ persons with experience in continuous improvement initiatives and establishing new processes for system test and reliability.
• Proficiency in defining test plans, creating & executing tests, and generating test reports for internal and external uses.
• Proven experience in test tools such as JIRA, Jenkin (CI/CD), and similar tools and proven success in implementing and maintaining processes for task/defect tracking and reporting.  
• Experience in troubleshooting with oscilloscopes, logic analyzers, and experienced with specialized test equipment such as PCI protocol analyzers and Traffic generators (Keysight/Ixia or Spirent)
• Proficiency in project management of system integration testing. Able to manage simultaneous projects, predict workloads/execution, and deliver per plan.
• Must be self-motivated and able to work independently in a fast-paced, multi-tasking environment 
• Experience with stress testing utilities, stressapp, stress-ng, iperf3, FIO, memtester, etc 
• Experience in Linux with RHEL, Ubuntu and CentOS. Understanding of x86/Firmware/Linux.
• Proficiency in scripting languages: Python, Perl, VBA, REST API, Shell scripting, Ansible, Terraform etc.
• Proficiency in collaborating with a cross-functional team of engineers involved in the specification, planning, design, development and test of hardware.
• System/software/firmware integration and verification experience

• Rack-Level System Integration Expertise: 5+ years is mandatory

  • Experience should include working with servers, accelerators, switches, CDU (Cooling Distribution Unit) systems, and high-power shelves. 

  • Understanding of AI compute rack configurations and the functionality of all integrated components. 

  • Possess a strong understanding of rack-level management functions and integration best practices 

  • Proven experience in testing and qualifying rack-level solutions to ensure they meet or exceed  performance and Reliability benchmarks. 

• Proficient in the management and use of AI tools and test automation

• Exceptional verbal and written communication skills 

• Exceptional problem-solving skills and attention to detail is critical for success 

Typical Education

•   BS degree or higher in Electrical Engineering / Computer Engineering / Computer Science   

Typical Experience

• 11+ years’ relevant experience

Physical Demands

• Duties of this position are performed in a normal office environment.
• Duties may require extended periods of sitting and sustained visual concentration on a computer monitor or on numbers and other detailed data. 
• Repetitive manual movements (e.g., data entry, using a computer mouse, using a calculator, etc.) are frequently required.
• Occasional travel may be required.

Notes

This job description is not intended to be an exhaustive list of all duties and responsibilities of the position. Employees are held accountable for all duties of the job. Job duties and the % of time identified for any function are subject to change at any time.