Staff Software Engineer - Full Stack (Performance Engineering) (Hybrid/Onsite)

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

This position is ideal for an experienced software engineer who is passionate about solving challenging business problems and building applications that have high availability & resiliency. The candidate will be extensively involved in hands-on performance & chaos engineering activities including POCs, design, documentation, code instrumentation, profiling and monitoring application, middleware & platforms components. You will be an integral part of the Open VisaNet performance engineering team.

Essential Functions

Responsible for developing performance plans and models for applications and infrastructure in a highly scalable, low-latency, high-availability, and high-throughput payment processing system.

Comprehend and take ownership of component-level performance engineering tasks, including code and data flow reviews.

Create and participate in performance and resiliency testing and fine-tuning of distributed components.

Collaborate with Development, Test Engineering, Support, and Product & Project Management teams to review and document requirements & timelines

Assess, create, and execute load-testing tools to simulate workload limits for Visas most critical payment applications.

Engage in performance engineering with a shift-left approach on microservices and establish benchmarks. Proficiency in Go Benchmarks and Go Profiling is advantageous.

Design and carry out chaos-resiliency evaluation experiments on microservices to identify potential resiliency weaknesses and failure points.

Perform root cause analysis for identified issues.

Analyze metrics and trends to find opportunities for enhancing application performance.

Explore methods to resolve issues and present corrective actions, conduct sessions with developers and architects to discuss recommended improvements or techniques.

Oversee the system performance lifecycle and identify key metrics for performance enhancements.

Generate regular performance engineering reports for review by architects, developers, and project managers.

Deliver technical presentations to guide teams on improving system and application performance.

Innovate solutions to enhance the effectiveness of performance engineering.

Contribute to performance tuning, scalability, reliability, and high availability of fault-tolerant distributed components.

Ensure proper documentation and internal publication of design trade-offs, code best practices, and lessons learned.

Support in triaging and troubleshooting highly distributed services in production environments.

Design and develop performance engineering aids like profiles, traces, or any observability frameworks.

Establish and promote performance best practices and influence non-functional requirements at early stages.

Work closely with application development teams to provide technical direction for improving application performance.

Mandatory Skillset

In-depth understanding of application performance KPIs, including their trade-offs and patterns

Capability in performance modeling using performance test data tailored to specific non-functional requirements of an application

Solid grasp of microservices architecture, CGROUP, and namespaces within Linux

Comprehensive knowledge of Linux memory management systems

Expert understanding of the IO subsystem, including its layers, various types of IO operations, caching mechanisms, file systems, and RAID configuration

Proficient in the compute subsystem with detailed knowledge of process, thread, and goroutines or LWT scheduling

Familiarity with process states, context switching, and system calls

Understanding of network layers, protocols, flow control, and congestion control

Basic knowledge regarding data centers

Proficiency in at least one programming language such as GoLang, Python, or Java, with the ability to write code independently

Skill in building a cost profile for applications based on CPU, memory, IO, and network test results

Ability to profile CPU and memory consumption or raw dumps and familiarity with traces, thread dumps, and debug logs

Experience in creating performance test strategies, including the development of test cases

Proficiency in designing chaos test case scenarios and formulating test strategies by identifying potential failure points

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

Basic Qualifications
5+ years of relevant work experience with a Bachelor’s Degree or at least 2 years of work experience with an Advanced degree (e.g. Masters, MBA, JD, MD) or 0 years of work experience with a PhD, OR 8+ years of relevant work experience.

Preferred Qualifications
6 or more years of work experience with a Bachelors Degree or 4 or more years of relevant experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or up to 3 years of relevant experience with a PhD
At least 8 years of practical experience in Performance Engineering
Bachelor’s or Master’s degree in computer science or a related field
Experience in Payment technologies is advantageous

Work Hours: Varies upon the needs of the department.

Travel Requirements: This position requires travel 5-10% of the time.

Mental/Physical Requirements: This position will be performed in an office setting.  The position will require the incumbent to sit and stand at a desk, communicate in person and by telephone, frequently operate standard office equipment, such as telephones and computers.

Visa is an EEO Employer.  Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status.  Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Visa will consider for employment qualified applicants with criminal histories in a manner consistent with applicable local law, including the requirements of Article 49 of the San Francisco Police Code.

U.S. APPLICANTS ONLY: The estimated salary range for a new hire into this position is 132,300.00 to 191,800.00 USD per year, which may include potential sales incentive payments (if applicable). Salary may vary depending on job-related factors which may include knowledge, skills, experience, and location. In addition, this position may be eligible for bonus and equity. Visa has a comprehensive benefits package for which this position may be eligible that includes Medical, Dental, Vision, 401 (k), FSA/HSA, Life Insurance, Paid Time Off, and Wellness Program.