Job Title: Senior AI Platform Engineer
Department: Front Office Support
About Business line/Function: Global markets – Data & AI Lab
Position Purpose:
We are looking for a highly‑motivated Python developer to join our data‑engineering team. Successful candidate will improve the performance of our AI Platform which includes ML Python code base by designing, developing, and maintaining large‑scale data pipelines using Apache Spark and PySpark, and help us make those pipelines visible and reliable through Grafana‑driven monitoring. You will work closely with data scientists, Devops, and infrastructure engineers to deliver robust, scalable solutions that enable data‑driven decision‑making across the bank.
Responsibilities
1. Design & implement high‑performance data pipelines in Python / PySpark for batch and near‑real‑time processing.
2. Optimize Spark jobs (caching, partitioning, join strategies) to meet latency and cost targets.
3. Improve the existing Python code base – refactor legacy scripts, introduce modern language features, and eliminate bottlenecks to boost execution speed and resource efficiency.
4. Profile and benchmark Python/PySpark workloads and apply best‑practice tuning.
5. Put in place monitoring for our Spark applications (metrics, logs, traces) and build Grafana dashboards (via Prometheus, InfluxDB or OpenTelemetry) to monitor throughput, latency, failure rates, and infrastructure utilisation.
6. Write clean, test‑driven code (unit, integration, end‑to‑end) and keep documentation up‑to‑date.
7. Collaborate with data‑science, product, and infrastructure teams to translate functional requirements into scalable technical solutions.
8. Participate in code reviews, knowledge‑sharing sessions, and continuous‑improvement initiatives (coding standards, linting, static analysis).
9. Monitor production jobs, troubleshoot failures, and set up Grafana
10. Create and maintain Jenkins pipelines and Docker containers for build, test, and deployment, ensuring reproducible environments.
Technical & Behavioral Competencies
Technical Skills:
- Programming ≥ 4 years of professional experience in Python (OOPs, PEP‑8, type hints, async, profiling).
- Data Engineering: Deep expertise with Apache Spark (core, SQL, Structured Streaming) and PySpark; ability to tune jobs for datasets > 1 TB.
- Performance Engineering: Proven track record of optimising Python code for speed and memory usage (vectorisation, Cython/Numba, multiprocessing, Spark‑specific tricks)
- Monitoring & Observability: Hands‑on experience exposing Spark metrics and building Grafana dashboards (or comparable tools).
- DevOps Foundations: Working knowledge of Jenkins pipelines and Docker containerisation.
- Version Control: Proficient with Git (branching, pull‑request workflow).
- Testing: Experience with pytest, unittest, and CI test automation.
- SQL: Strong SQL skills (PostgreSQL, Snowflake, Redshift, etc.).
- Linux: Comfortable in a Linux environment (shell scripting, package management).
- Agile: Experience in Agile/Scrum teams (sprints, backlog grooming).
Nice to have:
- Experience with Kubernetes/Ansible/Tower.
- Knowledge of CI/CD best practices and tools (Jenkins).
- Domain Knowledge: Exposure to quantitative finance, market data (Bloomberg, internal pricing tools), or structured/exotic derivatives.
Behavioral Skills:
Team Collaboration - Steer global partnership between the Data & AI Lab, front‑office desks and distributed IT teams, championing DevOps best‑practice knowledge sharing and guiding the strategic roadmap for the lab’s ML platform across Paris, London, New York, Singapore, Lisboa, India and Frankfurt.
- Strong communication – can convey technical concepts to non‑technical stakeholders.
- Proactive, enthusiastic, and driven to improve processes and controls.
- Fluent English (written & spoken).
Education Level: Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related quantitative field.
Location: Bangalore, India
About BNP Paribas Group:
BNP Paribas is the European Union’s leading bank and key player in international banking. It operates in 65 countries and has nearly 185,000 employees, including more than 145,000 in Europe. The Group has key positions in its three main fields of activity: Commercial, Personal Banking & Services for the Group’s commercial & personal banking and several specialized businesses including BNP Paribas Personal Finance and Arval; Investment & Protection Services for savings, investment, and protection solutions; and Corporate & Institutional Banking, focused on corporate and institutional clients. Based on its strong diversified and integrated model, the Group helps all its clients (individuals, community associations, entrepreneurs, SMEs, corporates and institutional clients) to realize their projects through solutions spanning financing, investment, savings and protection insurance. In Europe, BNP Paribas has four domestic markets: Belgium, France, Italy, and Luxembourg. The Group is rolling out its integrated commercial & personal banking model across several Mediterranean countries, Turkey, and Eastern Europe. As a key player in international banking, the Group has leading platforms and business lines in Europe, a strong presence in the Americas as well as a solid and fast-growing business in Asia-Pacific. BNP Paribas has implemented a Corporate Social Responsibility approach in all its activities, enabling it to contribute to the construction of a sustainable future, while ensuring the Group's performance and stability.
About BNP Paribas India Solutions:
Established in 2005, BNP Paribas India Solutions is a wholly owned subsidiary of BNP Paribas SA, European Union’s leading bank with an international reach. With delivery centers located in Bengaluru, Chennai and Mumbai, we are a 24x7 global delivery center. India Solutions services three business lines: Corporate and Institutional Banking, Investment Solutions and Retail Banking for BNP Paribas across the Group. Driving innovation and growth, we are harnessing the potential of over 10000 employees, to provide support and develop best-in-class solutions.
Commitment to Diversity and Inclusion
At BNP Paribas, we passionately embrace diversity and are committed to fostering an inclusive workplace where all employees are valued, respected and can bring their authentic selves to work. We prohibit Discrimination and Harassment of any kind and our policies promote equal employment opportunity for all employees and applicants, irrespective of, but not limited to their gender, gender identity, sex, sexual orientation, ethnicity, race, colour, national origin, age, religion, social status, mental or physical disabilities, veteran status etc. As a global Bank, we truly believe that inclusion and diversity of our teams is key to our success in serving our clients and the communities we operate in. As a global Bank, we truly believe that inclusion and diversity of our teams is key to our success in serving our clients and the communities we operate in.
