Technology Lead – Big Data & Spark

About the role

Infosys is looking for an experienced Technology Lead to design, develop, and optimise scalable big‑data solutions. The role focuses on building high‑performance batch and real‑time data pipelines using the Hadoop ecosystem and Spark framework.

Key responsibilities

• Design and implement scalable big‑data architectures for data ingestion, processing, and analytics.
• Develop and maintain batch and streaming pipelines with Apache Spark, Scala, and Kafka.
• Collaborate with data engineers, architects, and business stakeholders to deliver robust data processing systems.
• Ensure data quality, performance, and reliability across Hadoop, NoSQL, and cloud platforms.
• Participate in code reviews, performance tuning, and continuous improvement initiatives.

Required profile

• Authorized to work in Canada without employer‑based visa sponsorship.
• Bachelor’s degree or equivalent; or three years of progressive experience in lieu of each year of education.
• Minimum 4 years of IT experience with at least 4 years in big‑data technologies.
• Proven experience in banking or related domains is a plus.

Required skills

• Apache Spark (Core, SQL, DataFrames, RDDs)
• Scala and PySpark programming
• Kafka for real‑time streaming
• Hadoop ecosystem components (HDFS, Hive, Impala)
• NoSQL databases (HBase, MongoDB, Couchbase)
• SQL and data‑modeling expertise
• Experience with AWS cloud services