Lead Big Data Engineer

Since 1993, EPAM Systems, Inc. (NYSE: EPAM) has leveraged its advanced software engineering heritage to become the foremost global digital transformation services provider – leading the industry in digital and physical product development and digital platform engineering services. Through its innovative strategy; integrated advisory, consulting and design capabilities; and unique ‘Engineering DNA,EPAM’s globally deployed hybrid teams help make the future real for clients and communities around the world by powering better enterprise, education and health platforms that connect people, optimize experiences, and improve people’s lives. Selected by Newsweek as a 2021 Most Loved Workplace.

EPAM’s global multi-disciplinary teams serve 61,300 employees and customers in more than 50 countries across five continents.

As a recognized leader, EPAM is listed among the top 15 companies in Information Technology Services on the Fortune 1000 and ranked as the top IT services company on Fortune’s 100 Fastest-Growing Companies list for the last three consecutive years.

EPAM is also listed among Ad Age’s top 25 World’s Largest Agency Companies and in 2020, Consulting Magazine named EPAM Continuum a top 20 Fastest-Growing organization.

Responsibilities
• Design, implement, and optimize distributed computing systems for big data processing.
• Develop and maintain Apache Spark-based solutions for data processing and analytics.
• Write efficient and scalable code using programming languages such as Python, Java, or Scala.
• Utilize Hadoop v2, MapReduce, HDFS, and Sqoop for data storage, retrieval, and processing.
• Build stream-processing systems using technologies like Apache Storm or Spark Streaming.
• Implement messaging systems such as Kafka or RabbitMQ for real-time data processing.
• Utilize Big Data querying tools like Hive and Impala for data analysis and retrieval.
• Integrate data from multiple sources, including RDBMS, ERP systems, and files.
• Design and optimize SQL queries, joins, stored procedures, and relational schemas.
• Work with NoSQL databases like HBase, Cassandra, or MongoDB for specific use cases.
• Apply ETL techniques and frameworks for data transformation and integration.
• Perform performance tuning of Spark jobs to optimize data processing and analytics.
• Have experience with native Cloud data services, preferably AWS, Azure, or GCP.
• Demonstrate leadership skills in efficiently managing and leading a team.
• Design and implement end-to-end big data solutions that meet business requirements.
• Practice Agile methodology for project management and delivery.