Lead Big Data Engineer

Job Description : The Lead Big Data Engineer will play a crucial role in designing and implementing solutions to address complex Big Data challenges. The successful candidate will work closely with Product Management and Engineering leadership to drive innovation and technical excellence. Responsibilities : Collaborate closely with Product Management and Engineering leadership to devise and build the right solution. Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale. Design and implement systems to cleanse, process, and analyze large data sets using distributed processing tools like Akka and Spark. Understanding and critically reviewing existing data pipelines and coming up with ideas in collaboration with Technical Leaders and Architects to improve upon current bottlenecks. Take initiatives, show the drive to pick up new stuff proactively, and work as a Senior Individual contributor on the multiple products and features we have. Requirements : 8 years of experience in developing highly scalable Big Data pipelines. Hands-on experience in team leading and leading product or module development experience. In-depth understanding of the Big Data ecosystem, including processing frameworks like Spark, Akka, Storm, and Hadoop, and the file types they deal with. Experience with ETL and Data pipeline tools like Apache NiFi, Airflow, etc. Excellent coding skills in Java or Scala, including the understanding to apply appropriate Design Patterns when required. Experience with Git and build tools like Gradle/Maven/SBT. Strong understanding of object-oriented design, data structures, algorithms, profiling, and optimization. Have elegant, readable, maintainable, and extensible code style. Experience working closely with the US and India engineering teams to help build the Java/Scala-based data pipelines. Ability to troubleshoot live production server issues. Strong verbal and written communication skills with the ability to articulate problems and solutions effectively. Strong sense of urgency, with a passion for accuracy and timeliness. Ability to work calmly in high-pressure situations and manage multiple projects/tasks. Ability to work independently and possess superior skills in issue resolution. Passion for learning and implementing new technologies and analyzing and troubleshooting issues. (ref:hirist.tech)

Location: pune, IN

Posted Date: 3/26/2025