Data Engineer Lead (Bigdata Production support)
Basking Ridge, NJ, United States
Title Data Engineer Lead (Bigdata Production support) Rate Duration 9+ month Location Baskin, NJ (Day 1 onsite) - Pls share candidate consent of day 1 onsite. Mandatory Skills Provide on-call support between 5 PM and 9 PM EST. No of Open Positions 1
Job Description :
Responsibilities: Monitor production jobs for P1 system (Data Management and Reporting DMR).
Debug and restart in the event of production failures.
Acknowledge, respond and resolve production issues.
Understand change requests / enhancements requests.
Design and developed data loading strategies.
Build, develop, testing shared components that will be used across modules.
Extract source data files, stage, transform and load into EERA system ( PostGres SQL )
Ingest data from Mainframe system using Kafka and load into EERA .
Extract source data files, stage, transform and load into BDS system (Hive based)
Use Apache Spark for large data processing integrated with functional programming language Sca la
Create programs to continuously listen in requests from consumption layer, generate data extracts and publish it via email.
On call support during weekend
Handshake with offshore team on daily / need basis.
Mandatory Skills: Scala, Spark, Hadoop, Kafka
Postgres SQL
Hive
Airflow (or Control M / Oozie)
Kubernetes (or equivalent containerization tool)
Agile
Strong architecture, design & coding skills
Experience in production operations and support
Experience in distributed databases
Nice to have skills
Java as some of the UI applications are built in Java. Resource should know Java fundamentals.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
#J-18808-Ljbffr