Azure data engineer (Pittsburgh) Job at Cognizant, Pittsburgh, PA

bDlGNjh4UHhJS1VXOERtZlZKZ1V3K1gwUWc9PQ==
  • Cognizant
  • Pittsburgh, PA

Job Description

Job Title - DataBricks Data Engineer

Location - Pittsburg, PA

**Must have 8+ years of real hands on experience**

We are specifically seeking a Data Engineer-Lead with strong expertise in Databricks development.

The role involves:

  • Building and testing data pipelines using Python/Scala on Databricks
  • Hands on experience to develop and lead the offshore team to perform development/testing work in Azure data bricks
  • Architect data platforms using Azure services such as Azure Data Factory (ADF), Azure Databricks (ADB), Azure SQL Database, and PySpark.
  • Collaborate with stakeholders to understand business needs and translate them into technical solutions.
  • Provide technical leadership and guidance to the data engineering team and need to perform development.
  • Familiar with Safe Agile concepts and good to have working experience in agile model.
  • Develop and maintain data pipelines for efficient data movement and transformation.
  • Onsite and offshore team communication and co-ordination.
  • Create and update the documentation to facilitate cross-training and troubleshooting
  • Hands on experience in scheduling tools like BMC control-M and setup jobs and test the schedules.
  • Understand the data models and schemas to support the development work and help in creation of tables in databricks
  • Proficiency in Azure Data Factory (ADF), Azure Databricks (ADB), SQL, NoSQL, PySpark, Power BI and other Azure data tools.
  • Implementing automated data validation frameworks such as Great Expectations or Deequ
  • Reconciling large-scale datasets
  • Ensuring data reliability across both batch and streaming processes

The ideal candidate will have hands-on experience with:

  • PySpark, Scala, Delta Lake, and Unity Catalog
  • Devops CI/CD automation
  • Cloud-native data services
  • Azure databricks/Oracle
  • BMC Control-M

Location: Pittsburgh, PA

Job Tags

Part time, Work experience placement,

Similar Jobs

LexisNexis

Legal Practice Area Consultant Job at LexisNexis

Job Description Job Description This position exists to create preference and active-users in law schools and law firms in an assigned territory by assessing the needs of the customer, developing school & firm-specific business plans, and educating customers on the advantages...

Postal Jobs Assist

Mail Processor - No Experience Required Job at Postal Jobs Assist

Role Overview USPS is accepting applications for PSE Mail Processing Clerks nationwide. This role is vital to ensuring the timely and accurate processing of mail. USPS provides comprehensive training to help you succeed, even if you have no prior experience. Position Details...

LECOM Health

Nocturnist/Hospitalist Job at LECOM Health

 ...Join LECOM Health! LECOM Health is seeking a dedicated Nocturnist / Hospitalist to join our hospital medicine team. This position offers the opportunity to provide high-quality inpatient care during night shifts in a supportive, patient-focused, and collegial... 

ABM Industries

Janitorial Cleaner Job at ABM Industries

 ...performance solutions that improve the spaces and places that matter most. From curbside to rooftop, ABMs comprehensive services include janitorial, engineering, parking, electrical and lighting, energy and electric vehicle charging infrastructure, HVAC and mechanical,... 

Lambda

Senior Recruiter - Operations Job at Lambda

 ...Francisco or San Jose office location 4 days per week; Lambda's designated work from home day is currently Tuesday. Our Talent Acquisition team is seeking a strategic, results-driven Operations Recruiter to build world-class teams for our rapidly scaling AI infrastructure...