Roles and Responsibilities

• Develop and maintain robust data integration solutions leveraging Spark, SQL, and ETL processes to ensure smooth data movement across various platforms.
• Apply best practices for system integration to ensure compatibility between disparate systems, ensuring that data flows efficiently and accurately between applications and databases.
• Utilize Synapse and Databricks for building scalable data engineering solutions, integrating data from multiple sources and transforming it into actionable insights.
• Manage and implement Master Data solutions to ensure data consistency, accuracy, and integrity across systems and platforms.
• Design, develop, and manage APIs and integration middleware to connect internal and third-party systems. Ensure proper data exchange via API management platforms and messaging protocols like REST, SOAP, Kafka, or AMQP.
• Ensure that all integration solutions adhere to industry standards for data security, including encryption, authentication, and secure data transfer protocols.
• Implement and enforce data validation processes to ensure the integrity and quality of data during the integration process.
• Work in an Agile or Scrum development environment with version control tools like Git and SVN to collaborate effectively with other developers, product teams, and stakeholders.
• Recommend and implement improvements to integration frameworks and workflows to enhance performance, scalability, and maintainability.

Required Skills & Qualifications

• Expertise in Spark and SQL programming for building scalable, high-performance data processing solutions.
• Proven experience with Synapse and Databricks, including designing and optimizing data pipelines in these environments.
• Strong understanding of system integration principles, including the design, development, and validation of integration solutions across multiple platforms.
• Master Data Management (MDM) experience, ensuring data quality, consistency, and governance across systems.
• Proficiency with middleware tools and API management platforms, with a focus on building and managing APIs and integrations.
• Solid understanding of data security practices, including encryption, authentication, and secure data transfer techniques.
• Familiarity with Agile and Scrum methodologies, including experience with tools like JIRA and Confluence.
• Strong version control experience with Git, SVN, or similar tools.

Preferred Skills/Experience

• Experience with cloud data platforms (Azure, AWS, GCP) and related integration services.
• Knowledge of messaging protocols such as Kafka, AMQP, or similar tools for building event-driven data architectures.
• Familiarity with CI/CD pipelines for automated deployment and testing of integration solutions.
• Ability to work in a collaborative, fast-paced environment with cross-functional teams.
• Strong communication skills, both written and verbal, with the ability to explain complex technical concepts to non-technical stakeholders.

Posted By