Apache NiFi System Administrator

Responsibilities:

• Design, implement, and manage data flows within Apache NiFi, ensuring efficient and reliable data movement across systems.
• Build scalable workflows for data ingestion, transformation, enrichment, and delivery.
• Configure processors, connections, and controllers to optimize data throughput and performance.
• Regularly monitor and maintain NiFi clusters, ensuring high availability and fault tolerance.
• Implement appropriate security measures, including user access control, data encryption, and SSL configurations within NiFi.
• Ensure compliance with organizational data governance policies and regulations.
• Work on integrating NiFi with other systems such as databases, cloud services (e.g., Google Cloud Platform, AWS, Azure) and other big data or messaging platforms.
• Ensure seamless connectivity between NiFi and external data sources or destinations.
• Implement monitoring tools and set up alerts to detect failures or performance issues.
• Generate reports on data flow metrics and performance for stakeholders.
• Stay up to date with the latest NiFi releases and patches, ensuring that the environment is using the latest stable versions.
• Optimize the performance of data transfer workflows by configuring NiFi's backpressure settings, load balancing, and ensuring high availability across NiFi clusters.
• Monitor file transfer and API performance, troubleshooting bottlenecks and optimizing workflows as needed for both throughput and latency.
• Collaborate with cross-functional teams, including data engineers, security teams, and IT infrastructure teams from both entities, to ensure smooth data flow and integration.
• Provide detailed documentation on data flow architecture, security protocols, and API configurations to ensure that the integration process is well-understood and maintainable by both organizations post-acquisition.
• Ensure that the data transfer mechanisms built are scalable and sustainable for long-term use.
• Support the ongoing data synchronization between systems while moving toward full integration of IT ecosystems.

Requirements:

• 5+ years of experience in Apache NiFi for managing data flows, file transfers, and API integrations.
• Knowledge of secure file transfer protocols (SFTP, FTPS, HTTPS) and encryption methods.
• Experience integrating systems using APIs and working with open systems (cloud, on-premise, hybrid).
• Strong understanding of data governance, compliance, and security practices, particularly in high-stakes M&A scenarios.
• Excellent troubleshooting and optimization skills for large-scale data migration and integration projects.
• Excellent English communication skills both written and verbal.