Highly skilled Kafka Developer with a strong Java background to join our team. This role will be responsible for designing, building, and maintaining scalable, high-performance, and reliable real-time data pipelines and streaming applications.
The role also engaging in Apache Kafka, Java, and modern stream processing frameworks like Kafka Streams, KSQL, and Apache Flink to solve complex data integration and processing challenges.
The ideal candidate is passionate about distributed systems, event-driven architectures, and delivering robust solutions that drive business value.
Key Responsibilities:
- Design and Implement Data Pipelines: Design, develop, and deploy highly scalable, fault-tolerant, and low-latency real-time data streaming pipelines using Apache Kafka.
- Develop Kafka Components: Build robust Kafka Producers and Consumers in Java to integrate various data sources and sinks, ensuring high throughput and data integrity.
- Stream Processing: Design and implement complex stream processing applications using Kafka Streams and Apache Flink for real-time analytics, data transformations, and event-driven microservices.
- KSQL Development: Utilize KSQL (ksqlDB) to create continuous queries, perform real-time data filtering, aggregations, and joins on streaming data.
- Kafka Connectors: Develop, configure, and manage Kafka Connect connectors to integrate with various data sources and destinations, including databases, APIs, and third-party systems.
- Schema Management: Design and implement schema strategies using Confluent Schema Registry, ensuring data consistency, versioning, and compatibility across the platform.
- Best Practices: Define and enforce best practices for event-driven architecture, topic design, partitioning strategies, and data governance.
Required Skills and Qualifications:
- Bachelor’s degree in Computer Science, Engineering, or related field.
- 5+ years of experience.
- 2+ years of experience kafka development.
- Java Proficiency: Extensive professional experience in core Java programming.
- Kafka Expertise: Deep understanding of Apache Kafka architecture, core concepts and extensive hands-on experience building Kafka Producers and Consumers.
- Stream Processing Frameworks: Proven experience designing and developing applications using Kafka Streams and/or Apache Flink.
- KSQL/ksqlDB: Solid experience writing continuous queries and building real-time applications using KSQL.
- Kafka Connectors: Experience developing, configuring, and deploying Kafka Connect connectors for data integration.
- Schema Registry: Proficiency with Confluent Schema Registry for managing schemas, ensuring data compatibility, and enforcing schema governance.
Preferred Skills:
- Experience with Linux.
- Knowledge of containerization and orchestration technologies such as Docker and Kubernetes.
- Familiarity with monitoring and observability tools like Prometheus, Grafana, or Datadog.
- Version Control and CI/CD: Proficiency with Git and experience with continuous integration and deployment pipelines.
- Familiarity with cloud platforms (AWS, GCP, or Azure) and managed Kafka services (e.g., Confluent Cloud, Amazon MSK).
- Good knowledge of distributed systems concepts, event-driven architectures, and microservices design patterns.
- Good Knowledge of python development.