Data Product Developer
- Shanghai
- Permanent
- Full-time
- Design & Build:Develop scalable and reliable data pipelines to handle ingestion, validation, processing, aggregation, and storage in compliance with data warehousing principles and best practices
- Data Integration:Ingest data from diverse sources (e.g., Kafka, APIs, data files) into advanced cloud data platforms, like AliCloud MaxCompute or GCP BigQuery
- Data Modeling:Develop and maintain high-quality data models and transformations using tools such as dbt
- Quality Assurance:Implement robust data quality checks, tests, and monitoring processes to maintain accuracy and consistency across pipelines
- Optimization:Optimize pipelines and data workflows for maximum performance and efficiency. Continuously refine queries to improve speed and execution
- Performance Monitoring:Ensure peak performance of data warehouse systems while proactively addressing bottlenecks and opportunities for improvement.
- Data Warehousing Expertise:Deep understanding and application of data warehousing principles and best practices
- Proficiency in SQL & dbt:Ability to extract and manipulate data efficiently using SQL and dbt on platforms such as AliCloud MaxCompute or GCP BigQuery
- Version Control:Experience with Git and CI/CD pipelines to ensure seamless development workflows
- Programming Proficiency:Hands-on experience with Python or other scripting languages for building data pipelines
- Pipeline Orchestration:Familiarity with tools like Apache Airflow to automate and streamline workflows
- Streaming Technologies: Experience working with Kafka or similar data streaming platforms is a plus.