The Business Intelligence unit within University Relations is on a mission to modernize its data warehousing infrastructure. We are actively seeking a Senior Data Engineer with expertise in Apache Airflow to support this transformation. The Data Engineer will play a pivotal role in enhancing our ELT/ETL processes, ensuring seamless data flow from various in-premise and cloud sources into our MS Warehouse. This role is crucial in supporting the University of Maryland’s advancement operations, which encompasses development, alumni and donor relations, gift processing, and more. The Data Engineer will collaborate closely with business stakeholders, understanding their data requirements, and implementing scalable, Airflow-based solutions. They will also be responsible for supporting the Python data analysis stack and implementing best practices in data governance.

Responsibilities:

• Design, build, and maintain scalable cloud-based data pipelines using Python, Apache Airflow, and PySpark.
• Implement real-time data streaming solutions and batch processing using modern technologies.
• Work closely with data scientists and analysts to turn data into critical information and knowledge that can be used to make sound business decisions.
• Implement containerization solutions using Docker to ensure a consistent and scalable environment.
• Ability to learn and leverage current data visualization tools to create dynamic data applications and dashboards for various business units.
• Develop complex SQL queries for data analysis and reporting.
• Ensure data quality and integrity through robust testing frameworks.
• Collaborate with a cross-functional team on ms in an agile environment.
• Have working knowledge of ML concepts for data modeling