Senior Data Engineer (Remote)
Responsibilities: Team up with architects, scrum masters, leads, managers, and directors, you will work in an Agile environment to make the data on our Enterprise Data Platform accessible for the organization's needs.You will be working in a team with Data Architects and Analysts to build our next-generation data platform in Azure.You will be trailblazing to apply Software Development techniques such as Automated Testing and CI/CD to building data products.Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologiesUtilize programming languages like Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Snowflake and Redshift.Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community.Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance. Requirements: 3+ years of experience with designing, developing, implementing and maintaining solutions for Big Data or data warehouse system. 2+ Years of experience working ina cloud environment such as Azure, AWS or other private or public cloud.Good experience with bringing data into a centralized data repository or manipulating the available data to build additional data sets for Analytics and Reporting purposes.Experienced with maintaining data quality throughout the lifecycle of the data.Experience with Data Modeling, source to target mapping, automated testing frameworks, CI/CD pipelines and task automation using scripting.Experienced with working in Agile environment and end to end automation.Developing new and enhancing existing data processing (Data Ingest, Data Transformation, Data Store, Data Management, Data Quality) components.Strong working knowledge of SQL and the ability to write, debug and optimize SQL queries and ETL jobs to reduce the execution window or reduce resource utilization.Data Engineering experience focused on batch and real-time data pipelines development, Data processing/data transformation using ETL tools, Snowflake and DBT.Experience with Cloud Data Warehouse solutions experience (Snowflake, Azure DW, Redshift or similar technology in other private or public clouds).Complete software development lifecycle experience including design, documentation, implementation, testing, and deployment. Preferred Qualifications: Familiarity with Data Vault, DataBricks, Fishtown/DBT tool & Graph Databases.At least 1 year experience working on real-time data and streaming applications (Spark Streaming or Kafka).At least 1 year of experience with Agile engineering practices.At least 1 year Experience working with Cloud Data Warehouse solutions (ie Snowflake, Synapse, Redshift).