Mid-Level Data Engineer with DevOps Responsibilities Save

Overview:

Sagiliti is dedicated to leveraging data to drive informed business decisions for our clients by positively impacting people, profit and planet through utility and energy consulting services. We are seeking a talented Mid-Level Data Engineer who will also take on some DevOps responsibilities to help us scale our data architecture and workflows.

As a member of the Data Management and Engineering team, the Data Engineer creates and maintains an optimal data infrastructure, utilized by our internal teams to support our sustainability and financial management services to our clients. Because we are a small team, the Data Engineer carries some DevOps responsibilities. The ideal candidate has a strong Data Engineering background and experience locating production errors and issues and building integrations that improve users’ experience.

Responsibilities:

• Data Pipeline Development: Design, build, and maintain robust, modular data pipelines using dbt, Apache Airflow, Meltano, and Kiba to ensure data is accessible, accurate, and timely.
• Database Management: Manage and optimize PostgreSQL databases, including performance tuning, indexing, and query optimization.
• Cloud Infrastructure: Deploy and manage data solutions on Azure, leveraging Azure services for storage, computing, and security.
• DevOps/Containerization: Use Kubernetes to orchestrate and manage containerized applications, ensuring smooth deployment, scaling of data services, and management of Git repos.
• Collaboration: Work closely with data analysts, data scientists, and other stakeholders to understand data requirements and provide necessary data solutions.
• Monitoring and Maintenance: Implement monitoring solutions to ensure data pipelines and databases are functioning optimally, troubleshooting issues as they arise.
• Documentation: Maintain clear documentation of data pipelines, database schemas, and workflows to ensure knowledge transfer and compliance.
• Best Practices: Advocate for and implement best practices in data engineering and DevOps methodologies to improve team efficiency and code quality.

 Required Skills/Abilities:

• Proficient in PostgreSQL, with experience in performance tuning and optimization.
• Familiarity with Apache Airflow, dbt for orchestrating and managing data workflows.
• Experience with container orchestration, preferably with Kubernetes.
• Knowledge of Azure services and experience deploying data solutions in a cloud environment.
• Basic understanding of DevOps principles and tools (CI/CD, Infrastructure as Code, etc.).
• Strong problem-solving skills and the ability to work independently as well as in a team setting.
• Excellent communication skills to effectively collaborate with cross-functional teams.

 Preferred Qualifications:

• Experience within the Energy industry and/or Financial Services is a plus
• Exposure to AI/ML/LLM related workloads/pipelines is a plus
• Experience with additional data technologies (e.g., Spark, Kafka) and API development
• Familiarity with transactional data solutions, data warehousing, and data lakes

Education and Experience:

• Bachelor’s degree in Computer Science, Engineering, or a related field or equivalent work experience.
• 3-5 years of experience as a Data Engineer or similar role, with a focus on ETL processes and data modeling.
• Certification in Azure or relevant data engineering certifications preferred.

 Working Conditions:

• Prolonged periods of working on a computer.
• This position can be in-office, hybrid, or fully remote for the right candidate. Preference will be given to candidates living within 100 miles of the Excelsior, MN office.