Data Engineer

Position Summary

Data engineer will be responsible for designing, maintaining, and optimizing data driven applications and solutions for data collection, management, transformation, and access.

Essential Duties & Responsibilities

• Design, develop, and Implement ETL (Extract, Transform, Load) solutions for transferring data between various data sources and Snowflake
• Optimize the performance of Snowflake databases, including designing and implementing data structures and using indexes appropriately
• Create and manage data in the Snowflake environment
• Deploy and configure tools to monitor and report on the performance of the Snowflake platform
• Partner with Portfolio Managers to groom and implement stories
• Work on an agile team to quickly iterate and release solutions
• Develops and maintains automated and scalable data pipelines in Snowflake
• Develop new integrations to support continuing increases in data volume and complexity
• Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization
• Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it
• Writes unit/integration tests and documents work
• Automate testing and deployment in Snowflake across Azure
• Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues
• Works closely with a team of frontend and backend engineers, product managers, and architects

.

• Defines company data assets (data models), Qlik Talend/Replicate, Snowflake, and Coalesce jobs to populate data models
• Designs data integrations and data quality framework
• Works closely with all portfolio managers and engineering teams to develop strategy for long term data platform architecture
• Maintain the security of confidential information
• Stay abreast of advances in technology
• Demonstrate regular and prompt attendance
• Performs other related duties as necessary or assigned

..

Supervisory and/or Leadership Responsibilities

None

Knowledge, Skills & Abilities

• Knowledge of best practices and IT operations in an always-up, always-available service
• Knowledge of data pipeline guiding principles, and best practices to build and maintain data pipelines
• Knowledge of data architecture, structures, and principles with the ability to critique data and system designs
• Knowledge of schema design and dimensional data modeling techniques
• Knowledge of ETL processing and tools such as Talend or Matillion or Coalasce
• Knowledge of Agile Software Development and Data Warehousing methodologies
• Knowledge of modeling techniques including Dimensional and 3NF
• Knowledge of SQL, Python for data loading and transformation, manipulation, and analysis
• Knowledge of cloud-based data warehouse platforms as a data engineer focusing on data pipelines, semi structured data (JSON, XML) using the VARIANT attribute, re-clustering of data, ETL, CI/CD pipeline, etc.
• Knowledge of principles and processes for providing customer service. This includes customer needs assessment, meeting quality standards for services and evaluation of customer satisfaction
• Skilled in Snowpipe platform for handling streaming data
• Skilled in identifying, analyzing, and resolving problems
• Ability to manage projects in a process-oriented manner with detailed documentation
• Ability to communicate effectively verbally and in writing
• Ability to work within various databases, both SQL and Oracle, and knowledge of database optimization techniques

...

....

Minimum Qualifications - (Education, Experience)

• Bachelor’s degree in a related technical field
• Directly related experience may be substituted for education
• Four years of experience with data engineering
• Snowflake certification preferred

Physical Demands and Work Environment

The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable qualified individuals with disabilities to perform the essential functions.

• The employee may be required to stand; reach with hands and arms, stoop and kneel
• The employee may be required to sit or stand for long periods of time
• The employee may be required to lift, carry, push, pull or move up to 25 pounds
• The employee may be required to travel
• Exempt employees receive a fixed bi-weekly salary to compensate them for all hours worked during each pay period
• This position may be required to work more than 40 hours per week

This description is intended to indicate the kinds of tasks and levels of work difficulty required of the position given this title and shall not be construed as declaring what the specific duties and responsibilities of any particular position shall be. It is not intended to limit or in any way modify the right of management to assign, direct and control the work of employees under supervision. The listing of essential duties and responsibilities shall not be held to exclude other duties that may be assigned based on the needs of the Cooperative.