Glencore

Data Engineer

New York, United States

full_time

About This Job

Glencore

, headquartered in Baar, Switzerland is one of the world's leading integrated producers and marketers of commodities that industries around the world need. Glencore has worldwide activities in the production, sourcing, processing, refining, transporting, storage, financing and supply of metals and minerals, energy products, and agricultural products. We strive to be a reliable and competitive partner in the markets in which we serve and to support our customers and suppliers at each stage of their expansion and development.

Glencore Ltd

, a US-based branch of Glencore AG, is a wholly owned subsidiary of Glencore International AG, where it employees approximately 300 people.

Data Engineer – Commercial Engineering (New York, NY)

We are seeking a highly skilled and motivated Data Engineer to join our business; you will be part of a global Data Engineering team working on data engineering activities for our trading desks with a focus on the desks in New York.

The role reports to the Head of Data Architecture and Engineering.

Key Responsibilities:

•

Design, implement, and maintain scalable data engineering pipelines using Python, Spark, and SQL, applying clean coding principles, and optimizing for performance in Azure and Databricks.

•

Use good software engineering principles to enable modularity, reliability, and reusability of code.

•

Orchestrate complex data workflows using frameworks like Airflow, Databricks workflows (or) equivalent, adopting a code-first approach.

•

Provide data engineering support for trading analysts in NY.

•

Use CI/CD tools like GitHub actions or equivalent for deploying data pipeline to higher environments efficiently.

•

Implement data quality checks, validation frameworks, and monitoring systems to ensure data integrity and reliability.

•

Design effective data models, optimize storage and retrieval processes, and efficiently handle semi-structured and unstructured data.

•

Create and manage APIs for data access and integration and develop web scraping solutions using leading frameworks when necessary.

•

Build scalable data ingestion pipelines for various external data feeds with different formats, velocity, and patterns.

•

Collaborate closely with trading analysts in an agile environment, iterating data products in an efficient manner demonstrating strong understanding of the domain context.

•

Apply strong analytical and logical problem-solving skills to address complex data engineering challenges and develop innovative solutions.

•

Ensure data security and compliance with relevant regulations, implementing best practices for data protection and access control.

•

Continuously optimize pipeline performance, evaluate new technologies, and drive innovation in the data engineering ecosystem.

•

Maintain comprehensive documentation and contribute to the team's knowledge base, fostering a culture of continuous learning and improvement.

•

Collaborate closely with data engineers in London to implement as per standardized guidelines and guardrails.

Qualifications and Requirements:

•

Degree educated in either Computer Science, Mathematics, Engineering, or a related field.

•

Strong hands-on experience with Python, Spark, SQL and strong knowledge of clean coding principles.

•

Strong understanding of Data Governance best practices (DQ handling, Lineage..etc)

•

Experience in handling various data ingestion patterns.

•

Strong accountability and Entrepreneurialism in day-to-day work.

•

Storing working experience with Data Orchestration (Airflow), Containerization (Docker, K8) and CI/CD tools.

•

Desirable to have working knowledge in Microsoft Azure and Databricks.

•

Advantageous to have experience working with Commodity/Energy trading data.

•

Demonstrated ability to work effectively both independently and as part of a collaborative global team following internal Data engineering standards.

•

Excellent problem-solving skills and a strong attention to detail.

•

Strong stakeholder communication skills.

What We Offer:

•

The expected base salary ranges from $120K -$180K. Salary offers are based on a wide range of factors including relevant skills, training, experience, education, and, where applicable, certifications and licenses obtained. Market and organizational factors are also considered. In addition to salary and a generous employee benefits package, successful candidates are eligible to receive a discretionary bonus.

Data Engineer

About This Job

Qualifications and Requirements:

What We Offer:

Similar Jobs

Data Analyst

Electrical Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Scientist Engineer

Data Software Engineer

Data Engineer 1

Data Engineer II

Engineering Data Specialist

Data Engineering, Specialist

Data Engineering, Specialist

Data Engineering Manager

Data Engineering Manager

Junior Data Engineer

Data Engineer II, TO

Trending Jobs

Assistant General Manager, Navy Yard Electric Utility

Electrical Engineer

Division Order Analyst

Accounts Payable Clerk

Division Order Landman

Professional Landman

contract landman

Oil and Gas Land and Title Analyst - SAM Associate II

Attorney

Contracts Administrator

Senior Landman

Title Landman

Electrical Designer

Landman

Title Reviewer

Oil and Gas Title Attorney

Civil/Structural Designer

HSE Operations Manager

E &amp; I - Office/Field Administration

Senior Division Order Analyst

E & I - Office/Field Administration