Senior Data Engineer – AI Data Infrastructure & Automation

Common Forge Ventures

Salary: Gross salary $3000 - 5000
Type: Tiempo completo

Tags: Python SQL Cloud Computing Automation

Common Forge Ventures is seeking a Senior Data Engineer to join one of our venture studio startups, a high-growth company building the world’s most robust, contextualized buyer intelligence platform for investment banks, private equity, and strategic acquirers. You will own and evolve the data infrastructure that ingests, cleanses, and enriches large-scale data from APIs, web scraping agents, LLM outputs, and internal datasets. This role focuses on scalable pipelines, data governance, AI agent orchestration, and seamless integration with product teams to deliver reliable data APIs and feeds that power advanced buyer insights.

Apply to this job from Get on Board.

Key Responsibilities

Data Architecture & Pipelines: Design and optimize large-scale ETL/ELT pipelines across structured and semi-structured data sources. Implement data validation, normalization, and reconciliation frameworks to ensure data integrity. Automate ingestion and transformation processes using orchestration tools.
AI Agent Orchestration & Integration: Develop, train, and refine LLM-based data agents to collect, clean, update, and infer new attributes for buyer profiles. Build robust prompting architectures, JSON schema validation, and feedback loops to ensure high-quality structured outputs. Collaborate to optimize agent reasoning, reduce hallucinations, and integrate embeddings for context-aware inference.
Data Governance & Quality: Define standards for data versioning, lineage, and observability. Build quality-control layers for agent-generated data, including confidence scoring, human-in-the-loop validation, and automated correction mechanisms. Ensure compliance with data governance, privacy, and security requirements.
Collaboration & Product Integration: Work cross-functionally with AI, product, and engineering teams to deliver data APIs and feeds. Partner with leadership to prioritize data reliability, scalability, and innovation. Lead ongoing improvements to the data infrastructure roadmap.

Required Skills & Experience

5+ years in data engineering (or similar) with strong experience in Python (Pydantic a bonus), SQL, and orchestration frameworks. Proficiency with cloud data stacks (including GCP). Demonstrated ability to design or integrate AI/LLM agents for data collection, enrichment, or inference tasks. Track record of building structured data pipelines from unstructured sources, with error recovery, validation, and monitoring. Deep understanding of prompt engineering, function calling, embedding-based retrieval, and data labeling workflows. Excellent communication, documentation, and cross-team collaboration skills. You care deeply about your craft and maintain clean, scalable projects even when not directly supervised.

Desirable Skills & Experience

Familiarity with LangChain or other agent frameworks (e.g., LlamaIndex, Haystack), tool-calling, and similar agent orchestration technologies. Experience with data quality scoring, schema evolution, and metadata management (dbt, Great Expectations, etc.). Background in investment data, market intelligence, or deal-sourcing platforms. Experience leading small data or engineering teams.

Benefits & Perks

From Common Forge Enterprises, you will enjoy a remote working environment with flexible hours, enabling you to balance personal and professional commitments effectively. We prioritize opportunities for professional development and growth, fostering an inclusive and collaborative team culture. Join us to access the latest technologies and tools as we work toward impactful solutions together! 🚀

Fully remote You can work from anywhere in the world.

POSTULAR VÍA WEB

Source: GetOnBoard | Main Category: Other