Senior Python Data Scraping Engineer – Freelance

About the role

Mindrift is seeking a Senior Python Data Scraping Engineer to join the Tendem project. As an AI Pilot, you will work remotely on part‑time contracts, combining your expertise in web scraping with AI‑assisted tools to deliver high‑quality, structured datasets.

Key responsibilities

• Design, implement and own end‑to‑end data extraction workflows for complex, dynamic websites.
• Utilize internal tools such as Apify and OpenRouter, as well as custom Python scripts, to accelerate collection, validation and delivery of data.
• Handle JavaScript‑rendered content, AJAX calls, infinite scroll and other interactive elements.
• Enforce data quality through validation checks, cross‑source consistency and formatting standards.
• Scale scraping operations using batching or parallelisation, monitor failures and adapt to minor site changes.

Required profile

• Minimum 5 years of experience in data engineering, web scraping or software development.
• Bachelor’s or Master’s degree in Engineering, Computer Science, Applied Mathematics or related field (optional but a plus).
• Strong analytical mindset, detail‑oriented and able to work independently.
• Comfortable working with large language models (LLMs) and AI‑assisted workflows.

Required skills

• Python programming.
• Web‑scraping libraries such as BeautifulSoup and Selenium.
• Experience with dynamic content handling (JS, AJAX, infinite scroll).
• API integration and use of proxies.
• Data cleaning, normalization and validation.
• Output formats: CSV, JSON, Google Sheets.
• Familiarity with Apify and OpenRouter platforms.