Solution Architect – Web Scraping & Data Engineering
Job Description
1. Role Summary :
We are seeking a senior Solution Architect to lead the design and delivery of large-scale web scraping and data extraction solutions, with strong supporting expertise in data engineering. The architect will be the technical authority on all scraping initiatives – from pre-sales solutioning and effort estimation through to architecture, build, and stabilization. The primary mandate is web scraping: designing resilient crawlers, anti-bot strategies, distributed extraction systems, and compliance frameworks. The secondary mandate is data engineering: ensuring extracted data flows reliably into well-modelled, query-ready storage layers and downstream analytical or operational systems.
2. Pre-Sales, Proposal & Estimation Responsibilities :
This is a non-negotiable part of the role. The Solution Architect is mandatorily involved in every new scraping opportunity from the proposal stage onward, and is jointly accountable with the sales / delivery leadership for the technical correctness of all proposals.
Mandatory Involvement Areas :
– Author or co-author the technical solution section of every new RFP / RFI / proposal document for scraping projects.
– Lead technical discovery calls with prospective clients to understand target sources, data SLAs, volume, and compliance constraints.
– Conduct target-site feasibility assessments – anti-bot complexity, dynamic content, login walls, geo-restrictions, rate limits – and document findings before commercials are committed.
– Own end-to-end effort estimation for scraping engagements: crawler build effort, infrastructure sizing, proxy and CAPTCHA cost projections, maintenance overhead, and contingency.
– Produce solution architecture diagrams, tech stack recommendations, and assumption logs as part of every proposal.
– Define and document SLAs, KPIs, and acceptance criteria proposed to the client.
– Participate in client orals, technical defence sessions, and commercial negotiations as the technical SPOC.
– Maintain an internal estimation knowledge base – reusable estimation templates, complexity matrices, target-site classification, and historical actuals – and continuously refine it after every project closure.
– Sign off on the technical feasibility and risk profile of every proposal before submission. No scraping proposal goes out without architect approval.
Are you interested in this position?
Apply by clicking on the “Apply Now” button below!
#GraphicDesignJobsOnline
#WebDesignRemoteJobs
#FreelanceGraphicDesigner
#WorkFromHomeDesignJobs
#OnlineWebDesignWork
#RemoteDesignOpportunities
#HireGraphicDesigners
#DigitalDesignCareers
# Dynamicbrand guru
