We need an experienced Python developer with expertise in web scraping, machine learning, and automation to review, optimize, and run an existing domain crawler. The goal is to improve efficiency, accuracy, and reliability.
This project will be conducted in two stages:
Stage 1: Successfully Run the Existing Crawler Your first task is to set up and successfully run the crawler to validate that you can: - Install and configure all dependencies - Execute the crawler on a dataset of domain names - Generate valid output - Troubleshoot any initial setup issues
Once this is successfully demonstrated, we will move to Stage 2.
Stage 2: Propose a Better Solution or Manage Ongoing Runs - Depending on your expertise and analysis of the current crawler, you will either:
A) Propose & Implement Improvements (Rewrite or Optimize the Crawler) - Identify inefficiencies and suggest a better architecture - Improve performance & accuracy (e.g., async scraping, multiprocessing, better ML model) - Ensure scalability if running on large datasets - Refactor code for long-term maintainability OR B) Run & Maintain the Crawler Monthly - Execute the crawler on a scheduled basis - Monitor accuracy and adjust as needed - Fix bugs & ensure stability - Implement new features & improvements over time - Provide regular reports on data quality and insights
About the current crawler The current crawler is built in Python and classifies domain names based on their content (e.g., parked, active, expired). It utilizes: - Requests / Aiohttp (Web scraping) - Selenium (Browser automation) - BeautifulSoup (HTML parsing) - Langdetect / NLP (Language detection & text classification) - XGBoost (Machine learning classification)
Ideal Candidate: - Strong Python skills (async programming, OOP) - Web scraping & automation expertise (requests, Selenium, BeautifulSoup) - Machine learning experience (XGBoost, Scikit-learn, NLP) - Familiarity with databases (SQL/NoSQL) for storing results - Cloud/DevOps experience (if needed for large-scale deployment) - Ability to work independently and ensure high-quality results - High level in English
Project Timeline & Budget: Deliverables expected within 2-4 weeks Budget: Open to discussion based on experience & estimated work required.
Detailed 3D Keyring Design Category: 3D Animation, 3D CAD, 3D Design, 3D Graphic Design, 3D Layout, 3D Modelling, 3D Printing, 3D Rendering Budget: $10 - $30 AUD
27-Apr-2026 10:04 GMT
Amazon Specialist for Luxury Brand Launch Category: Amazon, Brand Management, Digital Marketing, Internet Marketing, Marketing, PPC Marketing, Sales, SEO Budget: ₹1500 - ₹12500 INR
27-Apr-2026 10:03 GMT
LinkedIn Ads Lead Generation Campaign Category: Advertising, Bulk Marketing, Content Marketing, Digital Marketing, Facebook Marketing, Internet Marketing, Lead Generation, Marketing, Marketing Strategy, Social Media Marketing Budget: €8 - €30 EUR
Canva Social Media Post Designs Category: Branding, Canva, Corporate Identity, Graphic Design, Photoshop, Photoshop Design, Social Media Marketing, Visual Design Budget: $15 - $25 USD
Professional Instagram Travel Reel Category: Adobe Premiere Pro, Content Creation, Final Cut Pro, Video Editing, Video Production, Video Services, Videography Budget: ₹600 - ₹1500 INR
Social Media and Content Management for Sada Category: Content Creation, Content Strategy, Graphic Design, Social Media Management, Social Media Marketing, Twitter, Video Editing, Videography Budget: $10 - $1000 USD
27-Apr-2026 09:53 GMT
Write "The Body Speaks" Book -- 2 Category: Academic Writing, Book Writing, Content Writing, Creative Writing, Editing, Fiction, Ghostwriting, Illustration, Proofreading, Writing Budget: ₹12500 - ₹37500 INR
