Web Scraping Project Ideas with Source Code (Python and No-Code, 2026)

Web scraping is one of the best ways to learn real programming: you deal with messy data, broken HTML and websites that fight back. It also makes great portfolio projects, because the result is useful on day one.

The problem: the classic tutorial (requests + BeautifulSoup) breaks on most modern websites. Prices load with JavaScript, content sits behind logins, and anti-bot checks block simple scripts.

This guide gives you 20 web scraping project ideas with source code, from beginner Python scrapers to AI agents that handle the hard sites for you. For each one, we show which approach fits: classic Python or TinyFish.

Last checked: October 2026.

TL;DR

• Beginner: scrape one simple site and save the data (prices, listings, scholarships).
• Intermediate: compare many sites at once, deal with JavaScript-heavy pages.
• Advanced: multi-step sites, logins, monitoring pages for changes over time.
• Source code: most projects link to open-source code in the TinyFish Cookbook on GitHub.
• Build it free: students get $35 in their TinyFish wallet at tinyfish.ai/students.

Classic Python vs TinyFish: which should you use?

TinyFish is one API that lets AI agents use the live web. Behind it:

• Fetch: give it a URL, get clean page content back (great instead of BeautifulSoup).
• Agent: describe a goal in plain English ("find the price and stock on this page") and it browses the site for you, clicks and all.
• Browser: a remote browser you control with Playwright when you want full control.
• Monitor: track a page on a schedule and get told when it changes.

The same task, two ways

Classic Python (works on simple static pages):

import requests
from bs4 import BeautifulSoup

html = requests.get("https://scrapeme.live/shop").text
soup = BeautifulSoup(html, "html.parser")
for product in soup.select("li.product")[:2]:
    name = product.select_one("h2").text
    price = product.select_one(".price").text
    print(name, price)

TinyFish Agent (no selectors; also works on JavaScript-heavy sites):

import requests

resp = requests.post(
    "https://agent.tinyfish.ai/v1/automation/run-sse",
    headers={"X-API-Key": "YOUR_API_KEY"},
    json={
        "url": "https://scrapeme.live/shop",
        "goal": "Extract the first 2 product names and prices",
    },
    stream=True,
)
for line in resp.iter_lines():
    print(line.decode())

Tip: start with the classic version to learn how HTML works, then switch to TinyFish when a site uses JavaScript or logins. See the TinyFish docs for Fetch, Agent, Browser and Monitor.

Beginner web scraping projects

1. Price tracker for one product

Scrape a product page every day and log the price to a CSV. Add an alert when it drops.

Approach: classic Python on simple shops; TinyFish Fetch or Monitor for modern stores.

2. Scholarship finder

Collect open scholarships for your field and region from scholarship sites.

Approach: TinyFish Agent (many different sites). Source code: scholarship-finder

3. Summer school finder

Scan university websites for summer programmes that match your subject.

Approach: TinyFish Agent. Source code: summer-school-finder

4. Tutor finder

Compare tutors for an exam across tutoring platforms.

Approach: TinyFish Agent. Source code: tutor-finder

5. Anime / TV streaming checker

Find which streaming platforms have a show right now.

Approach: TinyFish Agent. Source code: anime-watch-hub

6. News headline collector

Pull headlines from 3–5 news sites into one daily digest.

Approach: classic Python for simple sites; TinyFish Fetch for clean text from any page.

Intermediate web scraping projects

7. LEGO (or sneaker) restock hunter

Check stock and price across retailers for sold-out items.

Approach: TinyFish Agent. Source code: lego-hunter

8. Open-box deal finder

Search 8 retailers' open-box and refurbished sections at once.

Approach: TinyFish Agent. Source code: openbox-deals

9. Game price comparison

Compare game prices across 10 stores plus price history.

Approach: TinyFish Agent. Source code: game-buying-guide

10. Food delivery comparison

Find the best wings (or any dish) across delivery apps and review sites.

Approach: TinyFish Agent. Source code: wing-command

11. Rental listings tracker

Pull live rental listings and neighbourhood info to find a fair price.

Approach: TinyFish Agent. Source code: district-rent-shark

12. Pharmacy price comparison

Compare medicine prices across pharmacy chains.

Approach: TinyFish Agent. Source code: pharmacy-panic

13. Deal-alert Discord bot

A Discord bot that finds discounted collectibles across sites.

Approach: TinyFish Agent + a Discord bot. Source code: waifu-deal-sniper

14. Restaurant allergy checker

Compare restaurants by allergens and dietary needs before you go.

Approach: TinyFish Agent. Source code: restaurant-comparison-tool

Advanced web scraping projects

15. Research paper finder

Search 8 academic portals in parallel and collect verified paper details.

Approach: TinyFish Agent. Source code: research-sentry

16. Competitor pricing dashboard

Track competitors' pricing pages and show a side-by-side comparison.

Approach: TinyFish Agent + Monitor. Source code: competitor-analysis

17. Competitor research CLI

Ask plain-English questions about competitors from your terminal.

Approach: TinyFish Agent. Source code: competitor-scout-cli

18. Government tender tracker

Scrape tender portals in parallel and list new opportunities by sector.

Approach: TinyFish Agent + Monitor. Source code: tenders-finder

19. Supply chain monitor

Watch port congestion, carrier advisories and prices for changes.

Approach: TinyFish Monitor. Source code: logistics-sentry

20. Semiconductor stock tracker

Track chip availability across distributors.

Approach: TinyFish Agent. Source code: silicon-signal

How to start your first scraping project

1. Pick one idea and open its source code.
2. Get free credits: sign up at tinyfish.ai/students with your school email and a password for $35 in your TinyFish wallet.
3. Try it without code first in the TinyFish Playground. Our 15-minute guide shows you how.
4. Run it locally, then change one thing: a different site, city or product.
5. Ship it: deploy it, post a demo, and submit it as a bounty on TinyBounties for a gift card.

Want more ideas? See 25 AI project ideas for college students.

Keep reading

• What Is an AI Agent? Explained for Students (and How to Build One)
• How to Win a Hackathon with AI: Use TinyFish to Supercharge Your Project (2026)
• Best AI Agents for Students in 2026 (Free and Paid)

Follow TinyFish for new guides and student builds: X (@Tiny_Fish) · LinkedIn (TinyFish) · Instagram (@tinyfish_ai)