In DevTools you hunted for hidden flags by hand. Now you'll write a small Python script that does the same thing automatically, scanning any web page in milliseconds and printing every secret it finds.
A short Python script that fetches any web page and scans it for hidden info:
FLAG{...} strings developers forgot to deletetype="hidden" that contain user IDs, tokens, debug flags?debug=true, ?page=admin etc.By the end, you'll point your script at the Candy Shop and watch it dump every flag in seconds — the same flags you spent ages hunting for in DevTools.
Students often hit this one confusion early. The first few missions in the notebook don't fetch a real website. They give you a tiny pretend HTML page already typed out, stored in a variable called target_html. Here's why, and here's what to do.
target_html = '''<!DOCTYPE html>...''' — that's normalIt's a mini fake e-commerce page hard-coded into the cell. You don't need internet, you don't need to host the Candy Shop, and the same HTML loads every time. The missions then run regex on this string to teach you the patterns.
Why this matters: if requests.get() failed for any reason (offline, slow network, wrong URL), the whole notebook would break before you even got to learn regex. The hardcoded HTML guarantees there's something to work with.
target_html — the hardcoded HTML string. You write small regex patterns and verify they match the right things. No internet needed.
target_html = '''...''' for target_html = requests.get(url).text. Now your same regex code works on real public sites. The URL you pick is up to you (or your teacher).
Every mission cell has a placeholder line like:
Your job is to replace the ___ with a complete line of Python. Each cell tells you:
<!--(.*?)-->re.findall(pattern, target_html, re.DOTALL)len(comments))The pattern shown in the hint is just one piece of the full line. You can't paste it on its own — Python doesn't speak HTML and you'll get SyntaxError: invalid syntax.
re.findall(...) with where to look and the flag.The mental model: the pattern is a tool. re.findall is the worker who uses the tool. You have to tell the worker WHAT tool to use (pattern), WHERE to look (target_html), and how (re.DOTALL). Just handing them a tool with no instructions doesn't work.
Try it on your own first — getting stuck is part of learning. But here are the exact lines if you need a nudge:
Pattern to read these: re.findall(pattern, where_to_look, optional_flags). The pattern is the bit in the r"..." string. r just means "raw string — backslashes are literal".
Don't worry about understanding this yet — the notebook walks you through every line. This is just so you know what to expect.
~6 lines of real code does what you did manually in DevTools all session. That's the power of automating the boring stuff.
localhostColab runs on Google's servers, not on your laptop. So URLs like http://localhost:8000/CandyShop_Website.html will not work — Google's servers can't reach your machine.
If your code prints "Found 0" for everything, this is almost certainly why. Your html variable will be empty (or an error message), so the regex finds nothing.
What to use instead:
https://ck-cybersecurity-01.web.app/CandyShop_Website.html)https://books.toscrape.com — it's public, fast, and full of HTML comments to findprint(len(html)) — a big number means you got the page; 0 or a short number means the URL didn't workPython doesn't run inside web browsers directly, so we need somewhere to run it. Google Colab is the easiest choice:
CodeKids_HTML_Detective_Student.ipynb.
localhost, that won't work — Colab can't see your machine. Use the live Candy Shop URL your teacher provides.print(html[:1000])) to make sure you got the page. Then test your regex on a few characters of that output.The script is just requests.get(url) + some regex. Change the URL and you've got a brand-new investigation. Here are some safe, legal practice targets:
<a> tags or extract every section heading. Bigger pages = more fun patterns.
→ Visit
<a href="http...).robots.txt. Add /robots.txt to any URL to see what the site asks scrapers NOT to touch. Respect it..toscrape sites and OWASP Juice Shop exist specifically so you don't have to.requests library to fetch any web pageOnce you've worked through the student notebook on your own — even if you didn't finish every mission — you can grab the fully completed version to compare your code against, or to learn the bonus tricks.
scan_page() function that bundles all 3 regexes into one toolbooks.toscrape.com — same code, real data