Team Hexmos

Discover Hidden URLs and Endpoints with a Simple Bookmarklet

Sreedeep cv's photo
Sreedeep cv
·

3 min read

In the world of web development and cybersecurity, the ability to analyze a webpage’s structure and resources can be invaluable. Whether you're a developer debugging your site or a security professional investigating potential vulnerabilities, a tool to extract and visualize URLs and endpoints on a page is a powerful asset. Enter the BlackLine Scanner Bookmarklet, a simple yet effective script that brings this capability to your browser.

What is a Bookmarklet?

A bookmarklet is a small JavaScript program that can be executed directly from your browser's bookmarks bar. Unlike traditional browser extensions, bookmarklets require no installation or configuration. You create a bookmark, paste the JavaScript code as the URL, and click it whenever needed. This simplicity makes bookmarklets versatile for on-the-go tasks like scanning a webpage.

What Does This Bookmarklet Do?

The BlackLine Scanner bookmarklet is designed to scan any webpage for URLs and endpoints. Once activated, it creates a stylish terminal-like panel at the bottom of the page. This panel identifies and lists:

  1. Links from <a> tags.

  2. Resources from <img>, <script>, <link>, and <form> elements.

  3. Embedded URLs in inline JavaScript code and HTML attributes.

  4. Network resources using the browser's performance API.

The script automatically removes duplicates and presents the results in a neat, scrollable interface, making it easy to identify important links or exposed endpoints.

Here is the script(I found this in twitter don’t know the exact source) :

javascript:void(function(){let e=document.createElement('div');e.style.cssText='position:fixed;bottom:0;left:0;width:100%;height:300px;background:#1a1a1a;color:#00ff00;z-index:999999;padding:20px;overflow:auto;font-family:monospace;';e.innerHTML='<h3%20style="color:#00ff00">%F0%9F%94%8D%20BlackLine%20Scanner</h3><div%20id="results">Scanning...</div>';document.body.appendChild(e);let%20currentDomain=window.location.hostname;let%20foundUrls=new%20Set();function%20findUrls(){let%20urls=[];let%20sources=[...document.getElementsByTagName('a'),...document.getElementsByTagName('script'),...document.getElementsByTagName('img'),...document.getElementsByTagName('link'),...document.getElementsByTagName('form')];sources.forEach(element=>{if(element.href)urls.push(element.href);if(element.src)urls.push(element.src);if(element.action)urls.push(element.action);});let%20content=document.documentElement.innerHTML;let%20urlPattern=/(?:url\(|href="|src="|action="|url:|endpoint:|path:|route:)\s*['"]?([^'"\)\s>]+)/gi;let%20match;while((match=urlPattern.exec(content))!==null){if(match[1]&&!match[1].startsWith('data:'))urls.push(match[1]);}let%20scriptPattern=/"[^"]*"|'[^']*'/g;let%20scripts=document.documentElement.innerHTML.match(scriptPattern)||[];scripts.forEach(script=>{let%20urlMatches=script.match(/(?:\/[a-zA-Z0-9_-]+)+(?:\.[a-zA-Z0-9]+)?/g)||[];urlMatches.forEach(url=>urls.push(url));});performance.getEntriesByType('resource').forEach(entry=>urls.push(entry.name));return[...new%20Set(urls)];}let%20allUrls=findUrls();allUrls.sort();document.getElementById('results').innerHTML=`%20<div%20style="margin:10px%200;color:#00ff00">%E2%9C%85%20Found%20${allUrls.length}%20URLs%20&%20Endpoints%20on%20${currentDomain}</div>%20<div%20style="background:#2a2a2a;padding:10px;border-radius:5px">%20${allUrls.map(url=>`<div%20style="color:white;margin:5px%200;padding:5px;background:#333;border-radius:3px;word-break:break-all">${url}</div>`).join('')}%20</div>`;})();

Create a new bookmark and copy paste this in the URL bar.

How to Use It

  1. Create a new bookmark in your browser and paste the script (prefixed with javascript:) into the URL field.

  1. Navigate to any webpage you want to scan and click the bookmarklet.

  1. A panel will appear at the bottom of the page displaying all discovered URLs.

Why It’s Useful

  • For Developers: Debug your webpage by checking all links and resources in one place.

  • For Security Professionals: Discover potential vulnerabilities by finding hidden or unused endpoints.

  • For Researchers: Extract resources and URLs for web scraping or analysis.

The script can be tailored to fit specific needs. You can adjust the styling, filter URLs by pattern, or even export the results for further analysis.

Final Thoughts

With just one click, you can gain valuable insights into any webpage’s structure, resources, and potential vulnerabilities. Whether you're coding, testing, or exploring, this bookmarklet is a must-have for your toolkit.

I am currently working on LiveAPI; super convenient API documentation for teams. If you have started with a project and have no idea what APIs you have build or procrastinate over not finding time to create API documentation, then you are at right place.

bookmarklet