Visualizing Site Structure: LibreCrawl's New Interactive Graph Feature

A crawl table with thousands of rows tells you everything about individual pages and almost nothing about how they fit together. LibreCrawl's new Visualization tab turns your crawl data into an interactive graph built on Cytoscape.js, so you can see your site's architecture the way a search engine crawler experiences it: as a network of pages connected by links.

Why We Built a Graph View

Every crawler produces tabular data. URL, status code, title, depth, response time, one row per page. Tables are the right format for most audit work, and LibreCrawl's crawl table with its filters remains the place where you fix individual issues. But some of the most expensive SEO problems are structural, and structural problems are nearly invisible in a table.

Consider a cluster of 40 product pages that only link to each other, with a single weak link connecting them to the rest of the site. In a table, those pages look fine. They return 200, they have titles, they have internal links. You would only catch the problem by manually tracing link paths across hundreds of rows. In a graph, that cluster is immediately obvious: a clump of nodes hanging off the main structure by a single thread.

The same goes for depth problems. Our technical SEO audit checklist includes items like "important pages within 3 clicks from homepage" and "no orphaned pages". You can verify those with filters and exports, but a graph answers them in seconds because the shape of your architecture is the answer.

How the Visualization Tab Works

Open the Visualization tab during or after a crawl and LibreCrawl renders your site as an interactive graph. The building blocks are simple:

• Nodes are pages. Every crawled page becomes a node in the graph. The root page (the URL you started the crawl from) is drawn larger than the rest, so you always have a visual anchor for the center of your site.
• Edges are internal links. When one crawled page links to another crawled page, the graph draws an edge between them. The density and direction of these connections is your internal linking structure, made visible.

The graph is fully interactive. You can pan, zoom, and drag nodes around to untangle areas you want to inspect more closely. Clicking a node shows its details: the URL, HTTP status code, page title, and crawl depth. That last one matters more than people expect. Depth is the number of clicks from your root page, and it is one of the clearest signals of how discoverable a page actually is.

Status Codes at a Glance

Nodes are colored by HTTP status, which turns the graph into an instant health map:

• Green - 2xx success responses, the pages doing their job
• Blue - 3xx redirects
• Orange - 4xx client errors, mostly broken links and missing pages
• Red - 5xx server errors
• Gray - everything else

The colors do real work here. A healthy site renders as a mostly green graph with a few blue nodes scattered through it. A site with problems shows patterns: a red patch where a section is throwing server errors, orange nodes clustered around a template that links to deleted content, or a blue node with many edges pointing into it, which is a redirect that a lot of your pages still link to directly.

Live Updates While You Crawl

The graph updates in real time while a crawl runs. New nodes appear as pages are discovered and crawled, and edges fill in as LibreCrawl maps the links between them. Watching the graph grow is more than a novelty. You see the crawler work through your site level by level, which makes it obvious where your architecture branches, where it bottlenecks, and which sections the crawler reaches late.

If you have ever kicked off a crawl and wondered why it spent twenty minutes in a faceted navigation trap, watching the graph expand answers that question in a way no progress counter can.

Performance: The 500-Node Cap

Rendering graphs is expensive. Layout algorithms scale badly, and a browser asked to draw tens of thousands of nodes with physics-based positioning will grind to a halt. We were not willing to ship a feature that freezes the interface on exactly the large sites where structural analysis matters most.

So the Visualization tab caps the rendered graph at 500 nodes. When your crawl exceeds that, the graph shows the first 500 pages and tells you clearly that the view has been truncated. To be explicit about what this does and does not mean:

• Your crawl is not limited. LibreCrawl still crawls every page it finds. All data lands in the crawl table and exports as usual.
• Only the drawing is capped. The 500-node limit applies to what is rendered on screen, nothing else.
• You are told when it happens. No silently incomplete graphs. If you are looking at a truncated view, the tab says so.

For sites under 500 pages, which covers a large share of business sites, you see the complete picture. For bigger sites, the first 500 pages still capture the top of your architecture, which is usually where structural decisions are made and where they go wrong.

What to Look for in Your Graph

A graph is only useful if you know how to read it. These are the patterns we look for when reviewing a site's structure:

Orphaned Clusters

Groups of nodes connected to each other but barely connected to the main body of the site. These pages exist, they may even be indexed, but internal link equity barely reaches them. Common culprits: old campaign pages, blog archives after a redesign, and sections migrated from a previous CMS. In the graph they appear as islands or near-islands, and once you spot one, clicking its nodes tells you exactly which URLs are stranded.

Overly Deep Sections

Chains of nodes stretching far from the enlarged root node. Every extra hop from the homepage reduces both crawl priority and link equity. When you see a long tendril in the graph, click along it and check the depth values in the node details. Pages sitting at depth 6 or 7 that matter to your business are pages that need better internal links, not better content.

Redirect Hubs

A blue node with many inbound edges is a page that much of your site still links to even though it redirects somewhere else. Each of those internal links costs a redirect hop for users and crawlers. The fix is cheap (update the links to point at the destination), but finding these hubs in a table means cross-referencing your redirect list against your inlinks report. In the graph, they announce themselves.

Flat vs Deep Architecture

Step back and look at the overall shape. A flat site fans out wide from the root with most pages a few hops away. A deep site forms long branches with pages buried far from the center. Neither is automatically wrong, but the shape should match your intent. An e-commerce site that wants category pages ranking should not look like a narrow chain. A documentation site with a deliberate hierarchy should not look like a random tangle.

Graph First, Table Second

The workflow we have settled into is simple: use the graph to find structural problems, then switch to the crawl table to fix them. Spot an orphaned cluster in the graph, note the URLs from the node details, then use the table's filters to pull those pages up alongside their titles, status codes, and link data. The graph gives you the "where to look", the table gives you the "what exactly is wrong". Neither view replaces the other, and using both beats either one alone.

Works with Historical Crawls Too

The Visualization tab is not limited to live crawls. Load a past crawl from the dashboard and the graph renders from the stored data, exactly as it would during a live session. This pairs naturally with LibreCrawl's database persistence: crawl a site before a migration, crawl it again after, and load each crawl to compare the two graphs side by side. Structural regressions, like a section that got buried two levels deeper during a redesign, show up as a visibly different shape.

Conclusion

Site architecture has always been one of the hardest things to audit from tabular crawl data, and one of the most important items on any technical SEO checklist. The Visualization tab closes that gap. Nodes for pages, edges for links, colors for status codes, live updates while you crawl, and an honest 500-node rendering cap that keeps the interface fast without limiting your actual crawl.

Like everything in LibreCrawl, it is free and open source. Run a crawl, open the Visualization tab, and look at the shape of your site. Most people find something they did not know was there.

Key takeaways:

• The Visualization tab renders crawled pages as an interactive Cytoscape.js graph, with internal links as edges
• Node colors map to HTTP status: green 2xx, blue 3xx, orange 4xx, red 5xx, gray otherwise
• The graph updates live during crawls and works with historical crawls loaded from the dashboard
• Rendering caps at 500 nodes for performance, with a clear notice when truncated; crawling itself is never limited
• Use the graph to spot orphaned clusters, deep sections, and redirect hubs, then fix the details in the crawl table