S

Technical SEO Software

Siteliner Website Crawler Review

siteliner.com · checked 15 Oct 2026

Siteliner scans public websites to find duplicate material, broken links, and other quality issues.

How it plugs in

YesPublic API
NoOpen source
YesFree tier

Signal on Technical SEO Software7.5#2 of 29

Siteliner offers web and API tools for assessing website quality. Within a site, it measures internal text overlap, marks broken internal links, derives link-based Page Power, and logs pages it did not scan. Free use allows a 250-page scan every 30 days. Premium runs on prepaid credits and adds CSV downloads, XML sitemap creation, saved results, advanced controls, and API access.

Siteliner home page

Review

In a line

A Siteliner crawl reveals internal text duplication, broken links, and related website-quality problems.

You can use the service from its website or through an API.

What it does

Within one website, Siteliner compares pages and marks matching text, excluding shared menus, navigation, and other common material. Its internal-link check points out pages whose links lead to missing or moved destinations.

Site and page results are included in its reports. Page views show duplicate matches, incoming and outgoing internal links, plus outgoing external links. The crawler lists pages left out because of canonical tags, redirects, robots directives, noindex, nofollow, or retrieval errors.

A Page Power view uses internal linking patterns to identify pages that appear prominent to search engines. Reports may include related subdomains and options to scan them. Cached versions of scanned pages are retained for 60 days.

How it connects

Siteliner operates through the web and exposes a Premium API. Developers can start scans and receive JSON output, and the documentation includes PHP and Python examples. Scan-start calls support as many as 30,000 pages and four concurrent retrieval threads.

Site reports and the table on screen can be exported as CSV, and the service can produce a standard XML sitemap. Siteliner is proprietary rather than open source.

Only publicly reachable pages can be read unless the site allows requests without login from Siteliner's crawler servers. The bot fetches HTML only, excluding embedded images and videos, and sends no more than four requests simultaneously. A robots.txt rule aimed at the Siteliner user agent can block it.

Good fit

Siteliner suits site owners and SEO practitioners who want a focused check of internal text duplication and broken links. It also supplies page-level link information, reasons for skipped pages, cached copies, and downloadable reports.

Its focused scope may appeal to people browsing technical SEO software or SEO tools who want a crawler instead of a broad marketing platform.

Watch for

The crawler can inspect only pages reachable without logging in, so verify access rules before using results for restricted areas.

Each crawl accepts at most 30,000 pages. Summary reports stay available for 185 days; highlighted broken links, highlighted duplicate content, and cached pages remain for 60 days. API documentation describes six-month availability for summary and site reports, with page-level and cached reports retained for 60 days.

Its license is revocable and restricted to one user, and analysis is allowed only for sites you own or are explicitly permitted to inspect. The terms also forbid automated querying and provide the service as-is without accuracy or fitness warranties.

Pricing

The Siteliner Free plan costs nothing and allows up to 250 pages per 30-day period.

Premium is purchased as prepaid credits at a stated one-time price of $0.01. Each scanned page costs $0.01, as does every API request that retrieves results. Premium supports scans of up to 30,000 pages, unlimited scan frequency, saved results, and advanced controls.

Verdict

Siteliner makes sense when the priority is reviewing one site's internal content overlap, link condition, and crawl output without buying a broad suite. The free allowance suits smaller or occasional checks, while prepaid Premium credits support larger or repeated scans.

Consider another tool if you require private-page access, media retrieval, or crawls beyond 30,000 pages. Check the one-user license and permission requirements before scanning.

Our take

Siteliner is a practical option for focused website housekeeping. Its free tier gives smaller sites a recurring way to review duplicate text and broken internal links, while Premium adds larger scans, saved results, and API access. Constraints include access limited to public pages, a 30,000-page crawl ceiling, finite report retention, and HTML-only crawling. Teams should also consider the one-user license, permission requirement, and prohibition on automated querying.

  • The no-cost plan allows 250 pages every 30 days.
  • Detects internal text duplication while excluding common menu material.
  • Checks broken internal links and gives reasons for skipped pages.
  • Offers CSV exports, XML sitemaps, cached pages, and page-level link reports.
  • The Premium API returns JSON and provides PHP and Python sample code.
  • Scanning is limited to publicly reachable pages unless access is expressly allowed.
  • A single crawl cannot exceed 30,000 pages.
  • Embedded images and videos are not fetched.
  • Highlighted and cached reports remain available for only 60 days.
  • Terms restrict use to one user and disallow automated querying.
  • Pick it ifSite owners checking duplicate content and broken internal links; SEO practitioners who need page and link reports; Developers seeking scan control and JSON-result API access
  • Skip it ifYou need login-protected pages scanned without special access.; Your site needs a crawl larger than 30,000 pages.; You need media retrieval or a broad SEO suite.

Pricing

The free plan allows up to 250 pages once every 30 days. Premium uses prepaid credits: scanning a page costs $0.01, and each request for API results costs another $0.01; scans can reach 30,000 pages with unlimited frequency.

  • Siteliner Freeup to 250 pages · once every 30 daysFree
  • Siteliner Premiumprepaid credits; $0.01 per page scanned and $0.01 per API result-retrieval request · up to 30,000 pages · unlimited scan frequency · saved results · advanced scan controls$0.01 one-time

As siteliner.com lists them, checked 15 Oct 2026.

Questions

Is Siteliner free?

Yes. Siteliner Free costs nothing and allows a scan of up to 250 pages every 30 days.

What does Siteliner check?

It examines internal text overlap, broken internal links, omitted pages, and link relationships between pages.

Can Siteliner scan private pages?

Access is possible only for publicly reachable pages, or where the site permits non-logged-in requests from Siteliner's crawler servers.

Does Siteliner have an API?

Yes. Premium can start scans and return results in JSON, with official examples for PHP and Python.

How much does Premium cost?

Premium uses prepaid credits, with a $0.01 charge per scanned page and another $0.01 for each API request that fetches results.

How long are reports kept?

Summary and site reports remain available for about six months. Highlighted duplicate content, highlighted broken links, page-level reports, and cached pages remain for 60 days.

Can I download Siteliner results?

Yes. Complete site reports and the table currently displayed can be downloaded as CSV files, and Siteliner can create a standard XML sitemap.