BizKitHub
DocsAPI ReferenceVikitron/api/v1/vikitron/robots-txt-checker
getVikitronPublic API v1

/api/v1/vikitron/robots-txt-checker

Returns the version history of robots.txt snapshots for the given domain. Cached responses (5 min fresh + 1 h SWR) skip the outbound HTTP request — refresh cadence is driven by scout. If no snapshots exist yet OR ?force=true is passed, a fresh fetch is triggered before returning results (uncached).

vikitrongetApiV1VikitronRobots-txt-checker

Parameters

2 query

Query parameters

· 2
domainstringRequired

Domain name to check.

Exampleexample.com
forcestringOptional

Set to "true" to bypass the cache and trigger a fresh fetch.

Exampletrue

Response schema

1 status code documented

200SuccessRobots.txt version history.
recordsobject[]Required

Version history of robots.txt snapshots (newest first).

Each array item:
idstringRequired

Hashed version identifier.

contentstringRequired

Raw robots.txt content.

httpCodenumberRequired

HTTP status code when fetching robots.txt.

Examples200404
firstSeenDate | string | string | numberRequired

When this version was first observed.

One of 4:
Variant 1
Date

When this version was first observed.

Variant 2
stringdate-time
Variant 3
stringdate
Variant 4
number
updatedDateDate | string | string | numberRequired

When this version was last confirmed.

One of 4:
Variant 1
Date

When this version was last confirmed.

Variant 2
stringdate-time
Variant 3
stringdate
Variant 4
number

Response example

application/json
{
  "records": [
    {
      "id": "example_id",
      "content": "example_content",
      "httpCode": 200
    }
  ]
}

Request example

GET /api/v1/vikitron/robots-txt-checker

get
curl -X GET "https://api.bizkithub.com/api/v1/vikitron/robots-txt-checker?domain=example.com&force=true" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY"

Need an API key?

All BizKitHub public API endpoints require authentication via API key.

Get API Key