Documentation

Urlgate is parental DNS with teeth: a multi‑million‑domain mega-list, per-device filters, Force SafeSearch (so Google Images can’t be the loophole), and JSON APIs for lookups and WebView navigation. This guide covers household setup, how policy works, every public and authenticated HTTP endpoint, and how operators run the service.

Quick start (parents)

1. Sign in with your email (magic link), then finish authenticator 2FA.
2. Open Filters and create a filter (or edit the household default). Leave Force SafeSearch on unless this device is for an adult who needs unrestricted search.
3. Tick the categories you want blocked (adult, social, games, bypass, …).
4. On an Android phone: Settings → Network → Private DNS → provider hostname → paste the filter’s Android hostname (no https://).
5. On other clients: paste the private DNS link (full HTTPS URL) into Secure DNS / DNS-over-HTTPS settings that accept a URL.
6. Optional: mark one filter as home Wi‑Fi default and point the router’s DNS at Urlgate when UDP DNS is enabled on the server.
7. Building a WebView app? Use the filter’s Navigate API URL (same token as Android / DoH) so in-app browsing follows that filter’s categories and SafeSearch toggle — or the open default navigate endpoint for a shared score-based policy.

What Urlgate is

Four products in one service:

1. Mega-list intelligence — millions of domains merged from trusted child-safety and threat feeds, scored for time-waste, checkable on the homepage or via the lookup API.
2. Parental DNS — each filter is a policy: which categories to block, allowlist exceptions, and whether to force SafeSearch / YouTube Restricted Mode. Phones and home networks use that policy as their DNS resolver.
3. Public lookup API — GET/POST /api/v1/lookup exposes the same mega-list check as the homepage (no key).
4. Navigate API — POST /api/v1/navigate (open default) and POST /api/v1/navigate/<token> (per filter) return the single URL a WebView should load. The Vig and other apps use keyed POST /api/v1/check for private classify.

Blocking policy always lives in Urlgate (Elixir), not in CoreDNS zone files. CoreDNS is only the recursive upstream for names that are allowed.

Force SafeSearch — the loophole closer

Classic blocklists stop known porn and social domains. They do not stop a child from typing something explicit into Google Images or browsing unrestricted YouTube. DNS cannot open Google and filter result rows — but the major search engines publish forced-safe endpoints. When SafeSearch is enabled on a filter, Urlgate answers DNS for those front doors by resolving the safe name instead, then returning the addresses under the name the browser asked for. The same rewrite applies to the navigate endpoints when SafeSearch is required (always on the open default; optional per filter).

What gets rewritten

• Google Search, Images, Video, News, and country domains (e.g. google.co.uk) → forcesafesearch.google.com
• YouTube (youtube.com, m.youtube.com, youtu.be, music, nocookie, related API hosts) → restrict.youtube.com
• Bing → strict.bing.com
• DuckDuckGo → safe.duckduckgo.com

What is deliberately not rewritten

• accounts.google.com, play.google.com, Maps, Drive, Gmail hosts, googleapis.com, gstatic.com — so logins and the Play Store keep working.

The toggle lives on each filter and defaults to on. Turn it off only for an adult device that needs unrestricted search — or for a WebView that should not rewrite YouTube/Google when calling /navigate/<token>. Allowlisting a search host also skips the rewrite for that host.

Filters

A filter is your household policy object. After sign-in, manage them at Filters.

What you configure

• Name — e.g. “Maya’s phone”, “Household Wi‑Fi”.
• Blocked categories — only domains tagged with those categories are sinkholed by DNS and blocked by the per-filter navigate endpoint (see below).
• Allow hosts — exceptions that always resolve / always echo on navigate, even if listed (e.g. a school portal that shares a category with something you block).
• Deny hosts — always block these domains (Pi-hole-style denylist), even if they are not on the mega-list. Allow still wins if a host is on both lists.
• Force SafeSearch — on by default; see previous section. Controls DNS rewrite and per-filter navigate rewrite.
• Enabled — off keeps the DNS link alive but applies no blocks/rewrites (useful while testing).
• Home network default — the filter used for plain LAN UDP DNS when that listener is enabled on the server. Only one filter can be the default.

What you get after save

All three use the same token (the random string in the Android hostname). Regenerating the hostname rotates that token everywhere.

• Android Private DNS hostname — short token + zone, e.g. m4k7xq2n9pvr3.0xbase.eu. Hostname only; used for DNS-over-TLS on port 853.
• Private DNS link (DoH) — full HTTPS URL including a path token for clients that accept a URL (/dns-query/<token>).
• Navigate API (WebView) — POST /api/v1/navigate/<token> — same token; returns the URL the in-app browser should load under this filter’s categories and SafeSearch setting.

Regenerate hostname issues a new token. Old Android hostnames, DoH links, and navigate URLs stop working until devices / apps are updated — treat it like rotating a password.

Set up phones & Wi‑Fi

Android (built-in Private DNS)

1. Open Settings → Network & internet → Private DNS (wording varies by OEM).
2. Choose Private DNS provider hostname.
3. Enter the filter’s Android hostname exactly — no scheme, no path, no trailing slash.
4. Save. The phone will use DNS-over-TLS to Urlgate on port 853. Traffic still goes to the real sites for allowed names; blocked names fail to resolve.

DNS-over-HTTPS (browsers & apps)

Paste the filter’s private DNS link into any client that accepts a custom DoH URL (Firefox secure DNS “custom”, some MDM profiles, specialist Android DNS apps). The path contains the filter token; do not share it publicly.

Whole-home Wi‑Fi

When UDP DNS is enabled on the Urlgate host, set the router’s DHCP DNS to that server’s IP and mark one filter as the home default. Devices without their own Private DNS / DoH then inherit that filter. Phones that still use a private link keep their own policy.

Production may leave UDP DNS off while Android DoT and DoH remain available — check with the operator if router DNS is not resolving yet.

In-app WebViews

DNS alone does not control URLs loaded inside an app WebView. Call the Navigate API before loadUrl (see Navigate API). Prefer the per-filter URL so the same policy as that child’s Android DNS applies.

How DNS decisions work

For every DNS query that hits Urlgate with a filter:

1. Normalise the name (lowercase, strip a leading www.).
2. If the name (or a parent) is on the filter’s allow list → allow (forward upstream).
3. Look up the mega-list. If any of the host’s categories intersect the filter’s blocked categories → block (answer 0.0.0.0 / sinkhole, short TTL).
4. If SafeSearch is on and the name is a known search/video front door → rewrite to the forced-safe target, resolve that, return addresses under the original name.
5. Otherwise → allow: forward to CoreDNS, clamp answer TTLs (~60s).

Short TTLs on allow/block mean filter edits take effect within about a minute instead of leaving devices cached on an old answer for hours.

Block categories & weights

Sources tag domains with one or more of these categories. On a filter, only categories you tick are sinkholed (DNS) or redirected to the block page (navigate). On the homepage / public lookup API, any listed category contributes to the response; time_waste_score is the highest matching weight (0–10).

The open default navigate endpoint does not use your ticks — it blocks when that highest score is greater than 6 (so weight‑6 and below — e.g. social, games, blogs, celebrity — are allowed there unless you use a per-filter navigate URL that ticks them).

Child devices usually keep adult and bypass on together — bypass covers DoH/VPN/proxy evasion and similar ways around DNS filtering. Tick ads for Pi-hole-style ad/tracker blocking (off by default on new filters — lists are large and can break a few sites). Vendor telemetry lists are optional (can break ecosystem features).

Mega-list & sources

Urlgate downloads and merges many public lists (HaGeZi, StevenBlack, Blocklist Project, Université Toulouse UT1 packs, OISD NSFW, phishing feeds, and more). Defaults are seeded from the child-safety CSV shipped with the app. Operators can add, silence, or remove sources in the site console.

• Lookups use Postgres indexes on full host and bare name (no multi‑GB RAM load).
• Nightly refresh runs around 02:15 UTC; console can queue an immediate refresh.
• Matching is label-exact after normalisation — not substring. A listed tiktok.com also matches www.tiktok.com and subdomains via parent-host logic in the cache layer.

Big platforms used as search engines (Google, Bing) are usually not on adult lists on purpose — use SafeSearch instead of blocking google.com wholesale.

Homepage checker

The public homepage lets anyone paste a URL or host and see verdict, score, categories, and how many sources matched — useful for sanity-checking before you rely on a filter. That check uses the mega-list only (not your private filter categories). Filter policy applies when devices use your DNS link or your per-filter navigate URL.

What DNS cannot do

• It does not inspect HTTPS page content. If a site is not listed and not rewritten, DNS will resolve it.
• A device that ignores your DNS (hard-coded 8.8.8.8, another DoH provider, a VPN that tunnels DNS) bypasses Urlgate. Keep bypass blocked and prefer OS Private DNS / DoH locked where you can.
• Apps that embed their own resolvers or connect by raw IP are outside DNS policy. Use the Navigate API for in-app WebViews.
• SafeSearch quality is owned by Google/Bing/YouTube — Urlgate forces their safe mode; it does not re-implement a search filter.

Accounts & 2FA

1. Request a magic link at login (link valid ~15 minutes, single use).
2. First visit: scan the TOTP QR in an authenticator app and confirm a code.
3. Later visits: email link (when required) + TOTP code.
4. Each account only sees its own filters.

Recovery codes are not shipped yet — back up your authenticator secret when enrolling. Operators can clear TOTP fields in the database as a last resort.

HTTP APIs overview

Public lookup and navigate are CORS-open (*). Fair use only — see Terms. Do not publish filter tokens.

Public lookup API

Anyone can check a domain against the mega-list — same result shape as the homepage checker. No API key.

GET /api/v1/lookup?url=https://www.tiktok.com

POST /api/v1/lookup
Content-Type: application/json

{"host":"tiktok.com"}

Example response:

{
  "host": "tiktok.com",
  "matched_host": "tiktok.com",
  "verdict": "block",
  "time_waste_score": 6,
  "categories": ["social"],
  "sources": ["hagezi_social"]
}

• Query params or JSON body: url or host (full URL or bare host).
• verdict — block / allow from mega-list categories (not your private filter ticks).
• time_waste_score — 0–10 from the highest matching category.
• This endpoint does not rewrite YouTube/Google or return a block-page URL — use navigate for WebView load decisions.

Navigate API (WebView)

POST a URL, get back only the URL the WebView should load. No action enum — always loadUrl(response.url). No API key (CORS open). Fail open on the client if the request errors (network / 4xx / 5xx): load the original URL or show your own fallback.

Contract: request {"url":"…"} → response {"url":"…"} (echo, SafeSearch rewrite, or block page).

Open default

POST /api/v1/navigate — shared policy for apps that do not attach a household filter token (e.g. a default child-mode WebView before sign-in).

POST /api/v1/navigate
Content-Type: application/json

{"url":"https://www.youtube.com/watch?v=abc"}

→ {"url":"https://restrict.youtube.com/watch?v=abc"}

{"url":"https://tiktok.com/@x"}
→ {"url":"https://tiktok.com/@x"}
   (social score 6 is not > 6 on open default)

{"url":"https://www.pornhub.com/video/1"}
→ {"url":"https://urlgate.0xbase.eu/blocked?detail=Blocked+because+of+adult"}

Decision order:

1. Optional body allow_hosts (list of hosts) → echo unchanged
2. SafeSearch / YouTube Restricted rewrite — always on for this endpoint
3. Mega-list time-waste score > 6 → block page (/blocked?detail=… on this host by default)
4. Else → echo the input URL

Per-filter (same token as Android / DoH)

POST /api/v1/navigate/{token} — uses that filter’s blocked categories, allow-hosts, and Force SafeSearch toggle. If SafeSearch is off on the filter, YouTube/Google are not rewritten. Blocking is category intersection with your ticks — not the open score > 6 rule.

POST /api/v1/navigate/<token>
Content-Type: application/json

{"url":"https://www.tiktok.com/@x"}

→ {"url":"https://urlgate.0xbase.eu/blocked?detail=Blocked+because+of+social"}
   (when that filter blocks social)

→ {"url":"https://www.tiktok.com/@x"}
   (when that filter does not tick social)

Decision order:

1. Filter allow-hosts → echo
2. If Force SafeSearch is on → rewrite search/video front doors
3. Listed category intersects filter’s ticks → block page
4. Else → echo

Unknown token → HTTP 404. The Filters UI shows each filter’s navigate URL next to its DoH link. Block page base URL: env URLGATE_BLOCK_PAGE_URL (default https://urlgate.0xbase.eu/blocked).

Open default vs per-filter

Authenticated check API

For The Vig and private automations. Same payload as public lookup, but requires header X-Urlgate-Key (env URLGATE_API_KEY). Prefer this when you need a private integration channel or when operators want to revoke access without changing the public endpoint.

Check a URL or host

POST /api/v1/check
Content-Type: application/json
X-Urlgate-Key: YOUR_KEY

{"url":"https://www.tiktok.com"}

Bare hosts are accepted as url or host.

Health & stats

• GET /api/v1/health — liveness, no key: {"status":"ok"}
• GET /api/v1/stats — requires key; returns entry_count and last_refresh_at

The Vig is designed to fail open to its other classifiers if Urlgate errors — child browsing should not hard-brick solely because this service is briefly down.

DNS-over-HTTPS protocol notes

Filters expose RFC 8484 DoH at /dns-query/<token> (GET and POST). The token selects the filter. Android Private DNS does not use this URL — it uses DoT on port 853 with the hostname form {token}.0xbase.eu (zone from URLGATE_DOT_ZONE).

Architecture

Phone DoT / DoH / LAN UDP
    → Urlgate.Dns.Handler + Policy (+ SafeSearch)
        → block:   sinkhole 0.0.0.0 (short TTL)
        → rewrite: resolve forcesafesearch / restrict.youtube / …
        → allow:   CoreDNS :5300 → public recursive (TTL clamped)

WebView / app
    → POST /api/v1/navigate            (open: score > 6 + SafeSearch)
    → POST /api/v1/navigate/<token>    (filter ticks + SafeSearch toggle)
        → {"url": echo | rewrite | block-page}

The Vig (child classify)
    → POST /api/v1/check  (X-Urlgate-Key)
        → Lists.Cache (Postgres mega-list)

Anyone
    → GET/POST /api/v1/lookup
        → same mega-list shape as homepage checker

• DoT — TLS on an internal port; nginx stream proxies public :853. Filter chosen from TLS SNI hostname.
• DoH — HTTPS to the app; filter from URL path token.
• UDP — optional; uses the home-default filter.
• Navigate — HTTPS JSON; open default or path token → same mega-list / SafeSearch building blocks as DNS, but returns a URL instead of an A record.

Site admin console

Operators with admins.is_admin = true (set in Postgres) get:

• Overview — source stats, last refresh, recent checks
• Lists — coverage analytics
• Sources — add / edit / silence / remove / bulk-add remote lists
• Queries — search check history, filters, charts (retention on the order of tens of thousands of rows)

UPDATE admins SET is_admin = true WHERE email = 'you@example.com';

Operations & environment

Common release tasks

• Migrate: bin/urlgate eval "Urlgate.Release.migrate()"
• Refresh lists on a live node: bin/urlgate rpc "Urlgate.Release.refresh_lists()"
• Laptop deploy: bin/build.sh && bin/deploy.sh
• CoreDNS sidecar: deploy/coredns (compose, typically port 5300)

Important environment variables

• DATABASE_URL, SECRET_KEY_BASE, PHX_HOST, PORT
• URLGATE_API_KEY — check + stats API
• URLGATE_BLOCK_PAGE_URL — optional override for navigate block redirects (default /blocked on this host)
• CLOAK_KEY — encrypts TOTP secrets
• URLGATE_DNS_ENABLED, URLGATE_DNS_PORT, URLGATE_DNS_UPSTREAM_HOST, URLGATE_DNS_UPSTREAM_PORT, URLGATE_DNS_SINKHOLE_TTL, URLGATE_DNS_MAX_ANSWER_TTL
• URLGATE_DOT_ENABLED, URLGATE_DOT_PORT, URLGATE_DOT_ZONE, URLGATE_DOT_CERTFILE, URLGATE_DOT_KEYFILE — Android Private DNS
• Mailer: BREVO_SMTP_*, MAILER_FROM_NAME, MAILER_FROM_EMAIL

Dev magic-link mail: visit /dev/mailbox when dev routes are enabled. More Q&A: FAQ.

Make your own filters FAQ Terms