Get A Free Quote
Leave a Comment / Uncategorized / By

1) URL & Canonical Strategy (back-end source of truth)

1.1 Normalize every incoming URL (gateway/middleware)

  • Force HTTPS, preferred host (e.g., www vs root), and trailing-slash policy (pick one).

  • Lowercase slugs (unless case matters) and strip tracking params (utm_*, fbclid).

  • If the normalized URL differs → 301 to the normalized form.

1.2 Canonical selection algorithm (service)

For each resource (post/page/taxonomy):

  1. Use the primary slug from DB.

  2. Prepend the preferred host and base path.

  3. If the request includes non-content parameters (e.g., utm_*) → canonical must exclude them.

  4. If the document is noindex → still compute canonical for consistency, but front-end will pair canonical with <meta name="robots" content="noindex">.

  5. Expose the chosen canonical in the API response so the front-end can render <link rel="canonical" href="…">. Google treats canonical hints from page or HTTP header; header form is useful for non-HTML assets like PDFs. Google for Developers+1

For non-HTML (PDF, CSV, feeds): add
Link: <https://example.com/clean-canonical>; rel="canonical" in the HTTP response headers. (Google explicitly supports canonical via HTTP header for non-HTML.) Google for Developers+1

1.3 Slug history (stable migrations)

  • Tables:

    • slugs: id, ref_type, ref_id, slug, is_primary, created_at.

    • redirects: id, from_path, to_url, code, active_from, active_to, note.

  • On slug change: mark the new row as is_primary=true and the old one is_primary=false, then auto-create a 301 rule from old → new.

2) Redirects: which code & when

2.1 Status codes & intent

  • 301 (Moved Permanently): permanent slug change, HTTP→HTTPS, host unification, trailing-slash policy, long-term content move. Google treats 301 as a strong canonical signal. Google for Developers

  • 302 (Found / Temporary): short-term tests (preview, A/B), maintenance detours. Google treats 302 as a weak canonical signal and may keep the source indexed. Google for Developers

  • 410 (Gone): use when the content is intentionally removed and should drop from the index faster. (Avoid redirecting to home.) Google for Developers

  • 451 (Unavailable For Legal Reasons): when you must block a resource due to a legal demand. Include an explanation page. IETF Datatracker+1

(You mentioned Stan Ventures for 301 vs 302—fine for background—but follow Google’s own guidance for canonical/redirect behavior.) Google for Developers

2.2 Admin UX & rules

  • Bulk add/import redirect rules (CSV).

  • Auto-generate on slug edits.

  • Rule priority: exact path > wildcard prefix > regex.

  • Safety rails: detect redirect loops/chains and block deploys if found.

2.3 Implementation sketch (NestJS)

 
// redirects.middleware.ts @Injectable() export class RedirectsMiddleware implements NestMiddleware { constructor(private redirects: RedirectsService) {} async use(req: Request, res: Response, next: NextFunction) { const normalized = normalizeUrl(req); // host/https/trailing slash if (normalized !== fullUrl(req)) { return res.redirect(301, normalized); } const rule = await this.redirects.match(req.path); // DB lookup (exact->wildcard->regex) if (rule) return res.redirect(rule.code, rule.to_url); next(); } }

3) 404 pages done right (and how they differ from 410/451)

3.1 What a “good” 404 means for SEO

  • Return HTTP 404 with a helpful HTML page (search, popular links).

  • This prevents “soft 404s” (returning 200 with a “not found” message), which confuse Google. Google for Developers+1

  • Keep the 404 lightweight (no indexable content), and noindex it at template level with X-Robots-Tag: noindex or a <meta name="robots" content="noindex"> to avoid it being indexed by mistake.

3.2 When to choose 410 instead of 404

  • If the removal is permanent and you’re sure there is no replacement, 410 helps search engines drop the URL faster. (Still show a friendly UI to users.) Google for Developers

3.3 Legal blocks (451)

  • If access is legally restricted, return 451 and a page that briefly states why content is unavailable; optionally include a link to more info. IETF Datatracker+1

3.4 Backend implementation

 
// not-found.filter.ts @Catch(NotFoundException) export class NotFoundFilter implements ExceptionFilter { catch(exception: NotFoundException, host: ArgumentsHost) { const ctx = host.switchToHttp(); const res = ctx.getResponse<Response>(); // send noindex for safety res.setHeader('X-Robots-Tag', 'noindex, follow'); res.status(404).send(render404Page(/* site settings, nav links */)); } } // gone.filter.ts (for explicit 410 routes) res.setHeader('X-Robots-Tag', 'noindex, follow'); res.status(410).send(render410Page());

4) API Contracts your front-end will consume

4.1 Resource payload (post/page)

 
{ "id": "uuid", "title": "My Post", "slug": "my-post", "primarySlug": "my-post", "canonical": "https://example.com/blog/my-post", "noindex": false, "nofollow": false, "robots": "index,follow,max-snippet:-1,max-image-preview:large", "redirectSuggestion": null, // or { code: 301, to: "/new-path" } if needed "publishedAt": "2025-10-10T09:15:00Z", "updatedAt": "2025-10-14T07:22:02Z" }

4.2 Canonical for non-HTML assets

When a post has a downloadable PDF at /files/post.pdf, the asset response includes:

 
Link: <https://example.com/blog/my-post>; rel="canonical"

(That’s the Google-supported way to canonicalize non-HTML.) Google for Developers+1

5) Data model & migrations (key pieces)

  • slugs as above (many per resource, one primary).

  • redirects with from_path (unique), to_url, code (301|302), windows, and note.

  • Constraints: prevent overlapping rules; add btree index on from_path.

  • Audit logs for slug changes and redirect edits.

6) Edge cases you must codify

  1. Pagination & filters

    • Category/tag listings: canonical normally includes the page number (self-canonical) if each page is materially unique. If pages are near-duplicates (thin listings), canonical to page 1 and consider noindex for deeper pages. (Backend exposes a flag per taxonomy to control this.)

  2. Session or preview params

    • Anything that doesn’t change the content (e.g., ?preview=1) should redirect 302 → canonical or render with canonical stripped.

  3. Duplicate paths across locales

    • Store locale with the slug; expose hreflang mapping in the API so the front-end can output alternate links. (Sitemap hreflang clusters can be generated too.)

  4. Attachment URLs

    • Avoid indexing raw attachment URLs unless they’re canonical documents; otherwise, canonicalize to the parent page via header link.

7) Observability & QA

  • Unit tests for:

    • Canonical computation (params stripped, right host, slug history).

    • Redirect resolution & loop detection.

    • 404 vs 410 vs 451 filters.

    • HTTP Link rel=canonical on assets.

  • E2E tests:

    • Changing a slug auto-creates a 301 and old URL now 301s to the new one.

    • Non-HTML canonical header present.

    • 404s return 404 (not 200), contain X-Robots-Tag: noindex.

  • Search Console:

    • Watch “Duplicate, submitted URL not selected as canonical,” soft-404 reports, and redirect coverage. Tweak rules accordingly. Google for Developers

8) Example NestJS snippets (copy-paste friendly)

8.1 Add a canonical header for non-HTML

 
// canonical.interceptor.ts @Injectable() export class CanonicalHeaderInterceptor implements NestInterceptor { constructor(private canon: CanonicalService) {} intercept(ctx: ExecutionContext, next: CallHandler) { const req = ctx.switchToHttp().getRequest<Request>(); const res = ctx.switchToHttp().getResponse<Response>(); return next.handle().pipe( tap((body) => { if (isNonHtmlResponse(res)) { const url = this.canon.forRequest(req); // compute preferred canonical res.setHeader('Link', `<${url}>; rel="canonical"`); } }), ); } }

8.2 Redirect lookup by DB rule

 
// redirects.service.ts async match(path: string): Promise<{to_url: string; code: 301|302} | null> { const exact = await this.repo.findOne({ where: { from_path: path }}); if (exact) return { to_url: exact.to_url, code: exact.code as 301|302 }; // TODO: wildcard/regex tiers return null; }

8.3 451 example (legal block)

 
// legal.guard.ts if (this.legalService.isBlocked(req.path, req.ip, req.headers['cf-ipcountry'])) { res.status(451) .setHeader('X-Robots-Tag','noindex, follow') .send(render451Page()); return false; } return true;

9) Developer checklist (ship-ready)

  •  URL normalizer + host/HTTPS/trailing-slash 301s

  •  CanonicalService: compute canonical; expose in every resource payload

  •  HTTP Link rel=canonical for PDFs/feeds

  •  slugs table + auto 301 on slug change

  •  redirects table + Admin UI + loop/chain guard

  •  404 filter returns 404 + X-Robots-Tag: noindex

  •  410 path handling for retired content

  •  451 handling path/region lists (if needed)

  •  Tests & logs; GSC monitored for duplicates/soft-404s

References (authoritative)

Absolutely. Here’s a developer-ready, backend-first SEO plan you can hand to your NestJS team. It focuses on what the API and server must do so your front-end (Next.js or any client) automatically ships best-in-class SEO.

1) URLs, Canonicals, Redirects

Goals: one canonical URL per document, stable slugs, clean migrations.

  • Canonical selection

    • Persist one primary slug per resource; keep a slugs history table for changed slugs.

    • Expose canonical URL in API responses; front-end must render <link rel="canonical" …>.

    • For non-HTML (PDF, CSV) responses, set HTTP Link: <…>; rel="canonical" from the API when relevant. (Google for Developers)

  • Redirect service

    • 301 for permanent slug changes; 302 only for temporary moves (preview, experiments). (Stan Ventures)

    • Provide an admin UI for bulk redirect rules and auto-generation on slug edits.

    • Return 410 Gone for intentionally removed content; 451 if legally restricted.

  • Querystring policy

    • Define a safelist of parameters that don’t change content (e.g., utm_*) → append rel="canonical" to the clean URL; never index the parameterized URL. (Google for Developers)

2) Robots control (server-side)

  • robots.txt served by backend with env-aware toggles (block stage; allow prod). Remember robots.txt is not a noindex mechanism; use meta robots / X-Robots-Tag for that. (Google for Developers)

  • X-Robots-Tag support at the API layer for non-HTML (PDF, images, feeds) and for global rules (e.g., noindex, noarchive on search results). (Google for Developers)

  • Per-item flags in DB (noindex, nofollow, nosnippet, max-image-preview, etc.) surfaced via API so the front-end can render <meta name="robots" …>. (Google for Developers)

3) Sitemaps at scale (dynamic, chunked, cached)

  • Generate in backend (NestJS SeoModule) so only published, canonical URLs appear.

  • Split by type: /sitemap-posts.xml, /sitemap-pages.xml, /sitemap-tags.xml, /sitemap-categories.xml, optional /sitemap-media.xml, all referenced by /sitemap.xml (index).

  • <lastmod> accuracy matters; compute from updated_at of meaningful changes (content/links/structured data). Don’t rely on copyright date. (Google for Developers)

  • Ignore/changefreq/priority: set or omit as you like; Google largely ignores them; prioritize getting lastmod right. (Google for Developers)

  • Image & video extensions in sitemaps when posts have rich media (featured images, video embeds). (Spotibo SEO)

  • Automation: on publish/update, enqueue Search Console ping for /sitemap.xml. (Google for Developers)

  • Caching: cache rendered XML (Redis) for 30–120 minutes; auto-invalidate on publish/edit.

4) Structured Data (JSON-LD builder services)

Deliver JSON-LD objects from the API so clients can drop them straight into <script type="application/ld+json">.

  • Article/BlogPosting (default for posts): headline, image(s), author, datePublished, dateModified, publisher, mainEntityOfPage. (Google for Developers)

  • VideoObject when video present (self-hosted or oEmbed), including uploadDate, duration, thumbnailUrl.

  • ImageObject for featured images (width, height, caption/alt).

  • AudioObject for podcasts/audio posts (duration, contentUrl).

  • FAQPage when content blocks contain Q&A.

  • BreadcrumbList built from category hierarchy and slug path.

  • Organization/WebSite (site-wide) exposed via /seo/site-ld.json endpoint for front-end layout injection.

  • Validation: sanitize, ensure required fields per type, and never emit executable content.

  • API contract: GET /seo/jsonld?type=post&id=… (or embed in GET /posts/:id).

(Why JSON-LD? It’s the most robust, Google-preferred format.) (Google for Developers)

5) Meta data orchestration (server rules → client rendering)

Provide a computed SEO payload per document from backend:

{
  "title": "...",
  "description": "...",
  "canonical": "https://…",
  "robots": "index,follow,max-snippet:-1,max-image-preview:large",
  "og": { "title": "...", "description": "...", "image": "…" },
  "twitter": { "card": "summary_large_image", "title": "...", "image": "…" },
  "jsonld": { … }  // from §4
}

  • Smart fallbacks:

    • Title: seo.meta_title || post.title

    • Description: seo.meta_description || excerpt (n chars)

    • OG/Twitter image: seo.og_image || featured_image

  • Author & E-E-A-T: include author.name, author.url, author.sameAs[], and explicit datePublished & dateModified in payload so UI can render them consistently (and JSON-LD builder can reuse). (Google for Developers)

6) Pagination, faceting, and index management

  • Listing pages (archives, tags, categories): make them indexable only if useful and unique; otherwise ship noindex,follow. Provide a toggle in taxonomy settings. (Google for Developers)

  • Pagination: no rel next/prev (deprecated); keep clean canonical to page 1 only if pages are near-duplicate; if each page has unique value (different intros/featured), keep self-canonical per page. (Google for Developers)

  • Internal search results: default to noindex,follow via X-Robots-Tag or meta. (Google for Developers)

7) Feeds and discoverability

  • RSS/Atom/JSON Feed from backend with canonical links in each item.

  • Podcast feed variant when audio posts exist (enclosures, duration).

  • News-style freshness: ensure feed items’ dates mirror datePublished/Modified that match JSON-LD and sitemap <lastmod>. (Google for Developers)

8) Performance & crawl efficiency (server responsibilities)

  • HTTP caching: strong ETags + Last-Modified; leverage 304s on public GETs.

  • Conditional rendering: for heavy SEO endpoints (sitemap, feeds, JSON-LD preview), add Redis caching with precise cache keys.

  • Status codes: 200 for live, 301 for permanent moves, 302 temporary, 404 for missing, 410 for retired content; retry-after on 503 maintenance windows.

  • Rate limiting for anonymous API endpoints; don’t rate-limit Googlebot on public content.

9) Media SEO (images, video, audio)

  • Images

    • Store alt and caption; expose both to front-end.

    • Persist original dimensions and deliver multiple renditions via CDN; include width/height in API for correct HTML attributes (helps CLS).

    • Include images in image sitemap or as <image:image> entries in per-type sitemaps. (Spotibo SEO)

  • Video

    • Save duration, thumbnailUrl (poster), uploadDate; include VideoObject JSON-LD and optional video sitemap entries. (Spotibo SEO)

  • Audio / Podcasts

    • Provide waveform/duration; emit AudioObject JSON-LD and podcast feed as needed.

10) Internationalization (optional but powerful)

  • Language fields per document; emit inLanguage in JSON-LD and hreflang map in the API so the client can render <link rel="alternate" hreflang="…">.

  • Provide GET /seo/hreflang?postId=… that returns canonical + alternates for all locales.

11) Anti-duplication & crawl budget protections

  • Tag “thin” pages as noindex by default until they clear a min posts threshold.

  • Do not index parameterized pages (filters/sorts) unless they’re canonical content; enforce X-Robots-Tag: noindex at the API gateway for such routes. (Google for Developers)

  • Consistent canonical in sitemaps: only submit preferred URLs; sitemaps are taken as canonical hints. (Google for Developers)

12) Admin & automation

  • SEO Settings table (site name, base URL, default robots, default OG/Twitter images, publisher org JSON-LD).

  • Webhooks

    • On publish/update:

  • Pre-publish lint (server validation):

    • Missing title/description overflows, too-short content, missing featured image when schema type requires it, forbidden words in slugs, duplicate titles.

  • SEO Preview API: GET /seo/preview?type=post&id=… → returns computed <title>, meta, OG/Twitter, and JSON-LD for the client to render an editor preview.

13) Data model (seo-related fields)

  • seo.meta_title, seo.meta_description, seo.canonical_url

  • seo.og_title, seo.og_description, seo.og_image_id

  • seo.twitter_title, seo.twitter_description, seo.twitter_image_id, twitter_card

  • seo.schema_type (default “BlogPosting”), seo.jsonld (jsonb, additive overrides)

  • noindex (bool), nofollow (bool), robots (string for granular directives)

  • published_at, updated_at (used for JSON-LD & <lastmod>)

(Backed by server-side JSON-LD builder and Sitemap builder.) (Google for Developers)

14) QA & Monitoring

  • Automated tests:

    • JSON-LD schema validation (required fields per type),

    • sitemaps well-formed & chunked,

    • canonical consistency (API vs sitemap vs front-end),

    • robots flags honored,

    • redirect correctness (301 vs 302).

  • Search Console: verify sitemap submission and monitor coverage; resolve “Alternate page with proper canonical” & “Duplicate, submitted URL not selected as canonical” signals by adjusting rules. (Google for Developers)

  • Log crawler traffic (UA sampling) & 404/410 spikes.

15) Deliverables (what the backend must ship)

  1. SeoModule with:

    • SitemapService, SeoComputeService, JsonLdService, RobotsService, CanonicalService, RedirectService

  2. Endpoints:

    • GET /sitemap.xml (index) + chunked sitemaps

    • GET /robots.txt

    • GET /seo/preview, GET /seo/jsonld?type=&id=

    • Optional: GET /seo/site-ld.json (Organization/WebSite)

    • Feeds: GET /feed.xml, GET /feed.json

  3. Middleware/Filters:

    • X-Robots-Tag injector, canonical header for non-HTML, rate limiter

  4. Queues:

  5. Admin UI surfaces:

    • Per-item SEO panel (all fields above), redirects manager, robots defaults, JSON-LD overrides, hreflang map.

Quick rationale links

Want code?

I can generate a NestJS SeoModule starter with controllers/services for:

  • dynamic + chunked sitemaps (with <image>/<video> extensions),

  • robots.txt & X-Robots-Tag middleware,

  • JSON-LD builders for BlogPosting/Video/Audio/FAQ/Breadcrumb, and

  • a /seo/preview endpoint that returns the exact tags your front-end should render.

Leave a Comment

Your email address will not be published. Required fields are marked *

Subscribe
Subscribe