Cover illustration generated via social-worker’s generate_image MCP tool (OpenRouter / Flux).
Most social media tools are cloud-hosted SaaS wrappers around a scheduler. social-worker takes a different approach: it is a local-first, Docker-only, multi-modal publishing platform designed for both human authors and autonomous AI agents.
It combines a conversational thread composer, automated feed ingestion, Whisper audio/video transcription, programmatic image composition, content calendar scheduling, autonomous reply agents, and a full Model Context Protocol (MCP) server with 79 tools.
In fact, this entire post doubles as a live showcase of social-worker’s visual and publishing pipelines: every image below—from the Flux AI cover art and hybrid vector template card to the Mermaid architecture diagrams and Chart.js data visualizations—was generated through social-worker’s own MCP tools and bundled for atomic single-commit delivery via GitHubBlogPublisher.
Here is a detailed look under the hood at how the system is structured, why key technical choices were made, and how its moving parts fit together.
1. System Topology: A Polyglot Docker-Only Stack
social-worker runs exclusively inside Docker Compose. Every build, database migration, test suite, and runtime service executes in containers so that local development matches production without host SDK drift.
Rendered via render_diagram (image-engine + beautiful-mermaid + Sharp/Canvas).
Rather than forcing one runtime to do everything poorly, the stack uses a .NET 10 core surrounded by purpose-built sidecars:
| Service |
Tech Stack |
Responsibility |
Web SPA (web/) |
React 19, TypeScript, Vite, Tailwind CSS + MUI v6, Nginx |
Composer Studio, Drafts Library, Content Calendar, Review Outbox, Template Studio, Reply Agents Cockpit; Nginx also reverse-proxies /api/ |
Backend API (api/) |
.NET 10 ASP.NET Core Minimal API, EF Core 10 |
Core domain orchestration, publishing engine, scheduling, durable task workers, auth, and MCP server |
Database (db) |
PostgreSQL 16 |
Primary relational store and durable background job queue (FOR UPDATE SKIP LOCKED) |
Transcriber (transcriber/) |
Python 3.11, FastAPI, OpenAI Whisper, yt-dlp, ffmpeg |
Video/audio extraction and speech-to-text (with optional NVIDIA GPU acceleration) |
Image Engine (image-engine/) |
Node.js 22, Fastify 5, Sharp, @napi-rs/canvas, Chart.js, beautiful-mermaid |
Declarative JSON layer-tree rendering, statistical charts, and Mermaid diagram cards |
| Search & Feeds |
SearXNG + RSSHub |
Self-hosted meta-search and social feed bridging (Instagram, X, YouTube playlists, Reddit, TikTok, Bluesky) |
On the AI side, the backend uses Microsoft.Extensions.AI with an OpenAI-compatible SDK and configurable BaseUrl. That keeps LLM selection completely portable across OpenRouter, OpenAI, and local Ollama instances. All third-party platform credentials are stored encrypted at rest in PostgreSQL using AES-256.
2. Backend Design: Vertical Feature Slices & Domain Invariants
The .NET 10 backend (SocialWorker.Api) is organized into vertical feature slices under Features/ (Chat, Drafts, Publishing, Sources, Feeds, Workflows, ReplyAgents, Scheduling, BackgroundTasks, Templates, Media, and Mcp), with cross-cutting concerns isolated in Infrastructure/.
Three design principles shape the domain layer:
- Make invalid states unrepresentable: Domain entities restrict direct construction and enforce invariants through static factory methods (
Draft.Create, Source.CreateUrl, Source.CreateFile, FeedSubscription.Create, ReplyAgent.Create). Operation results use strongly typed records with static factories (Ok, NotFound, Fail) rather than raw tuples or dynamic JSON.
- CommonMark AST as the thread source of truth: Every draft is stored as a CommonMark Markdown document parsed via Markdig (
ThreadDocumentParser). Horizontal rules (---) partition a document into ordered ThreadSegments while preserving inline links, facets, and media references (media://{id}). Each platform validator consumes this AST to enforce platform-specific character budgets and embed rules.
- Server-enforced stage gates: Drafts move through explicit lifecycle stages:
RoughDraft -> Sourcing -> Refining -> Formatting -> Ready -> Sent
LLMs can propose stage transitions via tools, but the server enforces that publishing is rejected unless the draft is validated and confirmed.
3. Durable Queue & Content Scheduling Engine
Early prototypes used in-memory channels for background work, which meant container restarts dropped in-flight transcriptions and scrapes. social-worker replaces in-memory queues with a PostgreSQL 16 durable task queue backed by a "BackgroundTasks" table and FOR UPDATE SKIP LOCKED.
Rendered via render_diagram (image-engine + beautiful-mermaid).
How the Queue Works
- Atomic claiming & crash recovery:
PostgresBackgroundTaskQueue leases tasks with heartbeat tracking and exponential backoff retries. If a worker dies mid-job, expired leases are automatically reclaimed.
- Allow-list worker specialization:
BackgroundTaskHostedService runs configurable worker pools—separating heavy, long-running transcription jobs onto dedicated workers so fast scrapes, reply checks, and scheduled publishes are never head-of-line blocked.
- Automatic retention:
BackgroundTaskRetentionService prunes old completed/failed runs and expired schedule queue entries on a 7-day rolling window.
Time-Decayed Calendar Scheduling
Content scheduling (Features/Scheduling/) builds on top of this durable queue:
- Recurrence & pacing: Schedules define recurring slot rules parsed with Cronos and materialized by
SlotWindowCalculator.
- Priority & preemption: Every 15 minutes,
ScheduleEvaluationHostedService runs ScheduleSlotEvaluatorService to re-score queued drafts using Level-2 time-decay priority calculations and cascading preemption so time-sensitive posts can overtake evergreen backlog items.
- Human-in-the-loop triage: Drafts can route through a Review Outbox before dispatch, and
ScheduledPublishingTaskHandler can invoke DraftAutoRepairService to self-correct validation issues prior to posting.
Publishing is unified behind a single IPublisher interface with seven production-ready implementations:
- Bluesky: Multi-post ATProto threads, byte-offset facets, and image blobs.
- LinkedIn: Personal profiles and organization company pages via 3-legged OAuth.
- Meta Threads: Multi-post threads, carousel media containers, status polling, and token refresh.
- Twitter / X: OAuth 2.0 PKCE with refresh token rotation, chained reply tweets, weighted 280-character validation, and
t.co URL budgeting.
- Instagram: Professional/Creator single-image and 2–10 image carousel publishing with container polling.
- Mastodon: Federated dynamic OAuth 2.0 (
/api/v1/apps) with access-token fallback, public root / unlisted reply threads, 23-character URL budgeting, and Idempotency-Key support.
- GitHub Blog: Long-form static blog publishing (Jekyll preset) with hybrid YAML frontmatter merging, leading
# H1 promotion, automatic local media asset rewriting, YouTube thumbnail embeds, and atomic single-commit publishing via the GitHub Git Data API.
Failure Tolerance & Idempotency
Multi-post social threads fail in partial ways—network timeouts on post 4 of 7 can easily cause duplicate threads if retried naively. social-worker addresses this with three safeguards:
- Thread-level locking (
PublishLockService) prevents concurrent publish requests for the same draft and platform.
- Per-segment checkpointing (
PublishResumePlanner + PublishOutcomeRecorder) records each published segment as it succeeds, allowing retries to resume from the exact failed segment without duplicating earlier posts.
- Unified idempotency across REST endpoints, MCP tools, scheduled jobs, and outbox actions ensures duplicate publish requests return existing post records instead of re-posting.
5. Ingestion, Workflows & Autonomous Reply Agents
Polymorphic Source Ingestion
Authors and workflows can attach URLs, uploaded files, YouTube videos, and social profiles as sources. When a video or audio URL is attached, the transcriber sidecar follows a fast-path cascade:
- Direct media stream detection for
.mp4/.mp3/CDN streams (bypassing yt-dlp).
- YouTube caption fast path via
youtube-transcript-api with pacing locks and backoff to avoid rate limits.
yt-dlp + Whisper fallback extracting bestaudio/best, running thread-synchronized Whisper inference, and persisting timestamped JSON transcripts to a shared /transcripts volume.
Automation Workflows
Features/Workflows/ uses an extensible IWorkflowSourceHandler pattern (FeedWorkflowSourceHandler, DigestWorkflowSourceHandler) paired with RssHubRouteResolver and RssHubFeedDiscoveryService. Workflows poll feeds, deduplicate items, generate structured thread drafts with citations, and support credential-free JSON export/import for migrating pipelines between environments.
Autonomous Reply Agents
Features/ReplyAgents/ lets users maintain a reusable library of persona-driven reply agents assignable to workflows or directly to external/organic posts:
- Platform Adapters (
IReplyPlatformAdapter) fetch comment trees across Bluesky, Meta Threads, Twitter/X, and Mastodon.
- Enrichment (
CommentEnrichmentService) pre-fetches linked web pages, transcribes video attachments, and inspects images via vision models.
- Two-Phase Ranking (
CommentRankingService) scores incoming comments from 1–10, classifies intent, and detects terminal conversation closures (e.g., polite sign-offs) to avoid robotic over-replying.
- Tool-Augmented Synthesis (
ReplyGenerationService) runs multi-turn tool loops (web search, source lookup, video transcription) backed by a rolling CommentThreadSummarizer cache and hard stop criteria (branch depth limits, per-user caps, and bot self-reply prevention).
6. Visual Composition via image-engine
Visual assets are generated by a dedicated Node.js 22 Fastify service (image-engine/) using Sharp and @napi-rs/canvas (Skia-backed 2D canvas) with pre-bundled Inter and JetBrains Mono fonts:
POST /render: Evaluates declarative JSON layer trees (background, rectangle, text, image) for reusable social cards authored in the frontend’s Konva-based Template Studio.
POST /render/chart: Produces Chart.js statistical cards.
POST /render/diagram: Converts Mermaid syntax via beautiful-mermaid (with a custom CSS variable / color-mix() resolver for librsvg) into high-DPI framed diagrams.
POST /transform: Executes Sharp resize, crop, format conversion, and watermarking pipelines.
For generative visuals, the .NET API integrates OpenRouterImageClient for cloud Flux/SDXL image generation and supports hybrid compositing—feeding an AI-generated background directly into a vector JSON template slot so typography stays razor-sharp:
Live hybrid composite rendered via render_template (Dark Tech Release Card v2 in image-engine) layered over our Flux-generated backdrop.
7. Agent-Native Operations via Model Context Protocol (MCP)
A defining feature of social-worker is that every major operational workflow is first-class over MCP. External agents (Claude, Google Gemini Custom Apps, or CLI coding agents) can connect over:
- Stateless Streamable HTTP:
POST /api/mcp (alias /mcp)
- Server-Sent Events (SSE):
GET /api/mcp/sse (alias /mcp/sse)
Generated via generate_image (OpenRouter / Flux).
The McpToolRegistry exposes 79 tools across 9 domains (drafts, variants, media/templates/diagrams, sources/search/transcription, feeds, workflows, scheduling/calendar, accounts/tags, and reply agents) alongside read-only socialworker:// resources.
Authentication supports both SHA-256 hashed static API keys (sw_live_...) and a standards-compliant OAuth 2.1 flow:
- RFC 9728
401 challenges with WWW-Authenticate resource metadata discovery
- RFC 8414 authorization server metadata (
/.well-known/oauth-authorization-server)
- RFC 7591 Dynamic Client Registration (
POST /oauth/register)
- PKCE S256 authorization code exchange issuing 1-hour JWTs and 30-day refresh tokens
8. Frontend SPA & Verification Pyramid
The frontend (web/) is a React 19 + TypeScript SPA built with Vite and served by Nginx. Rather than stuffing active drafting state into one giant global store, it splits state cleanly:
ComposerDraftContext scopes the active CodeMirror 6 Markdown document, debounced autosave with version conflict detection, source polling, and the @assistant-ui/react native EventSource SSE chat runtime.
- Zustand stores handle lightweight global state (
draftStore, editorStore, authStore, templateEditorStore).
Everything is backed by a Docker-first verification pyramid:
Rendered via render_chart (image-engine + Chart.js).
- API Tests: 963 xUnit unit and integration tests via
docker compose --profile tooling run --rm dotnet test
- Web Tests: 552 Vitest component and logic tests via
web-test
- Image Engine Tests: 31 Vitest rendering and transformation contract tests via
image-engine-test
- MCP Stress Suite: Automated JSON-RPC & SSE integration stress testing
- Isolated E2E Suite: 10 Playwright spec suites running against a dedicated
docker-compose.e2e.yml stack (ports 8200/8201) with DEMO_LLM_PROFILE=getting-started, swapping external LLM calls for deterministic scenario responses so E2E runs are fast, zero-cost, and flake-resistant.
9. Development Progress History
social-worker evolved rapidly over roughly four and a half months (July–November 2026, spanning 263 commits and nearly 30 archived design plans under planning/archive/), progressing through five distinct phases:
Rendered via render_chart (image-engine + Chart.js) from the repository’s git history.
- July 2026 — MVP, Bluesky & Architectural Hardening (
v0.1.0–v0.1.5): Initialized on July 8, 2026 as a chat-assisted Bluesky thread composer with AES-encrypted credentials, media uploads, brand voice prompts, SearXNG search, RSS automation, and a Whisper transcription pipeline. Before scaling features further, late July focused on decoupling endpoints into services, codifying domain-invariant refactoring standards, integrating FluentValidation, and closing out early tech debt and test coverage gaps.
- August 2026 — Multi-Platform Foundations,
image-engine, Durable Queue & MCP: Added LinkedIn publishing (3-legged OAuth), multi-account support, and decoupled Automation Workflows with 2-tier generation. Introduced the Node.js image-engine microservice and Konva Template Studio, rebuilt navigation around a workflow-grounded Drafts page and full-width Composer, replaced in-memory background channels with the PostgreSQL SKIP LOCKED durable queue, and shipped the first native MCP server with API key auth and hardened Markdig AST parsing.
- September 2026 — Content Calendar, Scheduling & Worker Specialization: Built the Cronos-backed Content Calendar (Month/Week/Day views, slot rules, drag-and-drop queue sidebar, Review Outbox) and MCP scheduling tools, expanded LinkedIn to Company Pages, introduced extensible workflow source handlers, and added allow-list worker pool specialization so long transcriptions no longer block fast tasks.
- October 2026 — Platform Scale, Reply Agents & OAuth 2.1 MCP: Shipped Meta Threads (alongside the resilient
PublishResumePlanner and idempotency locks), Twitter/X (OAuth 2.0 PKCE), and Instagram profile ingestion via RSSHub. Launched Autonomous Reply Agents as a shared library with multi-platform adapters and multi-turn tool loops, upgraded scheduling with Level-2 time decay and cascading preemption, added OpenRouter image generation and credential-free workflow export/import, and brought MCP up to full OAuth 2.1 + PKCE + Dynamic Client Registration (Claude & Google Gemini Custom Connections compatible) with automated Hetzner deployment.
- November 2026 — Studio Polish, Visuals & Long-Form Publishing: Migrated frontend layouts to MUI v6 with a persistent Composer Studio sidebar and full-height CodeMirror editor, added Instagram and Mastodon publishing, introduced universal entity tagging, extended
image-engine with Chart.js (render_chart) and Mermaid (render_diagram) rendering, and shipped atomic single-commit GitHub Blog publishing.
Closing Thoughts
Building social-worker around local-first ownership, durable PostgreSQL primitives, strict domain invariants, and MCP-first interfaces turned what started as a thread composer into a programmable publishing engine. Whether you are iterating in the two-pane web composer, scheduling a week of curated feeds, or asking an AI agent to draft and illustrate a technical deep dive over MCP, every path flows through the same validated, checkpointed, and testable pipeline.