JarvisHub is an open harness that makes an editable Canvas (as agent workspace) the shared project state for multimodal creative agents, so prompts, drafts, versions, and feedback live as typed nodes an agent reads, writes, and revises under a permissioned protocol bridge instead of getting lost in chat history.
You’ve shipped a creative agent that generates a storyboard, then images per shot, then a video per image. On shot 7 the user says “keep the character but redo the lighting.” Today your agent probably rereads the chat log, loses which candidates were accepted, and regenerates too much. Commercial products like Claude Design and Google Stitch are moving toward canvas-style creative agents, but their internals are closed, so there’s no shared substrate for studying how such agents track state, recover, or accept feedback. JarvisHub is the open scaffold for that setting. It positions itself against three baselines: prompt-to-output tools that discard intermediate context, chat-based creative agents whose memory is a linear conversation, and node-based workflow tools like Node-based workflow tools (e.g., ComfyUI) that require humans to wire pipelines up front.
The core move is to stop treating the canvas as a display layer and start treating it as the agent’s memory and action space. Every artifact the project touches, references, prompts, draft images, video clips, rejected candidates, layout choices, user feedback, becomes a typed node with a stable ID. Edges between nodes carry meaning: this image was generated from that reference, this shot is version 3 of that shot, this slide depends on that diagram. The agent observes this graph directly rather than reconstructing it from conversation.
Each turn runs a fixed loop. The agent reads the current canvas plus the user request, and the protocol bridge hands it two things: a capability manifest (what node types and tools exist in this project) and an execution grant (what subset of those the agent is allowed to use this turn). The agent proposes an action; the bridge only executes it if it can be encoded as a checked tool call, a canvas mutation, an evaluation, a clarification, or a user-facing reply. The result, plus any human or evaluator feedback, is written back as new nodes and edges. Everything is logged as a trajectory tuple of (request, canvas-before, manifest, grant, action, observation, feedback, repair, canvas-after).
Tools are grouped into families: canvas edits, generation (image, video, audio), native execution (browser, code, files), recovery (checkpoint and local repair), and MCP (Model Context Protocol) extensions. On top of tools sit three orchestration aids: skills (reusable procedures like storyboarding or design-to-web), memory (preferences and prior decisions across turns), and subagents (parallel branches for independent subtasks like exploring alternative shots).
while not done:
canvas = read_canvas_state()
manifest, grant = bridge.capabilities(canvas, user_query)
action = agent.propose(user_query, canvas, manifest, grant)
if not bridge.validate(action, grant):
continue
obs = bridge.execute(action) # tool call or canvas mutation
feedback = collect(user_or_critic, obs)
repair = agent.decide_followup(feedback)
canvas = bridge.commit(canvas, action, obs, feedback, repair)
trajectory.append(...)
The prevailing default for creative agents is to treat the chat transcript as the source of truth and let tools return opaque blobs the model describes back in prose. This paper argues the opposite. The project state should live outside the model, as an inspectable typed graph both humans and agents mutate through the same checked interface, and the trajectory of those mutations, not just the final artifact, is what you evaluate and learn from. The evidence is the qualitative demonstration that three very different long-horizon workflows (narrative video, web build, slide deck) run on one harness while remaining fully inspectable, not a benchmark score.
The load-bearing result is structural, not numeric: the same canvas-plus-bridge runtime handles three long-horizon creative tasks end-to-end while keeping intermediate state addressable and repairable. The paper reports no quantitative benchmarks; the authors are explicit that experiments are qualitative demonstrations rather than a leaderboard.
•
Narrative media: a short-drama prompt becomes a shot sequence where character references, shot candidates, and dependency links stay visible on the canvas across generations.
•
Interactive web development: an aesthetic brief becomes a rendered photography site, with references, layout drafts, code artifacts, and previews all coexisting as nodes.
•
Presentation deck: a lecture topic becomes a multi-slide deck where diagrams, drafts, and PowerPoint previews are linked to their source content.
All runs use GPT-5.5 as the agent backend, GPT Image 2 for images, Seedance 2.0 for video, and Gemini 3.1 Pro as a multimodal evaluator. The paper doesn’t compare against a specific baseline system or report task success rates.
Reach for this when you’re building a creative agent that runs longer than one prompt, say, a marketing-asset pipeline or a design copilot, and you’re tired of stuffing project context into the system prompt. The pattern to steal: give the agent a typed graph API (create-node, link-node, update-status, checkpoint) instead of a free-form scratchpad, and gate every write through a permissions layer that logs what changed and why. Even without adopting JarvisHub itself, that separation between agent proposal and validated commit turns “why did it regenerate everything” into an answerable question.
Code is released at GitHub with model-backend configuration details. The paper doesn’t ship a benchmark or dataset, but the authors point to trajectories collected under this harness as raw material for future project-state benchmarks and training data, subject to consent, anonymization, and copyright filtering they flag as unresolved.
Make the workspace the memory. If your agent and your user can’t point at the same node, you don’t have a project, you have a transcript.
•
No quantitative evaluation. Three qualitative case studies do not establish that canvas-native beats chat-based or node-based agents on any measurable axis; the paper concedes this.
•
The harness enforces that actions are valid and logged, not that they’re good. Semantic correctness of creative decisions still rides entirely on the underlying models (GPT-5.5, GPT Image 2, Seedance 2.0).
•
The trajectory-as-training-data pitch depends on filtering, consent, and copyright review the authors flag but don’t solve; raw creative trajectories aren’t drop-in training corpora.