Skip to main content

Persistence

Everything the runtime needs to remember — agent definitions, sessions, messages, working memory, summaries, todos, LLM-call traces — is stored through a small set of repository ports. Two implementations ship: the default writes plain files to disk, the other keeps everything in Postgres. One setting switches between them; nothing else in the runtime, the tools or the agent definitions knows the difference.

Ports and adapters​

The runtime starter (mc-agent-starter-runtime) builds every store of the runtime from the Persistence a persistence starter declares — the starters themselves keep only what is installation-wide: the users and their API tokens.

Each domain module also ships a repository factory for its ports — AgentRepositoryFactory (agent runtime), MessageRepositoryFactory (messages), LlmRepositoryFactory (LLM configs and their prices) — with one implementation per backend: FileAgentRepositoryFactory, InMemoryAgentRepositoryFactory, PgAgentRepositoryFactory, and so on. Whoever assembles a runtime picks the factory once for the backend it wants; the repositories come from it.

The runtime follows a hexagonal (ports-and-adapters) design. Each persisted concept is an interface in port/out, and the file-based default is the matching adapter in adapter/file:

Port (port/out)StoresFile adapter (adapter/file)Postgres adapter (adapter/pg)
AgentDefinitionRepositoryAgent definitionsFileAgentDefinitionRepositoryPgAgentDefinitionRepository
AgentSessionRepositorySessions (incl. parent/child links)FileAgentSessionRepositoryPgAgentSessionRepository
MessageRepositoryConversation messagesFileMessageRepositoryPgMessageRepository
ConversationRepositoryConversation metadataFileConversationRepositoryPgConversationRepository
WorkingMemoryRepositoryPer-turn working-memory snapshotsFileWorkingMemoryRepositoryPgWorkingMemoryRepository
ConversationSummaryRepositoryCompaction summariesFileConversationSummaryRepositoryPgConversationSummaryRepository
TodoListRepositorySession todo listsFileTodoListRepositoryPgTodoListRepository
LlmCallTraceRepositoryLLM request/response tracesFileLlmCallTraceRepositoryPgLlmCallTraceRepository
LlmConfigRepositoryLLM configs (credentials encrypted)FileLlmConfigRepositoryPgLlmConfigRepository
LlmPriceRepositoryLLM prices, one per config, model and periodFileLlmPriceRepositoryPgLlmPriceRepository
FileStoreUploaded files (ports in mc-file-store-core)FilesystemFileStorePgFileStore
WorkflowDataRepositoryWorkflow definitions (workflow area)FileWorkflowDataRepositoryPgWorkflowDataRepository
WorkflowInstanceRepositorySuspended workflow runsFileWorkflowInstanceRepositoryPgWorkflowInstanceRepository

A port is a plain interface — for example:

public interface AgentSessionRepository {
    AgentSession save(AgentSession session);
    Optional<AgentSession> findById(SessionId id);
    List<AgentSession> findByAgent(AgentId agentDefinitionId, UserId userId);
    // …
}

The default: file persistence​

In the Spring apps the file adapters come from the mc-agent-starter-file starter (agents/springstarter/), all rooted at one directory (mindconnect.data.base-dir, default data/) and, below it, one directory per namespace — so the data of a default install lives under data/local/, and a namespace acme under data/acme/. Users, API tokens and the namespaces themselves are installation-wide and live under data/system/. The LLM-config repository is wrapped in an EncryptingLlmConfigRepository when the app has an EncryptionHelper, and LLM call traces live under conversations/ with a mindconnect.agent.trace.max-per-session retention cap (default 50).

Every adapter — file or Postgres — is bound to one namespace when it is built; the starters hand out a routing proxy per port that keeps one adapter per namespace and forwards each call to the one the current Scope names. The scope is bound per request (from the /ns/{namespace}/ prefix, the X-Mindconnect-Namespace header, or the namespace the user chose in the Admin UI) and per queued task; nothing above the repositories names a namespace. Start-up work — the runtime build, the tool warm-up, the seed loaders — binds mindconnect.namespace (default local) explicitly; a thread that binds nothing and touches a store fails instead of working there quietly. That is also the namespace every signed-in user may work in without an invitation.

This makes the runtime zero-dependency: it boots and persists with nothing but a writable folder — no database, no migrations, ideal for local development and single-node deployments.

Besides the file adapters there is a full in-memory family (InMemoryAgentSessionRepository, InMemoryMessageRepository, …) used by the Spring-free AgentRuntimeBuilder and in tests. Two of them are bounded so that a long-lived embedded runtime does not grow with every turn: the trace store keeps at most 50 LLM call traces per conversation (the file store's cap), and the builder tells its task queue to forget finished task trees at the next maintenance tick — taskRetention(Duration) keeps them longer (it reaches the task-queue feature too, when that is installed), null for the life of the process. Deleting a session takes its conversation, messages and traces with it, in every persistence mode.

Postgres​

Since 0.3.0

Postgres persistence ships from version 0.3.0 on. Earlier releases have the file and in-memory adapters only.

Set one variable and every port above is served by its Postgres adapter:

MC_PERSISTENCE=postgres                                   # default: file
MC_POSTGRES_URL=jdbc:postgresql://localhost:5432/mindconnect
MC_POSTGRES_USER=mindconnect
MC_POSTGRES_PASSWORD=…

The start.sh scripts read these from the git-ignored mc.env; as Spring properties they are mindconnect.persistence and mindconnect.postgres.url / username / password / pool-size. The Admin UI, the agent server and the standalone workflow admin app all honour the same switch. Behind it stand two Spring Boot starters in agents/springstarter/ — mc-agent-starter-file and mc-agent-starter-postgres; an application has both on the classpath and the property decides which one configures the repositories.

What happens on start:

  • One pooled DataSource (HikariCP) is opened and shared by every store.
  • Each store creates its tables if they are missing — CREATE TABLE IF NOT EXISTS, plus ADD COLUMN IF NOT EXISTS for columns a later version declares. There is no migration tool and nothing to run by hand; a plain Postgres without extensions is enough.
  • The bundled initial data — agent definitions, LLM configs, skills, example workflows — is seeded into the database exactly as it would be seeded into data/: into the start-up namespace at start, into every other namespace on its first use, and only what the namespace never had (mc_installed_seed remembers what it had).
  • Uploaded files go to the database as well (mc_file, content as bytea) unless mindconnect.file-store.backend names another backend — keeping records in Postgres and files on a volume is a valid pairing.
  • The file adapters step back automatically. mindconnect.data.base-dir still exists for what has no database form: workflow scratch files, vector-store files of the memory backend, code-execution mounts.

How the data is stored​

Each domain object is one row: the whole object as a JSONB document, with the few values a query filters, sorts or lists by as ordinary columns beside it — the id, a namespace, a name, a sequence number, a timestamp. The document is the truth and the columns are the index; both are written in the same statement. The namespace column is part of every primary key and every query, so two processes bound to different namespaces can share one database. A document in the database is the same JSON the file adapter would have written, rendered by the application's ObjectMapper.

The tables are named mc_agent_definition, mc_agent_session, mc_conversation, mc_message, mc_working_memory, mc_conversation_summary, mc_todo_list, mc_llm_call_trace, mc_llm_config, mc_llm_price, mc_workflow, mc_workflow_instance, mc_file, mc_vector_store_template, mc_vector_store_instance and mc_vector_store_member, plus the embedding index mc_embedding and mc_embedding_field when the database has pgvector. The small JDBC layer underneath — Sql, Row, DocumentTable — lives in common/mc-jdbc; the adapters are the agents/adapter/postgres/*-pg and workflow/mc-workflow-persistence-pg modules, the Spring wiring is mc-agent-starter-postgres.

Two lists are read without touching a document at all: AgentSessionRepository.findHeadersByUser (the chat sidebar) and LlmCallTraceRepository.findHeadersByConversation return AgentSessionHeader and LlmCallTraceHeader — views the aggregates implement themselves — straight from the columns.

What stays on disk​

The vector stores keep nothing on disk in Postgres mode — as long as the database has the pgvector extension. Their registry of templates and instances is in mc_vector_store_template and mc_vector_store_instance (filled once per namespace from vector-stores/templates|instances), and their vectors are in one pgvector table per store, in the application's own database: pgvector is the default backend there and needs no URL of its own. A plain postgres:16-alpine has no pgvector; then the vectors stay on the memory backend in <data-dir>/<namespace>/vector-stores/*.jsonl, with a warning at start. Install pgvector into the image — for an existing Alpine data directory an Alpine build, since a Debian image changes the text collation — and the stores move into the database on first use — see Vector stores.

Embedding without Spring​

AgentRuntimeBuilder.usePostgres(dataSource, dataDir) wires the same adapters for a plain-Java host; dataDir roots the file-based side channels as before. Workflows and uploads follow when their modules are on the classpath.

Moving an installation​

There is no automatic migration from data/ to Postgres. A fresh Postgres start seeds the bundled defaults; your own agents, sessions and conversations would need to be re-created or copied by a script through the two adapter families — both speak the same JSON, so that script is a read-and-save loop.

Workflows are the exception: the first time a namespace is used on Postgres, the definitions and suspended runs the file stores kept under <data dir>/<namespace>/workflows are imported, when mc_workflow and mc_workflow_instance have nothing for the namespace yet. Once — a row in mc_workflow_import records it, so what is deleted later stays deleted — and the files are left where they are.

Testing against Postgres​

The adapter tests run against a real Postgres and skip themselves when none answers on localhost:5433:

podman run -d -p 5433:5432 -e POSTGRES_PASSWORD=test pgvector/pgvector:pg17-trixie

Override with MC_JDBC_TEST_URL, MC_JDBC_TEST_USER, MC_JDBC_TEST_PASSWORD.

Other databases​

Every storage concept is a port, so a third backend means implementing those interfaces — the Postgres adapters are the template: a few dozen lines each on top of mc-jdbc, with the same @ConditionalOnProperty switch. Because each port is independent you can also migrate incrementally, one repository at a time.