DreamLake

The .dreamrc reference

.dreamrc has existing consumers with different schemas. This reference indexes them without reinterpreting visualization fields as launcher options or scene composition. New registration fields are implemented in this change; release availability must be verified separately.

Registration fields

FieldType / rule
repoOptional repository declaration; name and remote required together.
repo.nameNonempty registry name.
repo.remoteCredential-free HTTPS or SSH Git remote; pointer only.
repo.snapshot.commitFull 40-hex commit or 64-hex capture digest.
repo.snapshot.backends3 or github.
repo.snapshot.archive_keyExisting object key or pinned GitHub ref.
repo.snapshot.storage_nameRequired S3 storage name. No credentials.
udfs[]Definition list; names unique across UDFs and agents.
udfs[].nameDotted registry name.
udfs[].entrypointPython module:function; declared, not imported during upload.
udfs[].sourceRelative UTF-8 source file inside manifest root, at most 512 KiB.
udfs[].projectsOptional existing project slug list; stored affiliation.
agents[]Agent definition list.
agents[].nameDotted registry name.
agents[].sourceRelative Markdown file with YAML frontmatter and prompt body.
agents[].projectsExisting project slugs; overrides frontmatter projects when supplied.

See the complete upload example and agent frontmatter. The SDK compatibility change keeps repo, udfs, and agents out of launcher kwargs. Older SDK versions may forward unknown root keys as extras; use a build containing that change before combining execution and registration fields.

Execution fields

The SDK config lookup is explicit path, $DREAMRC, nearest ancestor .dreamrc, then home .dreamrc. The new registration command takes one explicit manifest path, defaulting to .dreamrc in the current directory; it does not inherit a home manifest. Current launch code resolves queue separately from runtime mode.

FieldMeaning
default_modeNamed default mode; SDK default local.
modes, legacy modeMapping of names to runtime configurations.
providersTagged launcher declarations: SSH, EC2, GCE, SLURM, Kube. Launcher-specific kwargs are open.
queueExisting CLI launch default, below --queue and LAKESHORE_QUEUE.
backendRuntime backend, e.g. local or fabric.
serverExecution endpoint.
runnerExecution runner; default process.
tagsPlacement/worker matching requirements.
resourcesResource requirements, passed as a mapping.
imageContainer image; pin a digest for repeatable environments.
envString environment mapping; keep credentials outside registered declarations.
timeout_sInvocation timeout in seconds.
providerNamed provider reference.
host_setupOrdered host preparation commands, once per worker.
run_setupOrdered commands before every invocation.
dreamlake.remoteDestination API for in-run artifacts.
dreamlake.projectArtifact project destination.
extrasOpen provider/launcher kwargs.
docker.networkDocker network setting, only with the Docker runner.
kube.namespace, kube.contextKubernetes namespace/context.
kube.cpu, kube.memoryPositive Kubernetes quantity strings or integers.
kube.gpuResource name plus positive count.
kube.poll_interval_sInteger 1–3600.
kube.imageLegacy nested image spelling; must agree with top-level image.

Runtime fields may appear in a named mode or at the root per the existing SDK contract. Provider-specific open fields cannot have a closed global enum; validate them against their provider's schema. See the owning SDK config.

Code capture fields

FieldDefault / meaning
code.mounts[]Required when code is supplied.
code.mounts[].local_pathSource file/directory, relative to .dreamrc; supported env/home expansion.
code.mounts[].targetRelative unpack destination; default depends on mount count/type.
code.mounts[].pypathfalse; adds destination to PYTHONPATH.
code.mounts[].filesworktree or tracked.
code.mounts[].excludeExclusion patterns.
code.mounts[].exclude_fromRelative file containing exclusion patterns.
code.mounts[].exclude_vcstrue; omit VCS internals.
code.mounts[].includePaths inside directory mounts; default ["."]. This is file selection, not configuration imports.
code.workdirWorking directory inside mounts; default first mount target.
code.max_archive_mbCompressed archive ceiling; default 100 MiB.
excludes, also_excludes, max_archive_mbLegacy repository-archive controls used by the existing snapshot path; not substitutes for the code block.

See snapshot/launch and the full capture bounds. Dataset dependencies use their own datasets.yaml list; automatic Source mounting remains separate from manifest validation.

Visualization and scenes

A dataset visualization .dreamrc uses version: 1, optional name and storage, required dataset, and required views. It is validated by the visualization package, not the execution SDK. Its complete schema is indexed in visualization configuration.

Scene composition uses dreamlake.layers.json, independently from .dreamrc: see scene upload and composition. Scene-layer references and XML includes are not UDF config imports. Existing configurations remain valid.

Imports, includes, and layers

No shared .dreamrc import/layer resolver is provided by the new registration command. YAML anchors work within a file; they are not cross-file imports. Do not claim scene composition semantics apply to execution or visualization.

Before adding a common resolver, specify layer order, map merges, list replacement versus extension, field origins, include-relative paths, cycles, missing inputs, and authorized remote references. Capture all inputs and resolved output with hashes and provenance. Changes to an include after capture must not change an old snapshot. Until then, unsupported composition keys must fail explicitly on registration rather than silently disappearing.