Experiments
Native project runs
The DreamLake-native backend stores new runs under a DreamLake project. These commands require a server with native Experiments enabled and your existing DreamLake login. The run key makes creation idempotent; inspect the run after an uncertain write before retrying.
Archive hides a run from default lists while preserving its data. Restore makes it visible again; neither operation deletes historical Dash data.
Runs can be moved to trash and restored without changing their archive state. Lifecycle-enabled servers also allow running runs to be trashed; older servers require a terminal run. Historical Dash runs remain on the old service.
Run data
These commands read and write one native run's data through the native
Experiments API of the selected project. Success prints the server's JSON.
Paged reads return one page and nextCursor; pass it to --cursor for the
next page. --data and --metadata read JSON from a file or - (stdin).
params get reads and params delete removes all parameters of a run.
--description, --tags and --metadata on metrics append are stored
with the points only when that append creates the metric. metrics list lists a run's metrics, metrics stats shows one metric's point
count and metadata, and metrics metadata updates its --description,
--tags or --metadata.
logs append --data appends up to 1000 {timestamp, level, message} entries.
A download never overwrites an existing path and is removed if its bytes do not
match the file's checksum. files list, files get, files update and
files delete complete the file operations.
Tracks
Tracks hold time-indexed entries ({timestamp, ...data}) on runs stored in
the native Dash store. append writes up to 10000 entries in one commit, and
read prints JSON or exports to a new file.
tracks list lists a run's tracks, and tracks create creates a track with an
optional --description, --tags or --metadata. For --format mcap, a topic
ending in / exports every track under it.
Run files use DreamLake's existing project file storage. Address the episode by the run key you created; file upload uses your DreamLake login.
Planned experiment sweeps
A planned sweep (configs × seeds, with fresh, resumed and reconnected attempts
compared one curve per seed in Dash) is set up and launched from Python, so it
has no dl command. The DreamLake Python SDK (pip install dreamlake) ships
the template and a small command line; Dash reads the result from any project
whose status lists the experiments capability:
Tested with Python 3.13.14, dreamlake 0.50.1 and params-proto 3.3.0. With
Python 3.14.5 and params-proto 3.3.0 the template's grid generator fails
(TypeError: cannot pickle 'itertools.product' object), so create the
environment with Python 3.13:
The project must already exist (dl create project <name> --namespace <ns>). The attempts are ordinary
native runs, so the run commands above list, inspect, archive and trash them. See
the experiment workflow guide
for the walkthrough and the expected counts after each step.
Historical Dash workspace binding
Unreleased. Requires the matching DreamLake and Dash server changes. These commands have not been released or verified on a hosted deployment.
Use your existing DreamLake login and explicitly choose the DreamLake organization namespace. An organization owner can discover eligible existing workspaces on the managed Dash instance. No Dash URL or token is accepted or copied by this flow. An identical organization name does not establish a binding.
Inspect the current binding, including its revision. Results contain stable DreamLake and Dash namespace identifiers; a slug is only a discovery aid.
Select a unique slug or namespaceId returned by discovery. connect first
previews the selected target without changing anything. Review its instanceId,
namespaceId, current binding, and expectedRevision.
Apply requires the discovered stable namespaceId (DASH_NAMESPACE_ID above),
copied from the reviewed preview's target.namespaceId; slugs are preview-only.
Set DASH_INSTANCE_ID to its target.instanceId. Apply requires this explicit
instance ID so configuration changes cannot silently redirect the selection.
The server verifies the current user's Dash organization ownership
and the instance again at apply time. A stale revision fails rather than
overwriting another owner's change. Refresh and review after a conflict; do not
automatically retry with a newer revision.
To detach, review a separate preview and apply its revision. Detaching disables the DreamLake binding without deleting remote experiments, changing Dash members, or revoking SDK credentials. It remains available when remote discovery is down. Detached rows keep their revision; only a namespace with no previous binding uses 0.
Success always emits a JSON receipt. With --json, errors emit
{ok:false,error:{code,message,remediation}} and exit nonzero. A rejected write
is not confirmation of its outcome: inspect binding before retrying after a
network interruption.
Binding administration preserves existing Dash workspaces for historical reads. It does not move their data or grant Dash data access. New native runs use DreamLake project permissions and storage.
Transfer complete experiments
dl experiments upload and download use the interoperable .dash directory
format. They use your selected DreamLake environment and existing dl login;
they never send new data to the historical Dash service.
Transfers include run folders/names, description, tags, metadata, Bindr names,
write protection, parameters, logs, metrics and run files. Both flat and directory
metric layouts are accepted. Protection is applied after the run's data and labels
are written. Files are checksum-verified and filenames preserve Unicode, quotes
and literal percent sequences. Downloads are installed only after every selected
section succeeds; --overwrite replaces a prior local run without appending its
data again. A failed replacement leaves the previous local copy intact.
Use --dry-run to inspect selection, --target namespace/project/folder to upload
under another directory, and --skip-logs, --skip-metrics, --skip-files or
--skip-params to omit sections. --batch-size controls log/metric batching;
downloads also accept --max-concurrent-metrics and --max-concurrent-files.
A failure exits nonzero and retains the state file. --resume --state-file PATH
skips fully completed runs for the same server, local root and target. A partially
written remote run can contain accepted appends: inspect it before repeating an
upload; resume is not transactional rollback or append deduplication.
Tracks are transferred separately as JSONL entries, or exported in a server format:
The parquet server format returns Arrow IPC bytes, preserving the existing
ML-Dash wire contract. A project/run selector can include folder paths or globs;
ordinary --project namespace/project selects all of that project's runs.
Native writers and jobs
Writers are metadata identities used to mint short-lived recording sessions. Create with a stable intent UUID; inspect before rotating or retiring an observed generation. Retiring is permanent and does not stop a compute process.
Native jobs create their experiment run in this DreamLake project and dispatch to
your explicitly configured host. The manifest contains requestId, target,
optional enrollmentId, execution: {kind: "uv-run", argv: ["script.py"]},
source: {files: [{path, contentBase64, sha256, mode: 420}]} and optional
timeoutSeconds. Files are explicit; no script or dependency is inferred from argv.
Keep the original manifest and request ID after an uncertain submission.
show returns the compute receipt and native experiment binding. Cancellation and
logs first verify the job belongs to the selected project. A job's SDK logging
context uses the worker broker; the compute receipt owns terminal status.