DreamLake

Connections

A Connection is a credential DreamLake holds on a namespace's behalf to reach another system — an S3-compatible bucket, a Hugging Face account, a Dropbox. Sources use one to read an external layer. A Connection is not a Lakeshore connection, and it is not how a computer joins DreamLake (host enrollment).

Only the namespace's credential managers (a personal namespace's owner, an organization's owners) can see or change its Connections. Everyone else gets not found, for the namespace's Connections and for any id. These commands use your signed-in user token; machine and service-account credentials cannot manage Connections, and agent service accounts cannot be granted one.

Secrets travel one way: in. No command prints a credential back, and none accepts one as a command-line argument.

Find out what this server accepts

What can be connected — connector kinds, their auth kinds, the inputs each asks for, and product or profile choices — is declared by the server, per deployment. Ask it rather than assuming:

bash
dl connection connectors
dl connection connectors --json

Each row is one connector and auth kind. mode is how a credential arrives:

modeWhat it meansHow to create
fieldsTyped inputs, some of them secretdl connection create with a credential file or stdin, or dl connection entry create
oauthBrowser sign-in at the providerdl connection oauth create
noneNo credential, possibly some configurationdl connection create --no-credential

Inputs marked * go in the credential; the rest go in private configuration. --json adds labels, select options and conditional requirements (for example, an S3 region required only for some profileId values). S3 offers an access key auth kind (access key id, secret, optional session token) and, for the aws profile only, a role auth kind (a role ARN DreamLake assumes, with an external id). Its profileId choices — such as aws, b2 or r2 — are the profiles this deployment has configured; another server may offer different ones.

Create one from a file or stdin

bash
# credential.json: {"accessKeyId": "...", "secretAccessKey": "..."}
# config.json:     {"profileId": "aws", "region": "us-east-1"}
dl connection create --label "Team bucket" --connector s3 --auth-kind access-key \
  --credential-file credential.json --private-config-file config.json
gpg --decrypt credential.json.gpg | dl connection create --label "Team bucket" \
  --connector s3 --auth-kind access-key --credential-stdin --private-config-file config.json

The server validates the credential before it creates the Connection; a refusal creates no Connection. Delete the credential file afterwards.

For the role way in, roleArn is configuration and externalId is the secret:

bash
# credential.json: {"externalId": "..."}
# config.json:     {"profileId": "aws", "region": "us-east-1",
#                   "roleArn": "arn:aws:iam::123456789012:role/dreamlake-read"}
dl connection create --label "Team bucket (role)" --connector s3 --auth-kind role \
  --credential-file credential.json --private-config-file config.json

The role's trust policy must already allow DreamLake's AWS identity to assume it, with an sts:ExternalId condition equal to externalId, and the role needs read access to the buckets you will import. The CLI does not show that identity; get it from whoever operates your DreamLake deployment. --idempotency-key reuses the key of an attempt that never answered, so a retry cannot create a second Connection.

Hand entry to the web form

When the secret should not touch this machine, start a browser entry instead. The link carries only an opaque id. Finish it yourself, signed in to the web app as the same user who started it: it works for no one else, only in that namespace, for 30 minutes, and only once.

bash
dl connection entry create --connector s3 --auth-kind access-key --label "Team bucket"
dl connection entry status <operation-id>
dl connection entry cancel <operation-id>

Open the printed link while signed in, choose the product, and enter the credential there. entry status then reports succeeded with the Connection id and its masked details, or pending, cancelled or expired. The connector and auth kind are fixed by the entry; the label is a suggestion you can change. Only fields auth kinds use an entry. dl connection entry reconnect <id> does the same for a new credential on an existing Connection, at the revision it was started from.

The DreamLake assistant uses the same entry when you ask it to connect storage: it gives you the link and reads the result, and never sees the credential.

Manage

bash
dl connection list
dl connection list --lifecycle disabled --label-contains team
dl connection show <id>
dl connection check <id>
dl connection affected-sources <id>

check probes a Connection without changing it. available means the provider accepted the credential at the Connection's endpoint; it says nothing about a particular bucket. unverified (reason target-required) means this way in has no check that works without a target. Bucket and prefix access is checked when you import a Source.

bash
dl connection rename <id> --label "Archive bucket"
dl connection reconnect <id> --credential-file new-credential.json
dl connection disable <id>
dl connection enable <id>
dl connection delete <id>

Each change reads the current revision first and is refused if someone changed the Connection in between. delete is refused while a Source still uses the Connection — affected-sources lists them. OAuth Connections reconnect with dl connection oauth reconnect.

Use it from a Source

bash
dl source import --connector s3 --connection <id> \
  --target '{"bucket": "team-data", "prefix": "episodes/"}'

The Connection decides the endpoint profile; the target names only the bucket and prefix. See Sources.

TypeScript

The same operations are on connections(namespace) in @dreamlake/dreamlake-cli/api, with typed results:

ts
import { createClient } from '@dreamlake/dreamlake-cli/api';

const c = createClient({ remote: 'https://api.dreamlake.ai', token }).connections('acme');
const connectors = await c.connectors();          // ConnectorInputs[]
const entry = await c.entry.begin({ intent: 'create', connectorKind: 's3', authKind: 'access-key' });
// give entry.url to the person; later:
const done = await c.entry.status(entry.operationId); // done.resultConnectionId

create, rename, reconnect, enable, disable and delete take an idempotency key as their last argument. All but create also take the settings revision you last read; a changed Connection refuses the call.