Logs on your node
A node’s SQLite file (logs.sqlite in the data directory) can hold two
kinds of records, marked by a source column:
- Queries the node answered. Stored locally when the profile’s logs are on and its destination is not Nowhere, whichever other destination it is. They never leave the node, except as answers to your own dashboard requests.
- Queries the cloud answered for the same profile, when the profile’s log destination is My self-hosted node or Both. These travel from the cloud to the node over its link.
The path of a cloud record
Section titled “The path of a cloud record”When a device sends a query to opdns’s cloud addresses (a phone on mobile data, say):
- The PoP resolves it and produces the record. Logs-off and client-IP-off apply here, on the PoP.
- The record crosses opdns’s internal
logsstream (held at most 24 hours). - Ingest applies domain logging and the destination. For a node destination it puts the record in a per-profile queue instead of the cloud database.
- The queue sends batches down the node’s link, one batch in flight. The node writes each batch to SQLite, then acknowledges it.
- Acknowledged batches are dropped from the queue.
If the node is offline, the queue holds records until it reconnects, for
at most 24 hours by default. The profile setting Hold logs for an
offline node (Settings;
node_queue_hours in the API) can shorten that to 12 hours or 1 hour, or
turn the queue off (Do not hold: records made while the node is
offline are dropped); 24 hours is the maximum. Records older than the
window are lost; they are not moved to the cloud database. The node’s card
on the dashboard’s Nodes page shows how many are waiting (Queued logs).
So choosing the node as destination means the cloud still processes those queries and holds each record briefly on the way. It does not keep them. The full account is on Architecture.
Reading logs
Section titled “Reading logs”With the destination My self-hosted node, the dashboard’s Logs and
Analytics pages send each request to the node over its link and show the
answer, marked with a Your node badge. The result
passes through opdns’s servers in transit, encrypted on each hop, and is
not stored. While the live tail is open, the cloud asks the node for new
rows every second (every 2 seconds when the dashboard falls back to
polling). A node runs at most link.max_inflight_queries relayed
queries against its database at once (default 4) and allows 10 seconds for
each. Another one waits up to a second for a free slot; if none frees up,
it is shed: answered at once with no rows and marked partial
(partial_reason: overload), which the dashboard shows as Partial
results: your node is busy. A live tail keeps its position and catches
up on the next poll. Lower the setting on small hardware. With the node
offline, the pages say so, show when it was last seen and retry by
themselves; see
Logs.
With opdns cloud or Both, the pages read the cloud’s copy instead, which holds only the queries opdns answered: the queries your node answered itself stay in its SQLite file and do not appear there.
Device names on your network
Section titled “Device names on your network”Plain DNS carries no device name, so queries your devices send to the node
over plain DNS would have an empty Device column. With
local.device_names on (the default), the node names such a client after
its host name from the DHCP lease files or local.hosts
(Local names), cleaned up as
any device name is, so Logs and Top devices show laptop or tv
instead of nothing. A device that sends its own name (DoH, DoT, DoQ)
keeps it. Without lease files or hosts there is nothing to name clients
by; local.device_names: false turns it off.
Analytics from your node
Section titled “Analytics from your node”The node answers the same analytics views as the cloud, from its SQLite file, with two differences worth knowing:
- Owners are classified on the node. The
top_ownersview (queries by the company they reached: Google, Apple, Meta, Amazon, Microsoft, Cloudflare, Akamai, Fastly) and theownerof each destination are worked out when the dashboard asks, from classification data built into the node’s release. A newer node release brings newer data, and it applies to all the logs already stored. - The node has no GeoIP data. Its
destinationsanswers carry an empty country for every address. opdns fills the countries in from its own database as the answer passes through its servers to your dashboard; nothing about your queries is looked up by a third party, and nothing is stored in the cloud.
A node older than these views answers a top_owners query with
501 shape_unsupported (update the node), and its destinations
answers get the country and owner columns added by opdns on the way.
How both lookups work: What ingest adds
(the country data is credited on Attributions).
Retention and size
Section titled “Retention and size”The node deletes days older than the profile’s retention (default 30 when
the profile sets none, store.retention_days). store.max_size_mb caps
the file size. The cloud’s 90-day limit applies to what opdns stores, not
to your node: an enrolled node follows the profile’s setting (1 to 90
days, as the dashboard allows), while store.retention_days and a
standalone node’s profile file have no upper limit.
Low disk space
Section titled “Low disk space”Below store.min_free_mb of free space on the data directory’s file
system (default 500 MB; 0 turns the guard off), the node pauses
logging until space is back:
- DNS keeps answering; only the log records are dropped, and counted;
- batches the cloud sends for the node are refused, so they stay queued in the cloud (for the profile’s usual window, 24 hours at most) instead of being lost right away;
- the local page says “logging paused: disk”.
Schema upgrades and damage
Section titled “Schema upgrades and damage”A release that changes the database schema copies the file to
backups/pre-v<N>.db before migrating it, and a database found corrupt at
start is moved to backups/corrupt-<time>.db and replaced by an empty one.
See what an upgrade changes on disk.
To keep the logs when you move a node, use
opdns-node backup.