Clearing what a finished execution leaves behind (ADR-0016).
An execution that has ended still stores its last position blob and,
on an adapter that keeps one, its whole input log (ADR-0010) - the
host's own event data, at rest, for the life of the store. prune/3
clears both for every execution that ended before a cutoff the host
names, and keeps the execution row itself: its status, its failure,
its metadata, its answer and its end stamp stay, so the drained query
still counts it, a parent can still read a child's answer, and the
execution id stays taken.
There is no clock here. The cutoff is a DateTime.t/0 the host
computes from its own policy; nothing in this module takes a duration,
defaults a window, or runs on a schedule (ADR-0012 decision 7). Which
rows a host may delete on its own, and which it must not, is
docs/retention.md.
Summary
Types
What prune/3 answers for single_batch: true: one batch's own counts
plus whether another call may find more.
Options prune/3 accepts.
Functions
Clears the position blob and the input log of every execution that
ended before cutoff, in batches, and answers what it cleared.
Types
@type batch_counts() :: %{ executions: non_neg_integer(), position_blobs: non_neg_integer(), inputs: non_neg_integer(), more?: boolean() }
What prune/3 answers for single_batch: true: one batch's own counts
plus whether another call may find more.
@type prune_opt() :: {:batch_size, pos_integer()} | {:scope, StatifierPersistence.Storage.Adapter.prune_scope()} | {:single_batch, boolean()}
Options prune/3 accepts.
Functions
@spec prune( store :: StatifierPersistence.Storage.t(), cutoff :: DateTime.t(), opts :: [prune_opt()] ) :: {:ok, StatifierPersistence.Storage.Adapter.prune_counts() | batch_counts()} | {:error, StatifierPersistence.Storage.error()}
Clears the position blob and the input log of every execution that
ended before cutoff, in batches, and answers what it cleared.
An execution is pruned when its status is :completed, :failed or
:cancelled and its ended_at is set and strictly before cutoff.
An execution with no end stamp is never touched, and neither is one
whose row carries a stamp but was written back to a status that is not
terminal. The execution row stays; see the moduledoc for what it keeps.
Answers {:ok, counts} summed over every batch: executions pruned,
position_blobs nulled among them, and input log rows deleted. It is
idempotent: a second call with the same cutoff answers zeros, because
an execution with nothing left to clear is not selected again - unless
a row another transaction held locked was skipped (on Postgres), which
a later call then takes (see :single_batch below).
Each batch is one atomic unit in the adapter and commits on its own,
unless a caller's transaction encloses the call (see below).
So an {:error, reason} from a later batch leaves the earlier batches
pruned, and calling again with the same cutoff carries on from where
the failed batch stopped.
{:error, :execution_pruning_unsupported} for a store whose adapter
does not declare the capability
(StatifierPersistence.Storage.execution_pruning_supported?/1),
before anything is read.
{:error, :unscoped_adapter} when :scope is given and the store's
adapter cannot confine a batch to it - the in-memory adapter is one -
before anything is cleared.
Raises ArgumentError when cutoff is not a DateTime.t/0 - a
duration, a number of days or a Date included - when :batch_size
is not a positive integer, and when :scope is not a non-empty keyword
list of distinct columns with no nil value.
Options
:batch_size- at most how many executions one batch prunes. Defaults to 500. Smaller batches hold shorter transactions; the answer is the same.:scope- a keyword list of column equalities, such as[tenant_id: "tenant-a"], that confines the prune to the rows holding every one of them. Every batch's selection, input log check, input log delete and position blob update carries the equalities, so a prune run inside one partition's transaction reads and writes no row of another. OnStatifierPersistence.Storage.Ectothe columns are ones the host placed with:leading_columns. Left out, the prune covers the whole store, as it always has.nilis refused rather than read asIS NULL, because an equality withNULLmatches no row.:single_batch-false(the default) drains every batch as before.trueruns exactly one batch and answers that batch's own counts plusmore?:truewhen the batch took as many executions asbatch_size:allows, so another call may find more;falsewhen it took fewer, the same point where the default drain stops. When the last due batch holds exactlybatch_size:executions, that call answersmore?: trueand the next one answers zeros withmore?: false. A row another transaction holds locked (on Postgres) is skipped, not waited on, so it is left for a later call either way.
Inside a transaction of your own
Each batch is its own transaction only when nothing encloses it. On
StatifierPersistence.Storage.Ecto a batch's transaction joins a
caller's enclosing transaction, so prune/3 called inside one runs the
whole drain - every batch - as one transaction that commits or rolls
back with the caller's. :batch_size then bounds each batch's
statements, not the transaction.
For one bounded transaction per batch - for example one that first sets
a partition's context - call prune/3 with single_batch: true inside
each of your transactions, and call again while more? is true:
def prune_tenant(store, cutoff, tenant_id) do
batch =
Repo.transaction(fn ->
# set the tenant's context for this transaction here, then:
opts = [scope: [tenant_id: tenant_id], single_batch: true]
case Retention.prune(store, cutoff, opts) do
{:ok, counts} -> counts
{:error, reason} -> Repo.rollback(reason)
end
end)
with {:ok, %{more?: true}} <- batch do
prune_tenant(store, cutoff, tenant_id)
end
endIt answers {:ok, counts} for the last batch, more?: false, or the
first {:error, reason}, with every earlier batch committed.