Parameter sweeps#

Parameter sweeps (decision D35).

A Monte-Carlo sweep is a stateless for loop: reconfigure (set_params(), decision D34), reseed (seed(), decision D6), run, collect. The loop below stays visible on purpose – a student must be able to read where the SNR is swept – and the caller writes one line. The Trainer/hook pattern is explicitly rejected by the architecture document.

Extracted from the identical skeleton of three validation/ scripts (core_awgn_ser.py, fec_coding_gain.py, ofdm_awgn_ser.py), per the decision’s trigger.

monte_carlo(chain: Sequential, param: str | Sequence[str], values: Sequence[Any], metrics: Mapping[str, Callable[[...], Any]], stimulus: Any, *, reference: str | None = None, seed: int | None = None, n_jobs: int = 1) → dict[str, ndarray][source]#

Run a chain over a range of parameter values and collect metrics.

Parameters:

  • chain (Sequential) – The chain to sweep. It is reconfigured in place.

  • param (str or sequence of str) – Dotted parameter path(s), e.g. "noise.snr_dB" (see Sequential.block_ids()). With several paths, each entry of values provides one value per path (zip semantics; the cartesian product is out of scope, per the decision).

  • values (sequence) – Parameter values, one sweep point each.

  • metrics (mapping of str to callable) – Metrics collected at every point. Called as metric(target, y) when reference names an observed block, else metric(y). Smaller-is-better quantities (BER, SER, EVM); there is no score() (decision D24).

  • stimulus – Input passed to the chain at every point (e.g. a symbol count).

  • reference (str, optional, keyword-only) – Block id whose observed output is the metric target. The observation is declared on the chain if it is not already (Sequential.observations).

  • seed (int, optional, keyword-only) – Master seed; every sweep point gets an independent child seed through the chain (decisions D6/D35), so the whole curve is reproducible.

  • n_jobs (int, optional, keyword-only) – Sweep points to run at once. Default 1, the plain loop below. -1 uses every core. See Running points in parallel.

Returns:

dict of str to np.ndarray – One array per metric, aligned with values.

Raises:

ValueError – If n_jobs is not 1 and no seed was given, or if the chain or a metric cannot be sent to a worker process.

Notes

Running points in parallel. The points of a sweep are independent by construction – each one reconfigures and reseeds the chain from scratch – so they can be run at the same time, and n_jobs says how many at once. Three consequences are worth knowing before turning it on.

The result does not change. Each point already draws from its own child seed, so a curve computed with n_jobs=4 is identical to the serial one, value for value, not merely statistically equivalent. This is asserted in the test suite. It is also why seed= becomes mandatory: without it, what a point draws would depend on which worker picked it up and in which order.

A worker is a process, so everything it needs is pickled – the chain, the metrics and the stimulus. A metric written as a lambda or as a closure cannot cross that boundary; define it at module level instead. The error says so rather than letting pickle explain it.

A stateful chain should stay serial. Each worker holds its own copy of the chain, so a block that carries something between points (an adaptive equalizer, an estimator fitted at the previous point) sees a different history than it would in the loop. Sweeps of such chains are order-dependent to begin with; this only makes it visible.

It is not free. Starting a worker and pickling the chain into it costs about 130 ms, once per worker, so the sweep has to be long enough to pay for it. Measured on four cores, a 24-point sweep of a sphere-decoded 4x4 link:

cost of one point

n_jobs=1

n_jobs=2

n_jobs=4

19 ms

0.46 s

0.51 s

0.39 s

222 ms

5.34 s

2.99 s

1.99 s

Below ~20 ms a point the pool costs more than it saves; at ~200 ms it returns 2.7x on four cores. The values are identical in every column.

Examples

>>> from comnumpy import AWGN, Sequential, SymbolGenerator
>>> from comnumpy.core.metrics import compute_ser
>>> chain = Sequential([SymbolGenerator(4), AWGN(snr_dB=0, name="noise")])
>>> out = monte_carlo(chain, "noise.snr_dB", [0, 10],
...                   {"power": lambda y: float(np.mean(np.abs(y)**2))},
...                   stimulus=1000, seed=1)
>>> print(out["power"].shape)
(2,)