Configuration files can make machine learning experiments easier to repeat

Separating experimental instructions from execution code allows radio localization researchers to test algorithms without rewriting pipelines, according to a new workflow design.

In computational research, two scientists running the same machine learning model on identical raw data can easily end up with different results. When code scripts mix data preparation, model parameters, and hardware settings into a single file, an undocumented change in a data transform or a software package breaks the ability to verify an experiment. Radio localization, which relies on interpreting signal patterns to pinpoint the physical location of wireless transmitters, suffers from this friction as researchers routinely rebuild baseline models and data loaders from scratch.

When experimental setups are difficult to repeat, evaluating whether a new algorithm genuinely improves accuracy becomes a guessing game. Building custom pipelines for every trial drains engineering time and leaves future researchers unable to confirm whether a reported gain came from the model architecture or a quiet difference in data formatting. If an algorithm claims to locate a transmitter within two meters instead of five, that claim cannot be trusted unless the exact data splits, filtering choices, and training parameters can be verified independently.

To address this challenge, Tim Strnad, Blaž Bertalanič, and Carolina Fortuna at the Jožef Stefan Institute in Slovenia introduced a framework designed to make localization experiments consistent. In a preprint posted to the arXiv server on August 21, 2026, the researchers outlined an approach that separates experimental instructions from execution code.¹

Why is repeating machine learning research difficult?

Repeating a machine learning experiment requires tracking several interconnected steps in precise order. First, raw sensor or radio readings must be cleaned, filtered, and split into training and evaluation sets. Next, specific numerical settings must be assigned to the learning algorithm before training begins. Then, the model must run inside an isolated computational environment so that underlying software versions do not skew calculations. Finally, the generated outputs and evaluation metrics must be archived alongside the exact code and configuration that produced them.

If any single link in this chain goes undocumented, later researchers cannot reproduce the published findings. Because many research teams build their own custom scripts for dataset formatting, execution, versioning, and baseline models, slight differences in execution lead to divergent experimental outcomes. In fields like wireless positioning, where slight variations in radio channel noise filtering dramatically alter model convergence, untracked modifications make external validation nearly impossible.

How does separating instructions from code change the workflow?

Separating experiment settings from execution logic allows researchers to adjust trial parameters without altering underlying source code. Strnad, Bertalanič, and Fortuna designed an application-specific system called LOCALIZE, which uses human-readable configuration files to declare how an experiment should proceed.¹ A workflow orchestrator then runs each step as an isolated computational process that interacts solely through explicit inputs and outputs.¹

Under this arrangement, code, raw data, configuration files, software environment specifications, and generated artifacts are versioned together in a single record. The system provides preconfigured datasets, data processing stages, model-development procedures, and experiment templates tailored to radio localization, while allowing researchers to modify the underlying pipeline when needed.¹

Managing state across isolated processes without steep overhead requires a disciplined data-passing strategy. Fortuna, a researcher at the Jožef Stefan Institute, said in response to questions from Primary that a stage writes its declared outputs to disk while the next stage reads those outputs.^contributed Fortuna explained that the data version control tool DVC tracks these files, so unchanged outputs can be reused instead of recomputed. Fortuna noted that this design adds file access and process startup costs, particularly for short stages, but prevents hidden state from carrying between stages and makes the pipeline inspectable and resumable. Fortuna added that across their benchmarks, complete pipeline wall time remained comparable because the model training stages dominated execution time.

Balancing standardized configuration with unexpected research needs presented an engineering trade-off during the development of LOCALIZE. Fortuna told Primary that the hardest part was balancing routine configuration with researchers' need to introduce methods the framework could not anticipate.^contributed As Fortuna observed, common experimental choices can be expressed clearly in configuration, but arbitrary data processing or new algorithms cannot without turning the configuration into another programming language. To maintain balance, new functionality is implemented directly in code and then exposed through configuration files for later reuse.

What did the tests reveal about system overhead?

Comparative benchmarks demonstrated that moving experiment definitions into configuration files reduced code modifications without creating significant processing delays. The authors evaluated LOCALIZE qualitatively against five existing experimentation platforms, and quantitatively against matched implementations in Jupyter notebooks and the data pipeline tool Kedro.¹

Selecting Kedro as the primary structured baseline allowed for a direct architectural comparison. Fortuna told Primary that Kedro provided the closest structured, general purpose baseline because its nodes, dependencies, and Data Catalog map closely to LOCALIZE's stages and declared inputs and outputs while leaving the domain logic to user code.^contributed This alignment allowed the authors to implement the same local workflow in both systems. By contrast, Fortuna noted that orchestrators such as Airflow target broader operational scheduling and deployment, which would have introduced infrastructure concerns outside the intended comparison.

For the localization workflows tested, adapting experiments to new settings required fewer edits to the codebase under LOCALIZE than under the baseline implementations.¹ Across these comparative runs, total wall-clock execution time and peak memory usage remained comparable to the traditional notebook and pipeline configurations.¹

In a controlled scaling test across one, five, and 10 times the base dataset volume, total central processing unit time and wall-clock duration grew sublinearly over the evaluated sizes.¹ Fortuna said in response to questions from Primary that the pipeline structure and search budgets did not change as the data volume increased, so orchestration and process isolation costs remained approximately constant while the data dependent work grew.^contributed As model training and optimization processed more data, these fixed costs represented a progressively smaller share of the total runtime, resulting in sublinear growth.

The codebase reduction holds primarily when researchers work within the library of existing components. Fortuna said that configuration covers experimental changes that use existing components.^contributed When genuinely new data processing, model, evaluation, or pipeline logic is required, it is implemented in the relevant component or added as a new stage without changing the orchestrator or unrelated stages. Once added, the functionality becomes available through configuration and can be reused across later experiments.

What can the framework results not establish?

The reported benchmarks reflect controlled laboratory evaluations on radio-localization tasks rather than a broad census of all machine learning workflows. The findings show how the framework behaves on specific radio datasets and baseline architectures, but they cannot predict performance on massive multimodal systems or distributed multi-node clusters. Furthermore, the paper was posted as a preprint and has not undergone formal peer review by independent experts.

Specific pipeline architectures could also degrade the runtime parity observed in the benchmarks. Fortuna told Primary that orchestration overhead would become noticeable only when the cost of passing intermediate results between stages approaches the cost of the computation itself.^contributed According to Fortuna, this degradation could occur in an unusually fragmented pipeline with many lightweight stages, or one that transfers very large intermediate results while performing relatively little computation.

The efficiency of any configuration-driven tool depends on whether the prebuilt templates match the experimental requirements of the user. If an investigator needs fundamentally new processing primitives that fall outside the provided templates, they must still write custom code modules within the pipeline.

For research teams working on wireless positioning and sensor tracking, standardizing experimental configurations offers a path toward directly comparable benchmarks. The next step for this line of engineering will be testing whether configuration-first architectures can support larger scientific datasets and more diverse hardware environments without increasing configuration complexity.

This piece was prepared from the arXiv preprint and public records together with answers from Carolina Fortuna to six questions from the Primary news team, completed August 2026.

References

This article is based on 1 source, with 6 statements from 1 contributor, listed in the order they are cited.

1. 1 TS Tim Strnad (Jožef Stefan Institute, Slovenia), Blaž Bertalanič (Jožef Stefan Institute, Slovenia), Carolina Fortuna (Jožef Stefan Institute, Slovenia) announcement · 21 Aug 2026 A Configuration-First Framework for Reproducible, Low-Code Machine Learning: a Localization Use Case See the source
2. 2 CF Carolina Fortuna added 26 Aug 2026 Contributor commentary — Carolina Fortuna 6 statements added to this article

Article history

1. 6 statements 26 Aug 2026, 12:47
   What was added

   A workflow orchestrator then runs each step as an isolated computational process that interacts solely through explicit inputs and outputs.

   On the record as reference 2 go to the sentence ↑
   What was added

   Across these comparative runs, total wall-clock execution time and peak memory usage remained comparable to the traditional notebook and pipeline configurations.

   On the record as reference 2 go to the sentence ↑
   What was added

   For the localization workflows tested, adapting experiments to new settings required fewer edits to the codebase under LOCALIZE than under the baseline implementations.

   On the record as reference 2 go to the sentence ↑
   What was added

   In a controlled scaling test across one, five, and 10 times the base dataset volume, total central processing unit time and wall-clock duration grew sublinearly over the evaluated sizes.

   On the record as reference 2 go to the sentence ↑
   What was added

   The authors evaluated LOCALIZE qualitatively against five existing experimentation platforms, and quantitatively against matched implementations in Jupyter notebooks and the data pipeline tool Kedro.

   On the record as reference 2 go to the sentence ↑
   What was added

   The system provides preconfigured datasets, data processing stages, model-development procedures, and experiment templates tailored to radio localization, while allowing researchers to modify the underlying pipeline when needed.

   On the record as reference 2 go to the sentence ↑
   CF Carolina Fortuna · Contributor A researcher at the Jožef Stefan Institute.
2. Published 28 Aug 2026, 09:44
   Assembled by the Primary desk from 1 source · 14 cited sentences