forked from SWISDATA/processintel
io_operations: define a versioned, data-only model export format #53
Labels
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
ffrauenberger/ProcessIntel#53
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Severity: improvement (medium)
Where
app/ui/export_ui/export_controller.py:158-166app/io_operations/export_operations.py:77-106app/io_operations/import_operations.py:51-69Problem
The "native model format" is
pickle.dumps(mining_model). Besides the security problem (#32):graphviz.Digraph, numpy arrays, logger objects and the full event log with filtered copies, so files are large and opaque.Related: #32.
Likely a duplicate of upstream GitHub issue SWISDATA/processintel#6 Model Format and Import.
Fix
A versioned, documented export consisting of:
Re-mining from log + parameters is deterministic for all algorithms except Genetic; add a random seed parameter for that one.
from
issues/ARCH-008-model-persistence-via-pickle-of-runtime-objects.md(code analysis of commit6bcc646, 2026-10-03)Reviewed by Codex (GPT-6) against origin/main
6bcc646eb464.Review: partially correct.
The persistence/portability concern is supported, but “any refactoring makes all existing exports unreadable” is incorrect. Pickle can survive compatible attribute changes or a refactor that preserves import paths/compatibility aliases; module/class moves and incompatible state changes are the relevant risks.
Also, a Genetic seed parameter must control both Python
randomand NumPy's RNG, and normalize set iteration order (_create_partitionturns a set into a list before shuffling). A seed alone does not guarantee identical exports across processes/hash seeds or dependency versions. If exact model preservation is required, store the mined result as data as well as log/parameters; distinguish equivalent mined behavior from byte-identical graph output.Code references: app/io_operations/export_operations.py:106, app/mining_algorithms/genetic_mining.py:562, app/mining_algorithms/genetic_mining.py:612.