ui: reduce the number of event log copies held per session #45

Open
opened 2026-10-06 13:52:01 +00:00 by ffrauenberger · 0 comments
Owner

Severity: improvement (medium for multi-user hosting)

Where

  • app/mining_algorithms/base_mining.py:16-146
  • app/ui/base_algorithm_ui/base_algorithm_controller.py:188-223, :294-327
  • app/ui/home_ui/home_controller.py

Problem

For a single session, the following can exist at the same time:

  • the uploaded file (Streamlit UploadedFile, in memory) for up to three uploaders (main, CSV→XES, XES→CSV);
  • the parsed DataFrame and the converter outputs (converted_csv_df, converted_csv_data, converted_xes_bytes);
  • the mining model with log, spm_filtered_log, node_frequency_filtered_log, two n×n matrices and algorithm-specific copies (e.g. filtered_log in Inductive, populations in Genetic);
  • the current graph and a second "fixed layout" graph (graph_fixed_position), each holding a graphviz Digraph.

With the 200 MB upload limit and several concurrent users, a single process can quickly run out of memory.

Short-term part of upstream GitHub issue SWISDATA/processintel#17 Update application to run with constant memory.

Fix

  • Release the uploaded file once the log dict is built (the DataFrame is already deleted; also clear the uploader)
  • Keep converter results only until they are downloaded
  • Avoid duplicate filtered logs where a view or index suffices
  • Set a per-session or global memory budget (#31)

from issues/PERF-009-session-memory-footprint.md (code analysis of commit 6bcc646, 2026-10-03)

**Severity:** improvement (medium for multi-user hosting) ## Where - `app/mining_algorithms/base_mining.py:16-146` - `app/ui/base_algorithm_ui/base_algorithm_controller.py:188-223`, `:294-327` - `app/ui/home_ui/home_controller.py` ## Problem For a single session, the following can exist at the same time: - the uploaded file (Streamlit `UploadedFile`, in memory) for up to three uploaders (main, CSV→XES, XES→CSV); - the parsed DataFrame and the converter outputs (`converted_csv_df`, `converted_csv_data`, `converted_xes_bytes`); - the mining model with `log`, `spm_filtered_log`, `node_frequency_filtered_log`, two n×n matrices and algorithm-specific copies (e.g. `filtered_log` in Inductive, populations in Genetic); - the current graph **and** a second "fixed layout" graph (`graph_fixed_position`), each holding a graphviz `Digraph`. With the 200 MB upload limit and several concurrent users, a single process can quickly run out of memory. Short-term part of upstream GitHub issue [SWISDATA/processintel#17](https://github.com/SWISDATA/processintel/issues/17) *Update application to run with constant memory*. ## Fix - Release the uploaded file once the log dict is built (the DataFrame is already deleted; also clear the uploader) - Keep converter results only until they are downloaded - Avoid duplicate filtered logs where a view or index suffices - Set a per-session or global memory budget (#31) --- from `issues/PERF-009-session-memory-footprint.md` (code analysis of commit 6bcc646, 2026-10-03)
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ffrauenberger/ProcessIntel#45
No description provided.