mining.base: absolute frequency thresholds use the wrong maximum, and slider values jump #11

Open
opened 2026-10-06 13:46:13 +00:00 by ffrauenberger · 1 comment
Owner

Severity: medium (the absolute slider applies a different cutoff than the user set)

Where

  • app/mining_algorithms/base_mining.py:475-505, :507-542, :632-637, :669-783

Problem

The UI offers Node/Edge Frequency (absolute) sliders next to the normalized ones. Filtering only ever uses the normalized value (get_node_frequencies, filter_edge); the absolute slider only drives the normalized value through _sync_thresholds.

Within one recalculation, that sync uses different maxima:

  1. The first call normalizes against self.appearance_frequency, the unfiltered log, before the SPM filter runs (recalculate_model_filters, lines 690-694).
  2. The actual filter normalizes against the maximum count in the SPM-filtered log (get_node_frequencies).
  3. The second call recomputes the absolute value against filtered_appearance_freqs.

If the SPM filter removes the most frequent activity, these maxima differ. Example: the user sets "absolute ≥ 50", the unfiltered max is 200 and the SPM-filtered max is 100. Then norm = 0.25, nodes with count ≥ 25 are kept, and the slider jumps back to 25.

Also, get_node_statistics stores the normalized value from self.node_frequencies under the key "absolute_frequency" (lines 523-538).

Steps to reproduce

  1. Open the application and import a log
  2. Select a miner with SPM and absolute frequency sliders
  3. Raise the SPM threshold until the most frequent activity is filtered out
  4. Set Node Frequency (absolute) to a value, e.g. 50

Expected behavior

Only nodes with an absolute frequency ≥ 50 are kept, and the slider keeps the value 50.

Actual behavior

A different cutoff is applied (e.g. ≥ 25) and the slider value jumps to that value.

Related: #46.

Fix

Filter on the absolute value directly, or derive both values from one consistently defined maximum (the one actually used for filtering). Fix or rename the absolute_frequency field in the node statistics.


from issues/BUG-011-absolute-frequency-thresholds-use-wrong-basis.md (code analysis of commit 6bcc646, 2026-10-03)

**Severity:** medium (the absolute slider applies a different cutoff than the user set) ## Where - `app/mining_algorithms/base_mining.py:475-505`, `:507-542`, `:632-637`, `:669-783` ## Problem The UI offers *Node/Edge Frequency (absolute)* sliders next to the normalized ones. Filtering only ever uses the **normalized** value (`get_node_frequencies`, `filter_edge`); the absolute slider only drives the normalized value through `_sync_thresholds`. Within one recalculation, that sync uses different maxima: 1. The first call normalizes against `self.appearance_frequency`, the **unfiltered** log, before the SPM filter runs (`recalculate_model_filters`, lines 690-694). 2. The actual filter normalizes against the maximum count in the **SPM-filtered** log (`get_node_frequencies`). 3. The second call recomputes the absolute value against `filtered_appearance_freqs`. If the SPM filter removes the most frequent activity, these maxima differ. Example: the user sets "absolute ≥ 50", the unfiltered max is 200 and the SPM-filtered max is 100. Then norm = 0.25, nodes with count ≥ 25 are kept, and the slider jumps back to 25. Also, `get_node_statistics` stores the **normalized** value from `self.node_frequencies` under the key `"absolute_frequency"` (lines 523-538). ### Steps to reproduce 1. Open the application and import a log 2. Select a miner with SPM and absolute frequency sliders 3. Raise the SPM threshold until the most frequent activity is filtered out 4. Set *Node Frequency (absolute)* to a value, e.g. 50 ### Expected behavior Only nodes with an absolute frequency ≥ 50 are kept, and the slider keeps the value 50. ### Actual behavior A different cutoff is applied (e.g. ≥ 25) and the slider value jumps to that value. Related: #46. ## Fix Filter on the absolute value directly, or derive both values from one consistently defined maximum (the one actually used for filtering). Fix or rename the `absolute_frequency` field in the node statistics. --- from `issues/BUG-011-absolute-frequency-thresholds-use-wrong-basis.md` (code analysis of commit 6bcc646, 2026-10-03)
Author
Owner

Reviewed by Codex (GPT-6) against origin/main 6bcc646eb464.

Review: partially correct.

The inconsistent normalization maxima and mislabeled absolute_frequency are supported. The ordering in point 1 needs correction: recalculate_model_filters() recomputes the SPM-filtered events/log at lines 685–688 before its first _sync_thresholds() call at 690–694. The actual defect is that the call still receives unfiltered self.appearance_frequency, while get_node_frequencies() normalizes against counts in the already recomputed SPM-filtered log. This preserves the reported cutoff/jumping problem but corrects its explanation.

Code references: app/mining_algorithms/base_mining.py:685, app/mining_algorithms/base_mining.py:690, app/mining_algorithms/base_mining.py:523.

Reviewed by **Codex (GPT-6)** against origin/main [`6bcc646eb464`](https://code.swisdata.eu/ffrauenberger/ProcessIntel/commit/6bcc646eb464e0dd897228b5e49867f65d7255c4). **Review: partially correct.** The inconsistent normalization maxima and mislabeled `absolute_frequency` are supported. The ordering in point 1 needs correction: `recalculate_model_filters()` recomputes the SPM-filtered events/log at lines 685–688 **before** its first `_sync_thresholds()` call at 690–694. The actual defect is that the call still receives **unfiltered** `self.appearance_frequency`, while `get_node_frequencies()` normalizes against counts in the already recomputed SPM-filtered log. This preserves the reported cutoff/jumping problem but corrects its explanation. Code references: [app/mining_algorithms/base_mining.py:685](https://code.swisdata.eu/ffrauenberger/ProcessIntel/src/commit/6bcc646eb464e0dd897228b5e49867f65d7255c4/app/mining_algorithms/base_mining.py#L685), [app/mining_algorithms/base_mining.py:690](https://code.swisdata.eu/ffrauenberger/ProcessIntel/src/commit/6bcc646eb464e0dd897228b5e49867f65d7255c4/app/mining_algorithms/base_mining.py#L690), [app/mining_algorithms/base_mining.py:523](https://code.swisdata.eu/ffrauenberger/ProcessIntel/src/commit/6bcc646eb464e0dd897228b5e49867f65d7255c4/app/mining_algorithms/base_mining.py#L523).
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ffrauenberger/ProcessIntel#11
No description provided.