nix.module: one app process serves all users, so mining uses only one CPU core #65

Open
opened 2026-10-08 06:15:17 +00:00 by ffrauenberger · 0 comments
Owner

Severity: improvement (high for the hosted service: 10 concurrent mining users make every run about 10× slower, while the other cores stay idle)

Where

  • module.nix:191-199 — the service starts one processintel process
  • module.nix:267-275 — the nginx upstream has one server
  • app/__main__.py — one uvicorn.run(app, …) serves streamlit.App for all users
  • app/ui/base_algorithm_ui/base_algorithm_view.py:188 — mining runs in the session's script thread

Problem

All users share one Python process. Streamlit runs each session's script in its own thread (ScriptRunner.scriptThread), but all threads share the GIL. Most mining code is pure Python. So all users together get the CPU time of one core, however many cores the host has (services#one-process-is-not-a-machine).

#31 and #54 state this, but it was never measured. Here is the measurement.

Setup:

  • Streamlit 1.58.0 and Python 3.13.13 with the GIL, on a 16-core host.
  • Each session builds GeneticMining(log) and calls generate_graph, as the controller does.
  • The log is tests/testcsv/CallcenterExample.csv (416 variants, 3,885 cases), with population 100 and 200 generations.
  • All sessions start at once, one thread each. Every run gets the same run ID, so that the global run ID (#3) does not cancel the others.
Concurrent sessions Each run takes All done after
1 1.31 s 1.31 s
2 2.48 s 2.58 s
5 6.49 s 7.18 s
10 11.62 s 14.84 s
10, one process each (ProcessPoolExecutor) 2.25 s 2.74 s

The total time grows linearly with the number of sessions (about n × 1.3 s), so the runs are serialised on one core. The same ten runs in ten processes finish 5.4× faster. A synthetic check (1 vs. 10 threads) gave about 8× slower runs for a pure-Python loop, and about 4× for NumPy matrix products, which release the GIL.

What this means:

  • Every user waits for every other user's mining.
  • The event loop that serves all WebSockets needs the GIL too, so the UI gets slower for everyone.
  • A bigger host adds no mining capacity.

#54 is the code-level fix: mining in a process pool. It needs #46 first, so it takes time. The deployment can spread users over cores now, without code changes. Neither #31 nor #54 covers that.

Fix

  1. Add an instances option to module.nix (default 1). Run one processintel process per instance, each on its own socket (e.g. a template unit processintel@<n>).
  2. List all instance sockets in the nginx upstream, with sticky sessions (ip_hash, or a cookie hash). A Streamlit session lives in the memory of one process, and its WebSocket and uploads must reach that same process. uvicorn.run(app, workers=N) is no alternative: Uvicorn needs an import string for workers, and it would not keep sessions sticky.
  3. Put the memory limit per instance (MemoryMax=, #31), so one large upload restarts only one instance.
  4. Add the benchmark as a script, for example under scripts/ (not a unit test, because timings vary). Record the capacity per instance in the deployment docs (testing#measure-before-optimizing).

This does not replace #54: within one instance, sessions still share a core. It also does not replace the limits of #31 (services#everything-is-bounded).

Check

  • With instances = 4, ten concurrent Genetic runs from different clients finish in about a quarter of the single-instance time.
  • A session stays on its instance: upload, mine, change a slider and export, with no lost session state.
  • Reproduce the single-process numbers in nix develop .#processintel --impure, with PYTHONPATH=.:$PYTHONPATH:
import threading, time
import pandas as pd
import app.mining_algorithms.genetic_mining as gm
from app.mining_algorithms.genetic_mining import GeneticMining
from app.transformations.dataframe_transformations import DataframeTransformations

gm.uuid.uuid4 = lambda: "same-run-id"  # keep the global run ID (#3) from cancelling the other runs

dt = DataframeTransformations()
dt.set_dataframe(pd.read_csv("tests/testcsv/CallcenterExample.csv"))
log = dt.dataframe_to_cases_dict("Start Date", "Service ID", "Operation")

def session():
    GeneticMining(log).generate_graph(0.0, 0.0, 0, 100, 200, 0.8, 0.2, 0.1, 5, 1, 1.01)

for n in (1, 10):
    threads = [threading.Thread(target=session) for _ in range(n)]
    t0 = time.perf_counter()
    for t in threads: t.start()
    for t in threads: t.join()
    print(f"{n} sessions: all done after {time.perf_counter() - t0:.2f}s")

Related: #31, #54, #39, #45.


from a session on 2026-10-08 (measured on commit 6bcc646); draft in issues/PERF-010-all-sessions-share-one-cpu-core.md

**Severity:** improvement (high for the hosted service: 10 concurrent mining users make every run about 10× slower, while the other cores stay idle) ## Where - `module.nix:191-199` — the service starts one `processintel` process - `module.nix:267-275` — the nginx upstream has one server - `app/__main__.py` — one `uvicorn.run(app, …)` serves `streamlit.App` for all users - `app/ui/base_algorithm_ui/base_algorithm_view.py:188` — mining runs in the session's script thread ## Problem All users share one Python process. Streamlit runs each session's script in its own thread (`ScriptRunner.scriptThread`), but all threads share the GIL. Most mining code is pure Python. So all users together get the CPU time of **one core**, however many cores the host has (`services#one-process-is-not-a-machine`). #31 and #54 state this, but it was never measured. Here is the measurement. **Setup:** - Streamlit 1.58.0 and Python 3.13.13 with the GIL, on a 16-core host. - Each session builds `GeneticMining(log)` and calls `generate_graph`, as the controller does. - The log is `tests/testcsv/CallcenterExample.csv` (416 variants, 3,885 cases), with population 100 and 200 generations. - All sessions start at once, one thread each. Every run gets the same run ID, so that the global run ID (#3) does not cancel the others. | Concurrent sessions | Each run takes | All done after | |---|---|---| | 1 | 1.31 s | 1.31 s | | 2 | 2.48 s | 2.58 s | | 5 | 6.49 s | 7.18 s | | 10 | 11.62 s | 14.84 s | | 10, one process each (`ProcessPoolExecutor`) | 2.25 s | 2.74 s | The total time grows linearly with the number of sessions (about n × 1.3 s), so the runs are serialised on one core. The same ten runs in ten processes finish 5.4× faster. A synthetic check (1 vs. 10 threads) gave about 8× slower runs for a pure-Python loop, and about 4× for NumPy matrix products, which release the GIL. What this means: - Every user waits for every other user's mining. - The event loop that serves all WebSockets needs the GIL too, so the UI gets slower for everyone. - A bigger host adds no mining capacity. #54 is the code-level fix: mining in a process pool. It needs #46 first, so it takes time. The deployment can spread users over cores now, without code changes. Neither #31 nor #54 covers that. ## Fix 1. Add an `instances` option to `module.nix` (default 1). Run one `processintel` process per instance, each on its own socket (e.g. a template unit `processintel@<n>`). 2. List all instance sockets in the nginx upstream, with sticky sessions (`ip_hash`, or a cookie hash). A Streamlit session lives in the memory of one process, and its WebSocket and uploads must reach that same process. `uvicorn.run(app, workers=N)` is no alternative: Uvicorn needs an import string for workers, and it would not keep sessions sticky. 3. Put the memory limit per instance (`MemoryMax=`, #31), so one large upload restarts only one instance. 4. Add the benchmark as a script, for example under `scripts/` (not a unit test, because timings vary). Record the capacity per instance in the deployment docs (`testing#measure-before-optimizing`). This does not replace #54: within one instance, sessions still share a core. It also does not replace the limits of #31 (`services#everything-is-bounded`). ## Check - With `instances = 4`, ten concurrent Genetic runs from different clients finish in about a quarter of the single-instance time. - A session stays on its instance: upload, mine, change a slider and export, with no lost session state. - Reproduce the single-process numbers in `nix develop .#processintel --impure`, with `PYTHONPATH=.:$PYTHONPATH`: ```python import threading, time import pandas as pd import app.mining_algorithms.genetic_mining as gm from app.mining_algorithms.genetic_mining import GeneticMining from app.transformations.dataframe_transformations import DataframeTransformations gm.uuid.uuid4 = lambda: "same-run-id" # keep the global run ID (#3) from cancelling the other runs dt = DataframeTransformations() dt.set_dataframe(pd.read_csv("tests/testcsv/CallcenterExample.csv")) log = dt.dataframe_to_cases_dict("Start Date", "Service ID", "Operation") def session(): GeneticMining(log).generate_graph(0.0, 0.0, 0, 100, 200, 0.8, 0.2, 0.1, 5, 1, 1.01) for n in (1, 10): threads = [threading.Thread(target=session) for _ in range(n)] t0 = time.perf_counter() for t in threads: t.start() for t in threads: t.join() print(f"{n} sessions: all done after {time.perf_counter() - t0:.2f}s") ``` Related: #31, #54, #39, #45. --- from a session on 2026-10-08 (measured on commit 6bcc646); draft in `issues/PERF-010-all-sessions-share-one-cpu-core.md`
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ffrauenberger/ProcessIntel#65
No description provided.