feat: 增加每日内存回收与 malloc arena 限制

流媒体会话会在 glibc malloc arena 中留下大量不归还操作系统的冷匿名内存，实测单个服务可累积到 2.5 GB 并占满整个 swap 文件（33 个 64/128 MB 匿名段，空闲时不增长，随会话单调抬升）。本次从根因与兜底两侧处理： - systemd/maa-online-server.service: 设置 MALLOC_ARENA_MAX=2，把线程 arena 数量从 8xCPU 压到 2，抑制 arena 膨胀 - systemd/maa-daily.service: 单元结束时经 ExecStopPost 重启 API 服务，回收当次会话残留 - systemd/maa-online-recycle.service, maa-online-recycle.timer: 新增每日 03:59（游戏每日刷新前）独立回收定时器，错开更新与日常任务窗口，Persistent=false 避免补跑撞车 - bin/maa-online-can-recycle: 回收守卫，检测日常任务（systemd 单元或手工运行）是否进行中，是则跳过回收，避免重启打断任务 - bin/maa-online-install-systemd: 安装并启用新增的回收单元 - README.md, user_config/README.md: 补充内存回收机制与定时器说明
2026-09-19 23:58:54 +08:00
parent 0264053b08
commit 7806524279
8 changed files with 84 additions and 10 deletions
@@ -146,14 +146,20 @@ sudo bin/maa-online-install-systemd

 | 单元 | 作用 |
 |---|---|
-| `maa-online-server.service` | 云游戏 API 常驻服务（`Restart=always`，启动前清除代理环境变量） |
+| `maa-online-server.service` | 云游戏 API 常驻服务（`Restart=always`，启动前清除代理环境变量，限制 malloc arena 数量） |
 | `maa-daily.timer` | 每天 04:00 起随机 0–2 小时触发日常任务 |
+| `maa-online-recycle.timer` | 每天 03:59（游戏每日刷新前）重启 API 服务，回收内存 |
 | `maa-update.timer` | 每周日 03:20 起随机 30 分钟执行更新 |

 日常任务由 `bin/maa-online-daily-run` 驱动：等待 API 就绪 → 启动云游戏 → 在停滞看门狗下执行
 `config/tasks/daily.toml` → 失败则释放云会话并重试。定时器使用 `Persistent=true`，机器重启
 错过触发时间后会自动补跑；同一单元不会并发执行。

+**内存回收**：一次流媒体会话会在 glibc 的 malloc arena 里留下大量不归还操作系统的冷匿名内存
+（实测单个服务可累积到 2.5 GB 并填满 swap），因此有两层回收：`maa-daily.service` 在结束时
+重启 API 服务（`ExecStopPost`），`maa-online-recycle.timer` 每天再独立回收一次。回收服务带
+`ExecCondition=bin/maa-online-can-recycle` 守卫，日常任务运行期间会被自动跳过，不会打断任务。
+
 重试采用**断点续跑**：`bin/maa-resume-tasks.py` 读取前几次尝试的日志，跳过已出现
 `<任务类型> Completed` 的任务，只运行剩余任务；若全部完成则直接判定成功，不再开启新会话。
 `StartUp` 永不跳过，因为每次 `/start` 都是全新的云实例。详见
@@ -162,7 +168,7 @@ sudo bin/maa-online-install-systemd
 查看运行情况：

 ```bash
-systemctl list-timers maa-daily.timer maa-update.timer
+systemctl list-timers maa-daily.timer maa-update.timer maa-online-recycle.timer
 journalctl -u maa-daily.service -n 50
 journalctl -u maa-online-server.service -n 50
 ```
@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+# Guard for maa-online-recycle.service.
+#
+# Exit 0 when it is safe to restart the cloud game API server, exit 1 when a
+# daily run is in progress. Restarting the server underneath a running task
+# breaks that attempt, and the recycle timer fires right next to the daily
+# window, so the recycle must never race with it.
+set -uo pipefail
+
+state=$(systemctl show -p ActiveState --value maa-daily.service 2>/dev/null || echo unknown)
+case "$state" in
+    active|activating|reloading|deactivating)
+        echo "maa-daily.service is '$state'; skipping recycle" >&2
+        exit 1
+        ;;
+esac
+
+# Cover a run started by hand (bin/maa-online-daily-run outside systemd).
+if pgrep -x maa >/dev/null 2>&1; then
+    echo "a maa task is running; skipping recycle" >&2
+    exit 1
+fi
+
+exit 0
@@ -7,7 +7,7 @@ set -euo pipefail
 ROOT="${MAA_ONLINE_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
 SRC="$ROOT/systemd"
 DST="${MAA_ONLINE_SYSTEMD_DIR:-/etc/systemd/system}"
-UNITS=(maa-online-server.service maa-daily.service maa-daily.timer maa-update.service maa-update.timer)
+UNITS=(maa-online-server.service maa-daily.service maa-daily.timer maa-update.service maa-update.timer maa-online-recycle.service maa-online-recycle.timer)

 if [ "$(id -u)" != "0" ]; then
    echo "error: must run as root (writes to $DST)" >&2
@@ -25,15 +25,15 @@ done

 systemctl daemon-reload
 systemctl enable --now maa-online-server.service
-systemctl enable maa-daily.timer maa-update.timer
-systemctl start maa-daily.timer maa-update.timer
+systemctl enable maa-daily.timer maa-update.timer maa-online-recycle.timer
+systemctl start maa-daily.timer maa-update.timer maa-online-recycle.timer

 echo
-systemctl --no-pager list-timers maa-daily.timer maa-update.timer
+systemctl --no-pager list-timers maa-daily.timer maa-update.timer maa-online-recycle.timer
 cat <<EOF

 MAA Online systemd units are active.
  server log:  journalctl -u maa-online-server -f
  daily log:   journalctl -u maa-daily -n 50
-  next runs:   systemctl list-timers maa-daily.timer maa-update.timer
+  next runs:   systemctl list-timers maa-daily.timer maa-update.timer maa-online-recycle.timer
 EOF
@@ -9,6 +9,10 @@ ExecStart=__ROOT__/bin/maa-online-daily-run
 # A normal run takes about 25 minutes; the runner's own per-attempt watchdog
 # aborts a hung session well before this ceiling.
 TimeoutStartSec=7200
-# Safety net: whatever stops the unit (success, failure, timeout), make sure
-# the remote cloud session is released.
+# Safety net: whatever stops the unit (success, failure, timeout), release the
+# remote cloud session first...
 ExecStopPost=/bin/sh -c "/usr/bin/curl --noproxy '*' -fsS -X POST http://127.0.0.1:22888/exit || true"
+# ...then recycle the API server. A streaming session leaves gigabytes of cold
+# anonymous memory in glibc malloc arenas that is never returned to the OS;
+# without a recycle it accumulates until the whole swap file is used up.
+ExecStopPost=/bin/systemctl --no-block restart maa-online-server.service
@@ -0,0 +1,12 @@
+[Unit]
+Description=Recycle the MAA Online cloud game API server (reclaim swapped anonymous memory)
+
+[Service]
+Type=oneshot
+# Never recycle while a daily run is in progress: the timer fires next to the
+# daily window, so a run can already be underway.
+ExecCondition=__ROOT__/bin/maa-online-can-recycle
+# Streaming sessions churn large temporaries across the asyncio/aiohttp/aiortc
+# thread pool; glibc keeps the freed arenas, so a restart is what actually
+# returns the memory (including anything parked in swap) to the OS.
+ExecStart=/bin/systemctl restart maa-online-server.service
@@ -0,0 +1,16 @@
+[Unit]
+Description=Daily recycle of the MAA Online API server just before the Arknights daily reset
+
+[Timer]
+# 03:59 sits immediately ahead of the 04:00 in-game daily reset and of the
+# maa-daily.timer window (04:00 + up to 2h), so the API server is freshly
+# recycled exactly when the daily run starts. Adjust both if the game server's
+# reset time differs.
+OnCalendar=*-*-* 03:59:00
+RandomizedDelaySec=0
+# No catch-up run: a late recycle could otherwise land inside the daily window.
+Persistent=false
+Unit=maa-online-recycle.service
+
+[Install]
+WantedBy=timers.target
@@ -11,6 +11,12 @@ RestartSec=5
 # Belt and braces: never let a proxy environment leak into the cloud gaming
 # server (bin/maa-online-server already unsets these).
 Environment=NO_PROXY=localhost,127.0.0.1,::1
+# Streaming sessions churn large temporaries (video frames, PNG buffers) across
+# the asyncio/aiohttp/aiortc thread pool. glibc gives each thread its own 64 MB
+# malloc arena and never returns it to the OS, so the service used to ratchet up
+# to gigabytes of cold anonymous memory and eventually fill the swap file.
+# Capping the arena count keeps the footprint flat.
+Environment=MALLOC_ARENA_MAX=2

 [Install]
 WantedBy=multi-user.target
@@ -202,13 +202,19 @@ curl -X POST http://127.0.0.1:22888/exit

 | 单元 | 作用 |
 |---|---|
-| `maa-online-server.service` | 云游戏 API 常驻服务，崩溃自动重启，启动前清除代理环境变量 |
+| `maa-online-server.service` | 云游戏 API 常驻服务，崩溃自动重启，启动前清除代理环境变量，限制 malloc arena 数量 |
 | `maa-daily.timer` | 每天 04:00 起随机 0–2 小时触发 |
+| `maa-online-recycle.timer` | 每天 03:59（游戏每日刷新前）重启 API 服务以回收内存 |
 | `maa-update.timer` | 每周日 03:20 起随机 30 分钟，更新 maa-cli / MaaCore / 资源 |

 `bin/maa-online-daily-run` 的流程：等待 API 就绪 → 规划剩余任务 → 启动云游戏 → 在停滞看门狗下
 执行任务 → 失败则释放云会话并重试（默认最多 3 次）。

+**内存回收**：流媒体会话会在 glibc malloc arena 中留下大量不归还操作系统的冷匿名内存（实测可累积
+到 GB 级并占满 swap）。因此 `maa-daily.service` 结束时会重启 API 服务，另有
+`maa-online-recycle.timer` 每天独立回收一次；回收服务在 `bin/maa-online-can-recycle` 守卫下运行，
+日常任务进行期间会自动跳过，不会打断任务。
+
 看门狗会在出现连续 `ScreencapFailed` 或日志中的 `Disconnected` 时中止当前尝试；后端本身也会在信令
 WebSocket 被远端关闭、或视频流停滞超过 `MAA_ONLINE_STALE_LIMIT_SECONDS`（默认 120 秒）时主动拆掉
 会话，避免云实例空挂。