From 9b9c501fa90588e138d37a5a92cef4994c3f0758 Mon Sep 17 00:00:00 2001 From: RRRosmontis Date: Wed, 23 Sep 2026 11:53:12 +0800 Subject: [PATCH] =?UTF-8?q?feat(dufe):=20=E6=96=B0=E5=A2=9E=20mode=3Dauto?= =?UTF-8?q?=20=E8=87=AA=E5=8A=A8=E7=AD=BE=E5=88=B0=EF=BC=8C=E9=9D=9E?= =?UTF-8?q?=E7=AD=BE=E5=88=B0=E7=A0=81=E9=80=BB=E8=BE=91=E4=BB=A5=20bot=20?= =?UTF-8?q?=E4=B8=BA=E5=9F=BA=E5=87=86=E5=90=8C=E6=AD=A5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - mode=auto: 通知列表(getMessageAll+busiMessage/list)发现签到活动 (bot ginkgo 同款过滤, 按 signInId 去重) → 逐个自动签到 - 连续重试: 首次+最多5次间隔3s (对齐 bot MAX_SIGNIN_ATTEMPTS/SIGNIN_BACKOFF), token 失效重登录一次立即重试; 限流中断本轮; 结果聚合 data.results 列表 - 基准同步 (bot ginkgo): 移植限流防护体系(check_response/is_rate_limited/ safe_error/_LoginErrors/_request), parse_course_url courseId 放宽 [0-9A-Za-z]+, commit_sign_in 错误包装, fetch_sign_in_info 增加 check_response 与 sess/token 注入 (auto 模式请求内复用登录态) - 验证: 免凭据单测 + 假凭据 401 线路测试 + bj 真账密只读联调 (notices=20/busi=18/活动12 类型解析正确) --- README.md | 7 +- docs/部署与运维.md | 16 +- .../dufe/ginkgostu_sign_in/ginkgo_sign_in.py | 686 ++++++++++++++++-- 3 files changed, 632 insertions(+), 77 deletions(-) diff --git a/README.md b/README.md index d0ae73a..6588e36 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ | openapi | `127.0.0.1:9008` | `source/main.py`（aiohttp） | 聚合层，纯转发，**唯一入口** | | pixiv | `127.0.0.1:9007` | `source/pixiv/pixiv.py`（Flask + waitress） | 图片代理 | | bilibili | `127.0.0.1:9006` | `source/bilibili/bilibili.py`（aiohttp + yt-dlp） | 视频代理 | -| dufe 签到 | `127.0.0.1:9005` | `source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py`（Flask + waitress 16 线程） | 银杏（DUFE）签到码查询，`GET\|POST /dufe/ginkgostu_sign_in` | +| dufe 签到 | `127.0.0.1:9005` | `source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py`（Flask + waitress 16 线程） | 银杏（DUFE）签到码查询，`GET\|POST /dufe/ginkgostu_sign_in`；`&mode=auto` 时拉取通知列表自动签到（与 bot ginkgo 同逻辑，连续重试，结果列表聚合返回） | | jm | `127.0.0.1:9004` | `source/jm/jm.py`（aiohttp + jmcomic） | 漫画下载打包（移植自 qqbot jm 模块），`GET /jm/get_comic?id=xxx[&range=起-止]` | | pixiv-token-refresh | — | `source/pixiv/refresh_token.py` | token 刷新守护（每 30–60 分钟随机） | | bilibili-refresh | — | `source/bilibili/refresh_cookie.py` | cookie 刷新守护（每 50–70 分钟随机） | @@ -138,7 +138,10 @@ for p in 9004 9005 9006 9007 9008; do timeout 3 bash -c "cat < /dev/null > /dev/ ## 版本要点 -- **2026-09-23**：新增 **jm 漫画下载打包服务**（`source/jm/jm.py`，aiohttp + jmcomic，`127.0.0.1:9004`，端点 `GET /jm/get_comic?id=xxx[&range=起-止]`），移植自 qqbot 的 jm 模块（去掉 QQ/代理轮换/上传 cf 逻辑，改为 HTTP 直返 ZIP；hk 可直连源站无需代理）。ZIP 为临时产物：返回后 600 秒延时删除，TTL 窗口内同名请求直接复用。聚合层 `main.py` 注册路由（新增 `JM_BACKEND`，复用既有 `proxy()`），nginx 无需改动（`location /` 兜底） +- **2026-09-23** + - dufe 签到服务新增 **`mode=auto`**：拉取通知列表（getMessageAll + busiMessage/list）→ 按 bot ginkgo 同款规则发现签到活动（互动消息 + 文本含"签到" + 可解析课程链接，按 signInId 去重）→ 逐个自动签到；**连续重试**（首次 + 最多 5 次，间隔 3s，对齐 bot `MAX_SIGNIN_ATTEMPTS=6 / SIGNIN_BACKOFF`；token 疑似失效时重登录一次立即重试，对齐 bot refresh 语义）；**所有结果聚合为 `data.results` 列表**返回（每活动含 success/message/attempts/title/signInCode 等） + - 非签到码签到逻辑与 bot ginkgo 比对后以 bot 为基准同步：移植限流防护体系（`check_response` / `is_rate_limited` / `safe_error` / `_LoginErrors`）、`parse_course_url` 的 courseId 放宽至 `[0-9A-Za-z]+`、`commit_sign_in` 的错误包装、`fetch_sign_in_info` 的 check_response 与外部登录态注入（`sess`/`token` 参数，auto 模式请求内复用同一登录态） +- **2026-09-23**（早些）：新增 **jm 漫画下载打包服务**（`source/jm/jm.py`，aiohttp + jmcomic，`127.0.0.1:9004`，端点 `GET /jm/get_comic?id=xxx[&range=起-止]`），移植自 qqbot 的 jm 模块（去掉 QQ/代理轮换/上传 cf 逻辑，改为 HTTP 直返 ZIP；hk 可直连源站无需代理）。ZIP 为临时产物：返回后 600 秒延时删除，TTL 窗口内同名请求直接复用。聚合层 `main.py` 注册路由（新增 `JM_BACKEND`，复用既有 `proxy()`），nginx 无需改动（`location /` 兜底） - **2026-09-13**：新增 **dufe 银杏签到码查询服务**（`source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py`，Flask + waitress 16 线程，`127.0.0.1:9005`，端点 `GET|POST /dufe/ginkgostu_sign_in`），并在聚合层 `main.py` 注册该路由——为此新增 `proxy_forward()`（保留请求方法与请求体，原 `proxy()` 保持固定 GET 不变，零改动既有行为）。同期修复该服务两处缺陷：base64 参数补 `urllib.parse.unquote`（兼容 iOS 快捷指令的 `%3D` 编码），`commit` 判定改为 fail-closed（须 `commit.success is True` 才算签到成功） - **2026-09-12** - 端口重排 `9888/9088/19990 → 9008/9007/9006`，全部仅绑 `127.0.0.1`；pixiv 由 `0.0.0.0:9088` 收口 diff --git a/docs/部署与运维.md b/docs/部署与运维.md index 04a7146..6ab541c 100644 --- a/docs/部署与运维.md +++ b/docs/部署与运维.md @@ -11,7 +11,7 @@ |---|---| | `/pixiv/get_image?id=xxx` | `127.0.0.1:9008`（main.py 聚合层）→ `127.0.0.1:9007`（pixiv） | | `/bilibili/get_video?bv=xxx` | `127.0.0.1:9008`（main.py 聚合层）→ `127.0.0.1:9006`（bilibili） | -| `/dufe/ginkgostu_sign_in` | `127.0.0.1:9008`（main.py 聚合层）→ `127.0.0.1:9005`（dufe 签到码查询，支持 GET\|POST） | +| `/dufe/ginkgostu_sign_in` | `127.0.0.1:9008`（main.py 聚合层）→ `127.0.0.1:9005`（dufe 签到码查询，支持 GET\|POST；`&mode=auto` 自动签到） | | `/jm/get_comic?id=xxx[&range=起-止]` | `127.0.0.1:9008`（main.py 聚合层）→ `127.0.0.1:9004`（jm 漫画下载打包） | | `/img-master/`、`/img-original/` | nginx 直接反代 `i.pximg.net`（无本地后端） | | `/` | `127.0.0.1:9008`（302 跳转文章页） | @@ -134,7 +134,19 @@ ## 变更记录 -**2026-09-23** +**2026-09-23（dufe mode=auto + bot 基准同步）** + +- dufe 签到服务新增 **`&mode=auto`**（url 参数可省略）：SSO 登录一次后复用登录态 → 拉取通知列表（`GET /student/message/getMessageAll` pageSize=20/readFlag=0 + `POST /student/busiMessage/list`）→ 按 bot ginkgo 同款规则发现活动（仅互动消息、文本含"签到"、jumpUrl 可解析为 `/course/detail/.../signin//`，query 不入库，按 signInId 去重）→ 按时间序逐个自动签到 +- **连续重试**：每活动首次 + 最多 5 次重试、间隔 3 秒（`MAX_SIGNIN_ATTEMPTS=6` / `SIGNIN_BACKOFF=(3,3,3,3,3)`，对齐 bot）；上游错误/登录错时重登录一次立即重试（对齐 bot `_call_sign_in_local` 的 refresh 语义）；**除限流外任何未确认成功都消耗重试次数**（对齐 bot `complete_signin`）；限流即中断本轮，已完成结果照常返回（`data.rate_limited=true`） +- **结果聚合**：`data.results` 列表逐活动返回（activityKey/signInId/type/typeName/title/signInCode/status/success/message/attempts），另附 total/succeeded/failed/activities 计数 +- **与 bot ginkgo 非签到码签到逻辑比对后以 bot 为基准同步**： + - 移植限流防护体系：`RateLimitedError` / `_RATE_MESSAGE` / `_RATE_CODES` / `is_rate_limited` / `_valid_retry_after` / `_LoginErrors` / `check_response` / `safe_error` / `_request`（原实现完全无限流检测） + - `parse_course_url`：courseId 由 `\d+` 放宽为 `[0-9A-Za-z]+`，报错文案对齐 + - `commit_sign_in`：经 `_request` 包装（`check_response`），失败消息 "request to InTheSign failed: %s" % safe_error(e)（原为 "upstream request failed"） + - `fetch_sign_in_info`：getSignInInfo 响应过 `check_response`；新增 `sess`/`token` 参数注入（auto 模式请求内复用登录态，省去每活动一次 SSO）；补齐 "Malformed getSignInInfo response" 非 dict 校验 +- 验证：免凭据单测（发现/过滤正反用例）+ 假凭据 401 线路测试 + bj 真账密**只读**联调（登录 OK，notices=20/busi=18，发现 12 个活动含 1 个普通签到，类型/标题解析正确）；未做真实提交（避免与运行中的 bot 轮询交叉签到） + +**2026-09-23（早些）** - 新增 **jm 漫画下载打包服务**：`source/jm/jm.py` + `jm.service`（`127.0.0.1:9004`，aiohttp + jmcomic，端点 `GET /jm/get_comic?id=xxx[&range=起-止]`） - 移植自 qqbot 的 jm 模块（bj `C:/call/bot/utility/jm.py`）：保留章节范围解析 / 文件名清洗 / webp→jpg 打包逻辑；去掉 QQ 回复、代理轮换（hk 可直连源站）、上传 cf 文件服务器三部分，改为 HTTP 直返 ZIP diff --git a/source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py b/source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py index 07a4be6..5e8fd38 100644 --- a/source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py +++ b/source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py @@ -6,17 +6,27 @@ Ginkgo (DUFE) sign-in code query service. Endpoint: GET|POST /dufe/ginkgostu_sign_in params: username, password(base64), url(base64) [commit=true] optional, default true: also POST the code back + [mode=auto] optional: fetch the notification list, discover + sign-in activities and auto sign-in each one + (bot ginkgo parity, consecutive retries); + results returned as an aggregated list. + mode=auto 时 url 参数可省略。 Listen: 127.0.0.1:9005 (WSGI: waitress) """ import base64 import binascii +import html as _html import json import logging import re import sys +import time import urllib.parse import uuid +from datetime import datetime, timedelta, timezone +from email.utils import parsedate_to_datetime +from html.parser import HTMLParser try: import requests @@ -43,6 +53,17 @@ GINKGO = "https://ginkgostu.dufe.edu.cn" SSO = "https://sso.dufe.edu.cn" CUSTOM_CODE = "kb_dconline" +NOTICE_PAGE = GINKGO + "/notice" +SYSTEM_PAGE = GINKGO + "/notice/system" + +PAGE_SIZE = 20 # 单类单次抓取条数（与 bot ginkgo 一致） +# 自动签到重试参数（与 bot ginkgo 的 MAX_SIGNIN_ATTEMPTS / SIGNIN_BACKOFF 一致： +# 首次 + 最多 5 次重试，每次间隔 3 秒） +MAX_SIGNIN_ATTEMPTS = 6 +SIGNIN_BACKOFF = (3, 3, 3, 3, 3) + +TZ8 = timezone(timedelta(hours=8)) + UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " "(KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36") @@ -65,13 +86,183 @@ class AuthError(Exception): class NoSignInCodeError(Exception): - """The sign-in activity exists but has no code configured.""" + """The numeric-code activity exists but has no code configured.""" + + +class UnsupportedSigninType(Exception): + """The activity requires a flow this service cannot safely automate.""" + + +SIGNIN_TYPE_NAMES = { + 0: "二维码签到", + 1: "数字码签到", + 2: "普通签到", + 3: "直播签到", +} + + +def signin_type_label(value): + try: + signin_type = int(value) + except (TypeError, ValueError): + return "未知签到类型" + return SIGNIN_TYPE_NAMES.get(signin_type, "未知签到类型(%s)" % signin_type) class UpstreamError(Exception): """Remote service returned an unexpected or erroneous response.""" +class RateLimitedError(Exception): + """An upstream explicitly refused requests due to throttling.""" + + +# ------------------------------------------------------------------------------ +# Rate-limit detection & credential-safe errors +# (synced from bot ginkgostu.py 2026-09-23; bot is the baseline) +# ------------------------------------------------------------------------------ +_RATE_MESSAGE = re.compile( + r"too\s+many\s+requests|rate[\s_-]*limit(?:ed|ing|\s+exceeded)?|" + r"request(?:s)?\s+throttl|throttl(?:ed|ing)|" + r"(?:请求|访问|操作|登录|登陆|接口|提交).{0,12}(?:过于频繁|太频繁|频繁|频率.{0,6}(?:超|限制))|" + r"(?:限流|请求次数.{0,8}(?:超限|超出|超过)|访问次数.{0,8}(?:超限|超出|超过))", + re.I, +) +_RATE_CODES = {"429", "4290", "40029", "90029", "RATE_LIMIT", "RATE_LIMITED", "RATE_LIMIT_EXCEEDED", + "TOO_MANY_REQUESTS", "REQUEST_TOO_FREQUENT", "THROTTLED", + "THROTTLING", "FREQUENT_REQUESTS"} +_FIELDS = ("code", "msg", "message", "error") + + +def _status(value): + try: + return int(value) + except (ValueError, TypeError): + return 0 + + +def _explicit(value, depth=0): + if depth > 3: + return False + if isinstance(value, dict): + for key in _FIELDS: + item = value.get(key) + if key == "code" and isinstance(item, (str, int)): + if str(item).strip().upper().replace("-", "_") in _RATE_CODES: + return True + elif isinstance(item, dict): + if _explicit(item, depth + 1): + return True + elif isinstance(item, str) and _RATE_MESSAGE.search(item[:1024]): + return True + return False + + +class _LoginErrors(HTMLParser): + """Extract only explicitly marked login error elements; ignore scripts/notices.""" + def __init__(self): + super().__init__() + self.stack = [] + self.messages = [] + + def handle_starttag(self, tag, attrs): + attrs = dict(attrs) + marker = (attrs.get("id", "") + " " + attrs.get("class", "")).lower() + explicit = bool(re.search(r"(?:^|[\s_-])(?:error|errors|errormsg|errormessage)(?:$|[\s_-])", marker)) + ignored = tag in ("script", "style", "template") or bool(self.stack and self.stack[-1][2]) + active = explicit or bool(self.stack and self.stack[-1][1]) + if tag not in ("input", "br", "img", "meta", "link", "hr", "wbr", "source"): + self.stack.append((tag, active, ignored)) + + def handle_endtag(self, tag): + for index in range(len(self.stack) - 1, -1, -1): + if self.stack[index][0] == tag: + del self.stack[index:] + break + + def handle_data(self, data): + if self.stack and self.stack[-1][1] and not self.stack[-1][2]: + self.messages.append(data) + + +def is_rate_limited(status, payload=None, text=""): + """Check HTTP status and explicit error fields, never arbitrary payload data/HTML.""" + status = _status(status) + if status == 429 or _explicit(payload): + return True + # Plain response bodies are evidence only on HTTP errors, and must be short. + if status >= 400 and isinstance(text, str): + plain = text.strip() + if len(plain) <= 1024 and not re.search(r"<[^>]+>", plain): + return bool(_RATE_MESSAGE.search(plain)) + return False + + +def _valid_retry_after(value): + if not isinstance(value, str) or not value.strip(): + return False + value = value.strip() + if re.fullmatch(r"[0-9]+", value): + return True + try: + return parsedate_to_datetime(value).tzinfo is not None + except (ValueError, TypeError, OverflowError): + return False + + +def check_response(response, what="upstream"): + """Return response unchanged, or raise RateLimitedError before caller parsing. + + `what` is deliberately not interpolated: URLs and upstream bodies may contain + credentials. Safe to use as a requests response hook through a small lambda. + """ + if response is None: + return response + status = _status(getattr(response, "status_code", 0)) + message = "上游请求过于频繁，请稍后重试" + if status == 429: + raise RateLimitedError(message) + headers = getattr(response, "headers", {}) or {} + retry_after = next((v for k, v in headers.items() if str(k).lower() == "retry-after"), None) + if status >= 400 and _valid_retry_after(retry_after): + raise RateLimitedError(message) + try: + payload = response.json() + except (ValueError, TypeError, AttributeError): + payload = None + text = getattr(response, "text", "") or "" + if is_rate_limited(status, payload, text): + raise RateLimitedError(message) + # Login pages can return HTTP 200 with an explicit error element. Never scan + # the whole successful HTML document, inline scripts, or announcement text. + if payload is None and isinstance(text, str) and "<" in text: + parser = _LoginErrors() + try: + parser.feed(text[:262144]) + except (ValueError, TypeError): + pass + if _RATE_MESSAGE.search(" ".join(parser.messages)[:1024]): + raise RateLimitedError(message) + return response + + +def safe_error(exception): + """Propagate explicit throttling, otherwise return a credential-free diagnostic. + + Never stringify arbitrary exceptions: requests URLs, query strings, tokens, + passwords and JSON decode snippets can all include secrets. + """ + if isinstance(exception, RateLimitedError): + raise exception + response = getattr(exception, "response", None) + if response is not None: + check_response(response, "request error") + status = _status(getattr(response, "status_code", 0)) + if status: + return "upstream HTTP %d" % status + return "upstream request or response error (%s)" % type(exception).__name__ + + # ============================================================================== # Part 1: Pure-python AES-128 # ============================================================================== @@ -247,12 +438,20 @@ def new_session(): def parse_course_url(url): - m = re.search( - r"/course/detail/(?P\d+)/(?P[^/]+)/" - r"(?P\d+)/signin/(?P\d+)/(?P\d+)", url) - if not m: + parts = urllib.parse.urlsplit(str(url or "")) + m = re.fullmatch( + r"/course/detail/(?P[0-9A-Za-z]+)/(?P[^/]+)/" + r"(?P\d+)/signin/(?P\d+)/(?P\d+)/?", + parts.path) + try: + port = parts.port + except ValueError: + port = -1 + if (parts.scheme != "https" or parts.username is not None or parts.password is not None + or port not in (None, 443) + or (parts.hostname or "").lower() != "ginkgostu.dufe.edu.cn" or not m): raise ValueError( - "URL is not a sign-in page (missing /signin//): %s" % url) + "URL is not a sign-in page (missing /signin//)") return m.groupdict() @@ -321,18 +520,46 @@ def exchange_token(sess, cfg, code): headers={"Content-Type": "application/x-www-form-urlencoded"}) try: j = r.json() - except ValueError as e: - raise UpstreamError("authLogin returned non-JSON: %s" % e) - if j.get("code") != "2000": - raise AuthError("OAuth code exchange refused: %s" - % json.dumps(j, ensure_ascii=False)) + except ValueError: + raise UpstreamError("authLogin returned non-JSON") from None + if not isinstance(j, dict) or j.get("code") != "2000": + raise AuthError("OAuth code exchange refused") return j["data"]["token"] +def login(username, password): + """Complete SSO login flow. Returns (sess, token). + + auto 模式一次登录后复用同一 session/token 逐个活动签到， + 对齐 bot ginkgo 复用轮询缓存登录态的思路（请求内级）。""" + sess = new_session() + cfg = get_unify_config(sess) + csrf, page_url = open_authorize(sess, cfg) + slider = exploit_slider(sess, page_url) + resp = submit_login(sess, page_url, csrf, username, password, slider, page_url) + + loc = resp.headers.get("Location", "") + m = re.search(r"[?&]code=([A-Za-z0-9_\-]+)", loc) + if not m: + raise AuthError( + "Login failed: no OAuth code in redirect (HTTP %s)" % resp.status_code) + token = exchange_token(sess, cfg, m.group(1)) + return sess, token + + # ============================================================================== # Part 4: Sign-in API # ============================================================================== +def _request(sess, method, *args, **kwargs): + """Also guard fake sessions and HTTPError.response before any JSON parsing.""" + try: + response = getattr(sess, method)(*args, **kwargs) + except requests.RequestException as exc: + raise UpstreamError(safe_error(exc)) from None + return check_response(response, "signin") + + def api_headers(token, fingerprint=None): return { "User-Agent": UA, @@ -345,8 +572,8 @@ def api_headers(token, fingerprint=None): def call_api(sess, path, data, token): - return sess.post(GINKGO + path, data=data, timeout=20, - headers=api_headers(token)) + return _request(sess, "post", GINKGO + path, data=data, timeout=20, + headers=api_headers(token)) def commit_sign_in(sess, token, sign_in_id, batch_no, referer, code): @@ -368,14 +595,14 @@ def commit_sign_in(sess, token, sign_in_id, batch_no, referer, code): headers["Referer"] = referer try: - r = sess.post( + r = _request(sess, "post", GINKGO + "/student/signIn/InTheSign", data={"signInId": str(sign_in_id), "code": str(code)}, timeout=20, headers=headers, ) - except requests.RequestException as e: - return False, "request to InTheSign failed: %s" % e, None + except (requests.RequestException, UpstreamError) as e: + return False, "request to InTheSign failed: %s" % safe_error(e), None if r.status_code != 200: return False, "InTheSign returned HTTP %d" % r.status_code, None @@ -383,76 +610,349 @@ def commit_sign_in(sess, token, sign_in_id, batch_no, referer, code): try: j = r.json() except ValueError: - return False, "InTheSign returned non-JSON: %s" % r.text[:200], None + return False, "InTheSign returned non-JSON", None + if not isinstance(j, dict): + return False, "Malformed InTheSign response", None api_code = str(j.get("code", "")) - server_msg = (j.get("message") or j.get("msg") or "").strip() - if api_code == "2000": - return True, server_msg or "accepted", j - - if server_msg: - reason = server_msg - elif api_code: - reason = "upstream returned code %s" % api_code - else: - reason = ("unrecognized upstream response: %s" - % json.dumps(j, ensure_ascii=False)[:200]) - return False, reason, j + return True, "accepted", j + # Never return/log arbitrary upstream messages or bodies: they may contain secrets. + return False, "InTheSign refused the sign-in request", j -def fetch_sign_in_info(username, password, course_url, do_commit=True): +def fetch_sign_in_info(username, password, course_url, do_commit=True, + sess=None, token=None): route = parse_course_url(course_url) sign_in_id = route["signInId"] batch_no = route.get("batchNo") - sess = new_session() - cfg = get_unify_config(sess) - csrf, page_url = open_authorize(sess, cfg) - slider = exploit_slider(sess, page_url) - resp = submit_login(sess, page_url, csrf, username, password, slider, page_url) - - loc = resp.headers.get("Location", "") - m = re.search(r"[?&]code=([A-Za-z0-9_\-]+)", loc) - if not m: - raise AuthError( - "Login failed: no OAuth code in redirect (HTTP %s)" % resp.status_code) - auth_code = m.group(1) - - token = exchange_token(sess, cfg, auth_code) + if sess is None or not token: + # 无外部登录态时自登录（保持向后兼容）：直接复用上方 SSO 流程 + sess, token = login(username, password) + # 否则：sess/token 由调用方提供（auto 模式请求内复用同一登录态），跳过 SSO 登录 r = call_api(sess, "/student/signIn/getSignInInfo", {"signInId": sign_in_id}, token) + check_response(r, "getSignInInfo") try: j = r.json() - except ValueError as e: - raise UpstreamError("getSignInInfo returned non-JSON: %s" % e) + except ValueError: + raise UpstreamError("getSignInInfo returned non-JSON") from None + if not isinstance(j, dict): + raise UpstreamError("Malformed getSignInInfo response") data = j.get("data") or {} + if not isinstance(data, dict): + raise UpstreamError("Malformed getSignInInfo data") model = data.get("signInModel") or {} + if not isinstance(model, dict): + raise UpstreamError("Malformed getSignInInfo model") code = model.get("code") + raw_type = model.get("type") + if raw_type is None or raw_type == "": + raw_type = route.get("type") + try: + signin_type = int(raw_type) + except (TypeError, ValueError): + raise UpstreamError("Malformed sign-in type") from None + type_name = signin_type_label(signin_type) - if code is None or code == "": - raise NoSignInCodeError( - "Sign-in activity %s has no code configured" % sign_in_id) + if signin_type == 1 and (code is None or code == ""): + raise NoSignInCodeError("数字码签到未配置签到码") + if signin_type == 2: + code = "" result = { "signInId": sign_in_id, "signInCode": code, - "type": model.get("type"), + "type": signin_type, + "typeName": type_name, "title": model.get("title"), "status": data.get("signStatus"), } if do_commit: - ok, reason, _raw = commit_sign_in( - sess, token, sign_in_id, batch_no, course_url, code - ) - result["commit"] = {"success": ok, "message": reason} + if signin_type not in (1, 2): + result["commit"] = { + "success": False, + "message": "%s暂不支持自动签到，需特定流程" % type_name, + } + else: + canonical_referer = GINKGO + urllib.parse.urlsplit(course_url).path + ok, reason, _raw = commit_sign_in( + sess, token, sign_in_id, batch_no, canonical_referer, code + ) + result["commit"] = {"success": ok, "message": reason} return result +# ============================================================================== +# Part 4b: Auto sign-in (mode=auto) +# 通知列表发现签到活动并逐个自动签到 —— 逻辑对齐 bot ginkgo 模块： +# fetch_all（通知+互动消息）→ _validated_signin_item（busi + 文本含"签到" + +# 可解析课程链接）→ 按 signInId 去重 → 每活动连续重试（首次+最多5次，间隔3s， +# token 失效重登录一次）→ 结果聚合为列表返回。 +# ============================================================================== + +_TAG_RE = re.compile(r"<[^>]+>") + + +def _html_to_text(raw): + if not raw: + return "" + text = re.sub(r"(?i)", "\n", raw) + text = re.sub(r"(?i)", "\n", text) + text = _TAG_RE.sub("", text) + text = _html.unescape(text) + text = re.sub(r"[ \t\u3000]+", " ", text) + text = re.sub(r"\n{2,}", "\n", text) + return text.strip() + + +def _parse_ts(value): + text = (value or "").strip() + if not text: + return 0.0 + if text.endswith("Z"): + text = text[:-1] + "+00:00" + m = re.search(r"([+-]\d{2})(\d{2})$", text) + if m: + text = text[:m.start()] + m.group(1) + ":" + m.group(2) + try: + dt = datetime.fromisoformat(text) + except ValueError: + try: + dt = datetime.strptime(text[:19], "%Y-%m-%d %H:%M:%S") + except ValueError: + return 0.0 + if dt.tzinfo is None: + dt = dt.replace(tzinfo=TZ8) + return dt.timestamp() + + +def _normalize_notice(raw): + title = (raw.get("title") or "").strip() or "(无标题)" + ts = _parse_ts(raw.get("createTime")) + return { + "kind": "notice", + "key": "N%s" % raw.get("id"), + "id": str(raw.get("id") or ""), + "title": title, + "text": _html_to_text(raw.get("content") or ""), + "ts": ts, + "link": NOTICE_PAGE, + "jump": "", + } + + +def _normalize_busi(raw): + text = _html_to_text(raw.get("content") or "") + jump = (raw.get("jumpUrl") or "").strip() + if jump.startswith("http"): + link = jump + elif jump.startswith("/"): + link = GINKGO + jump + else: + link = SYSTEM_PAGE + ts = _parse_ts(raw.get("createTime")) + return { + "kind": "busi", + "key": "B%s" % (raw.get("uuid") or ""), + "uuid": str(raw.get("uuid") or ""), + "id": "", + "title": text[:120] or "(无内容)", + "text": text, + "ts": ts, + "link": link, + "jump": jump, + } + + +def _api_headers(token): + return { + "User-Agent": UA, + "Origin": GINKGO, + "Referer": GINKGO + "/", + "Content-Type": "application/x-www-form-urlencoded", + "token": token, + "fingerprint": uuid.uuid4().hex, + } + + +def _call_json(sess, token, method, path, params=None, body=None, what="接口"): + """消息类 API 调用（对齐 bot _call）：check_response + code 必须为 2000。""" + headers = _api_headers(token) + if method == "GET": + r = sess.get(GINKGO + path, params=params, headers=headers, timeout=20) + else: + headers["Content-Type"] = "application/json" + r = sess.post(GINKGO + path, data=json.dumps(body if body is not None else {}), + headers=headers, timeout=20) + check_response(r, what) + try: + payload = r.json() + except ValueError: + raise UpstreamError("%s returned non-JSON (HTTP %s)" % (what, r.status_code)) + if not isinstance(payload, dict): + raise UpstreamError("%s JSON must be an object" % what) + if is_rate_limited(None, payload): + raise RateLimitedError("%s 触发频率限制" % what) + if str(payload.get("code")) != "2000": + raise UpstreamError("%s returned code=%s" % (what, payload.get("code"))) + return payload + + +def fetch_message_all(sess, token, page_size=PAGE_SIZE): + """取两类消息（均为全部，含已读），返回 (notices, busi) 归一化列表。""" + j1 = _call_json(sess, token, "GET", "/student/message/getMessageAll", + params={"pageSize": page_size, "pageNo": 1, "readFlag": 0}, + what="通知列表") + notices = [_normalize_notice(x) for x in ((j1.get("data") or {}).get("list") or [])] + j2 = _call_json(sess, token, "POST", "/student/busiMessage/list", body={}, + what="互动消息列表") + busi = [_normalize_busi(x) for x in (j2.get("data") or [])] + return notices, busi + + +def _validated_signin_item(item): + """Return an annotated copy only for a real interactive sign-in activity. + + 与 bot ginkgo 的 _validated_signin_item 同逻辑：仅互动消息、文本含"签到"、 + 跳转链接可解析为课程签到路由。query 参数可能携带 token，不入库不回显。""" + if (not isinstance(item, dict) or item.get("kind") != "busi" + or "签到" not in str(item.get("text") or "")): + return None + link = str(item.get("link") or "").strip() + try: + route = parse_course_url(link) + except (ValueError, TypeError, AttributeError): + return None + row = dict(item) + # Persist/display only the canonical path: query parameters may carry tokens + # and must never reach receipts or push logs. + row["link"] = GINKGO + urllib.parse.urlsplit(link).path + # signInId is the stable activity identity（同一活动的多条消息去重） + row["signin_activity_key"] = "S%s" % route["signInId"] + try: + row["signin_type"] = int(route["type"]) + except (TypeError, ValueError, KeyError): + return None + row["signin_type_name"] = signin_type_label(row["signin_type"]) + return row + + +def _signin_one_with_retry(username, password, row, sess, token): + """对单个活动执行签到：失败连续重试（首次 + 最多 5 次，间隔 3s）， + token 疑似失效(上游错误/登录错)时重新登录一次后立即重试（对齐 bot refresh 语义）。 + + 与 bot complete_signin 一致：除限流（外层中断）外，任何未确认成功都消耗重试次数； + 重试用尽记最终失败。返回 (entry, sess, token)。""" + link = row["link"] + base = { + "activityKey": row.get("signin_activity_key"), + "signInId": parse_course_url(link).get("signInId"), + "routeType": row.get("signin_type"), + "routeTypeName": row.get("signin_type_name"), + "link": link, + } + attempts = 0 + refreshed = False + last_err = "unknown error" + last_data = None + while attempts < MAX_SIGNIN_ATTEMPTS: + attempts += 1 + try: + data = fetch_sign_in_info(username, password, link, do_commit=True, + sess=sess, token=token) + last_data = data + commit = data.get("commit") + if isinstance(commit, dict) and commit.get("success") is True: + entry = dict(base) + entry.update({ + "type": data.get("type"), + "typeName": data.get("typeName"), + "title": data.get("title"), + "signInCode": data.get("signInCode"), + "status": data.get("status"), + "success": True, + "message": "签到成功", + "attempts": attempts, + }) + return entry, sess, token + message = ((commit or {}).get("message") + or "no commit result returned") + last_err = str(message) + except RateLimitedError: + raise + except (UpstreamError, AuthError) as e: + # 疑似 token 失效：重登录一次后立即重试（不消耗退避间隔，对齐 bot） + if not refreshed: + refreshed = True + try: + sess, token = login(username, password) + continue + except Exception: + pass # 重登录也失败：落入常规退避重试 + last_err = str(e) or type(e).__name__ + except NoSignInCodeError as e: + last_err = str(e) + except Exception as e: + last_err = safe_error(e) + if attempts < MAX_SIGNIN_ATTEMPTS: + time.sleep(SIGNIN_BACKOFF[min(attempts - 1, len(SIGNIN_BACKOFF) - 1)]) + entry = dict(base) + if isinstance(last_data, dict): + entry.update({ + "type": last_data.get("type"), + "typeName": last_data.get("typeName"), + "title": last_data.get("title"), + "signInCode": last_data.get("signInCode"), + "status": last_data.get("status"), + }) + entry.update({"success": False, + "message": "签到失败（重试 %d 次）: %s" % (attempts, last_err), + "attempts": attempts}) + return entry, sess, token + + +def auto_sign_in_all(username, password): + """mode=auto 主流程。返回聚合结果 dict（results 为列表）。""" + sess, token = login(username, password) + notices, busi = fetch_message_all(sess, token) + + activities = {} + for item in notices + busi: + row = _validated_signin_item(item) + if row: + activities.setdefault(row["signin_activity_key"], row) + + results = [] + rate_limited = False + for key in sorted(activities, key=lambda k: activities[k].get("ts") or 0): + row = activities[key] + try: + entry, sess, token = _signin_one_with_retry( + username, password, row, sess, token) + except RateLimitedError: + # 对齐 bot：限流即中断本轮，已完成的结果照常返回 + rate_limited = True + break + results.append(entry) + + succeeded = sum(1 for r in results if r.get("success")) + return { + "mode": "auto", + "activities": len(activities), + "total": len(results), + "succeeded": succeeded, + "failed": len(results) - succeeded, + "rate_limited": rate_limited, + "results": results, + } + + # ============================================================================== # Part 5: HTTP service (WSGI app) # ============================================================================== @@ -496,17 +996,24 @@ def ginkgostu_sign_in(): url_b64 = args.get("url") or "" commit_arg = (args.get("commit") or "true").strip().lower() do_commit = commit_arg not in ("0", "false", "no") + mode = (args.get("mode") or "").strip().lower() - if not (username and password_b64 and url_b64): + if not (username and password_b64): + return _error(400, 400, + "Missing parameters: username, password(base64)") + if mode != "auto" and not url_b64: return _error(400, 400, "Missing parameters: username, password(base64), url(base64)") try: password = _b64_decode(password_b64, "password") - course_url = _b64_decode(url_b64, "url") + course_url = _b64_decode(url_b64, "url") if url_b64 else "" except ValueError as e: return _error(400, 400, str(e)) + if mode == "auto": + return _auto_response(username, password) + log.info("query start user=%s commit=%s", username[:2] + "***", do_commit) try: data = fetch_sign_in_info(username, password, course_url, do_commit=do_commit) @@ -524,28 +1031,61 @@ def ginkgostu_sign_in(): "data": data, }) - log.info("query ok user=%s code=%s", username[:2] + "***", data.get("signInCode")) + log.info("query ok user=%s type=%s", username[:2] + "***", data.get("type")) return jsonify({"code": 200, "msg": "ok", "data": data}) - except AuthError as e: - log.warning("auth failed: %s", e) - return _error(401, 401, str(e)) - except NoSignInCodeError as e: - log.info("no code: %s", e) - return _error(404, 404, str(e)) - except UpstreamError as e: - log.error("upstream error: %s", e) - return _error(502, 502, str(e)) - except ValueError as e: - log.warning("bad request: %s", e) - return _error(400, 400, str(e)) - except requests.RequestException as e: - log.error("upstream request failed: %s", e) + except AuthError: + log.warning("authentication failed user=%s", username[:2] + "***") + return _error(401, 401, "Authentication failed") + except NoSignInCodeError: + log.info("numeric code missing user=%s", username[:2] + "***") + return _error(404, 404, "数字码签到未配置签到码") + except UpstreamError: + log.error("upstream response error user=%s", username[:2] + "***") + return _error(502, 502, "Upstream response error") + except ValueError: + log.warning("bad request user=%s", username[:2] + "***") + return _error(400, 400, "Invalid request") + except requests.RequestException: + log.error("upstream request failed user=%s", username[:2] + "***") return _error(502, 502, "Upstream request failed") - except Exception: - log.exception("internal error") + except Exception as exc: + log.error("internal error type=%s", type(exc).__name__) return _error(500, 500, "Internal error") +def _auto_response(username, password): + """mode=auto：发现 + 自动签到 + 聚合列表返回。""" + log.info("auto sign-in start user=%s", username[:2] + "***") + try: + data = auto_sign_in_all(username, password) + except AuthError: + log.warning("auto sign-in authentication failed user=%s", username[:2] + "***") + return _error(401, 401, "Authentication failed") + except RateLimitedError: + log.warning("auto sign-in rate limited user=%s", username[:2] + "***") + return _error(429, 429, "Upstream rate limited, please retry later") + except UpstreamError: + log.error("auto sign-in upstream error user=%s", username[:2] + "***") + return _error(502, 502, "Upstream response error") + except requests.RequestException: + log.error("auto sign-in upstream request failed user=%s", username[:2] + "***") + return _error(502, 502, "Upstream request failed") + except Exception as exc: + log.error("auto sign-in internal error type=%s", type(exc).__name__) + return _error(500, 500, "Internal error") + log.info("auto sign-in done user=%s total=%s succeeded=%s rate_limited=%s", + username[:2] + "***", data["total"], data["succeeded"], + data["rate_limited"]) + if data["rate_limited"]: + msg = ("partial: upstream rate limited, %d/%d activities processed" + % (data["total"], data["activities"])) + elif data["failed"]: + msg = "ok with failures: %d/%d succeeded" % (data["succeeded"], data["total"]) + else: + msg = "ok" + return jsonify({"code": 200, "msg": msg, "data": data}) + + @app.route("/healthz", methods=["GET"]) def healthz(): return jsonify({"code": 200, "msg": "ok", "data": None})