feat(dufe): 新增 mode=auto 自动签到,非签到码逻辑以 bot 为基准同步

- mode=auto: 通知列表(getMessageAll+busiMessage/list)发现签到活动
  (bot ginkgo 同款过滤, 按 signInId 去重) → 逐个自动签到
- 连续重试: 首次+最多5次间隔3s (对齐 bot MAX_SIGNIN_ATTEMPTS/SIGNIN_BACKOFF),
  token 失效重登录一次立即重试; 限流中断本轮; 结果聚合 data.results 列表
- 基准同步 (bot ginkgo): 移植限流防护体系(check_response/is_rate_limited/
  safe_error/_LoginErrors/_request), parse_course_url courseId 放宽 [0-9A-Za-z]+,
  commit_sign_in 错误包装, fetch_sign_in_info 增加 check_response 与
  sess/token 注入 (auto 模式请求内复用登录态)
- 验证: 免凭据单测 + 假凭据 401 线路测试 + bj 真账密只读联调
  (notices=20/busi=18/活动12 类型解析正确)
This commit is contained in:
RRRosmontis
2026-09-23 11:53:12 +08:00
parent 9ecbbbc6d8
commit 9b9c501fa9
3 changed files with 632 additions and 77 deletions
+5 -2
View File
@@ -23,7 +23,7 @@
| openapi | `127.0.0.1:9008` | `source/main.py`(aiohttp) | 聚合层,纯转发,**唯一入口** | | openapi | `127.0.0.1:9008` | `source/main.py`(aiohttp) | 聚合层,纯转发,**唯一入口** |
| pixiv | `127.0.0.1:9007` | `source/pixiv/pixiv.py`(Flask + waitress) | 图片代理 | | pixiv | `127.0.0.1:9007` | `source/pixiv/pixiv.py`(Flask + waitress) | 图片代理 |
| bilibili | `127.0.0.1:9006` | `source/bilibili/bilibili.py`(aiohttp + yt-dlp) | 视频代理 | | bilibili | `127.0.0.1:9006` | `source/bilibili/bilibili.py`(aiohttp + yt-dlp) | 视频代理 |
| dufe 签到 | `127.0.0.1:9005` | `source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py`(Flask + waitress 16 线程) | 银杏(DUFE)签到码查询,`GET\|POST /dufe/ginkgostu_sign_in` | | dufe 签到 | `127.0.0.1:9005` | `source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py`(Flask + waitress 16 线程) | 银杏(DUFE)签到码查询,`GET\|POST /dufe/ginkgostu_sign_in`;`&mode=auto` 时拉取通知列表自动签到(与 bot ginkgo 同逻辑,连续重试,结果列表聚合返回) |
| jm | `127.0.0.1:9004` | `source/jm/jm.py`(aiohttp + jmcomic) | 漫画下载打包(移植自 qqbot jm 模块),`GET /jm/get_comic?id=xxx[&range=起-止]` | | jm | `127.0.0.1:9004` | `source/jm/jm.py`(aiohttp + jmcomic) | 漫画下载打包(移植自 qqbot jm 模块),`GET /jm/get_comic?id=xxx[&range=起-止]` |
| pixiv-token-refresh | — | `source/pixiv/refresh_token.py` | token 刷新守护(每 30–60 分钟随机) | | pixiv-token-refresh | — | `source/pixiv/refresh_token.py` | token 刷新守护(每 30–60 分钟随机) |
| bilibili-refresh | — | `source/bilibili/refresh_cookie.py` | cookie 刷新守护(每 50–70 分钟随机) | | bilibili-refresh | — | `source/bilibili/refresh_cookie.py` | cookie 刷新守护(每 50–70 分钟随机) |
@@ -138,7 +138,10 @@ for p in 9004 9005 9006 9007 9008; do timeout 3 bash -c "cat < /dev/null > /dev/
## 版本要点 ## 版本要点
- **2026-09-23**:新增 **jm 漫画下载打包服务**(`source/jm/jm.py`,aiohttp + jmcomic,`127.0.0.1:9004`,端点 `GET /jm/get_comic?id=xxx[&range=起-止]`),移植自 qqbot 的 jm 模块(去掉 QQ/代理轮换/上传 cf 逻辑,改为 HTTP 直返 ZIP;hk 可直连源站无需代理)。ZIP 为临时产物:返回后 600 秒延时删除,TTL 窗口内同名请求直接复用。聚合层 `main.py` 注册路由(新增 `JM_BACKEND`,复用既有 `proxy()`),nginx 无需改动(`location /` 兜底) - **2026-09-23**
- dufe 签到服务新增 **`mode=auto`**:拉取通知列表(getMessageAll + busiMessage/list)→ 按 bot ginkgo 同款规则发现签到活动(互动消息 + 文本含"签到" + 可解析课程链接,按 signInId 去重)→ 逐个自动签到;**连续重试**(首次 + 最多 5 次,间隔 3s,对齐 bot `MAX_SIGNIN_ATTEMPTS=6 / SIGNIN_BACKOFF`;token 疑似失效时重登录一次立即重试,对齐 bot refresh 语义);**所有结果聚合为 `data.results` 列表**返回(每活动含 success/message/attempts/title/signInCode 等)
- 非签到码签到逻辑与 bot ginkgo 比对后以 bot 为基准同步:移植限流防护体系(`check_response` / `is_rate_limited` / `safe_error` / `_LoginErrors`)、`parse_course_url` 的 courseId 放宽至 `[0-9A-Za-z]+`、`commit_sign_in` 的错误包装、`fetch_sign_in_info` 的 check_response 与外部登录态注入(`sess`/`token` 参数,auto 模式请求内复用同一登录态)
- **2026-09-23**(早些):新增 **jm 漫画下载打包服务**(`source/jm/jm.py`,aiohttp + jmcomic,`127.0.0.1:9004`,端点 `GET /jm/get_comic?id=xxx[&range=起-止]`),移植自 qqbot 的 jm 模块(去掉 QQ/代理轮换/上传 cf 逻辑,改为 HTTP 直返 ZIP;hk 可直连源站无需代理)。ZIP 为临时产物:返回后 600 秒延时删除,TTL 窗口内同名请求直接复用。聚合层 `main.py` 注册路由(新增 `JM_BACKEND`,复用既有 `proxy()`),nginx 无需改动(`location /` 兜底)
- **2026-09-13**:新增 **dufe 银杏签到码查询服务**(`source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py`,Flask + waitress 16 线程,`127.0.0.1:9005`,端点 `GET|POST /dufe/ginkgostu_sign_in`),并在聚合层 `main.py` 注册该路由——为此新增 `proxy_forward()`(保留请求方法与请求体,原 `proxy()` 保持固定 GET 不变,零改动既有行为)。同期修复该服务两处缺陷:base64 参数补 `urllib.parse.unquote`(兼容 iOS 快捷指令的 `%3D` 编码),`commit` 判定改为 fail-closed(须 `commit.success is True` 才算签到成功) - **2026-09-13**:新增 **dufe 银杏签到码查询服务**(`source/dufe/ginkgostu_sign_in/ginkgo_sign_in.py`,Flask + waitress 16 线程,`127.0.0.1:9005`,端点 `GET|POST /dufe/ginkgostu_sign_in`),并在聚合层 `main.py` 注册该路由——为此新增 `proxy_forward()`(保留请求方法与请求体,原 `proxy()` 保持固定 GET 不变,零改动既有行为)。同期修复该服务两处缺陷:base64 参数补 `urllib.parse.unquote`(兼容 iOS 快捷指令的 `%3D` 编码),`commit` 判定改为 fail-closed(须 `commit.success is True` 才算签到成功)
- **2026-09-12** - **2026-09-12**
- 端口重排 `9888/9088/19990 → 9008/9007/9006`,全部仅绑 `127.0.0.1`;pixiv 由 `0.0.0.0:9088` 收口 - 端口重排 `9888/9088/19990 → 9008/9007/9006`,全部仅绑 `127.0.0.1`;pixiv 由 `0.0.0.0:9088` 收口
+14 -2
View File
@@ -11,7 +11,7 @@
|---|---| |---|---|
| `/pixiv/get_image?id=xxx` | `127.0.0.1:9008`(main.py 聚合层)→ `127.0.0.1:9007`(pixiv) | | `/pixiv/get_image?id=xxx` | `127.0.0.1:9008`(main.py 聚合层)→ `127.0.0.1:9007`(pixiv) |
| `/bilibili/get_video?bv=xxx` | `127.0.0.1:9008`(main.py 聚合层)→ `127.0.0.1:9006`(bilibili) | | `/bilibili/get_video?bv=xxx` | `127.0.0.1:9008`(main.py 聚合层)→ `127.0.0.1:9006`(bilibili) |
| `/dufe/ginkgostu_sign_in` | `127.0.0.1:9008`(main.py 聚合层)→ `127.0.0.1:9005`(dufe 签到码查询,支持 GET\|POST) | | `/dufe/ginkgostu_sign_in` | `127.0.0.1:9008`(main.py 聚合层)→ `127.0.0.1:9005`(dufe 签到码查询,支持 GET\|POST;`&mode=auto` 自动签到) |
| `/jm/get_comic?id=xxx[&range=起-止]` | `127.0.0.1:9008`(main.py 聚合层)→ `127.0.0.1:9004`(jm 漫画下载打包) | | `/jm/get_comic?id=xxx[&range=起-止]` | `127.0.0.1:9008`(main.py 聚合层)→ `127.0.0.1:9004`(jm 漫画下载打包) |
| `/img-master/`、`/img-original/` | nginx 直接反代 `i.pximg.net`(无本地后端) | | `/img-master/`、`/img-original/` | nginx 直接反代 `i.pximg.net`(无本地后端) |
| `/` | `127.0.0.1:9008`(302 跳转文章页) | | `/` | `127.0.0.1:9008`(302 跳转文章页) |
@@ -134,7 +134,19 @@
## 变更记录 ## 变更记录
**2026-09-23** **2026-09-23(dufe mode=auto + bot 基准同步)**
- dufe 签到服务新增 **`&mode=auto`**(url 参数可省略):SSO 登录一次后复用登录态 → 拉取通知列表(`GET /student/message/getMessageAll` pageSize=20/readFlag=0 + `POST /student/busiMessage/list`)→ 按 bot ginkgo 同款规则发现活动(仅互动消息、文本含"签到"、jumpUrl 可解析为 `/course/detail/.../signin/<id>/<type>`,query 不入库,按 signInId 去重)→ 按时间序逐个自动签到
- **连续重试**:每活动首次 + 最多 5 次重试、间隔 3 秒(`MAX_SIGNIN_ATTEMPTS=6` / `SIGNIN_BACKOFF=(3,3,3,3,3)`,对齐 bot);上游错误/登录错时重登录一次立即重试(对齐 bot `_call_sign_in_local` 的 refresh 语义);**除限流外任何未确认成功都消耗重试次数**(对齐 bot `complete_signin`);限流即中断本轮,已完成结果照常返回(`data.rate_limited=true`)
- **结果聚合**:`data.results` 列表逐活动返回(activityKey/signInId/type/typeName/title/signInCode/status/success/message/attempts),另附 total/succeeded/failed/activities 计数
- **与 bot ginkgo 非签到码签到逻辑比对后以 bot 为基准同步**:
- 移植限流防护体系:`RateLimitedError` / `_RATE_MESSAGE` / `_RATE_CODES` / `is_rate_limited` / `_valid_retry_after` / `_LoginErrors` / `check_response` / `safe_error` / `_request`(原实现完全无限流检测)
- `parse_course_url`:courseId 由 `\d+` 放宽为 `[0-9A-Za-z]+`,报错文案对齐
- `commit_sign_in`:经 `_request` 包装(`check_response`),失败消息 "request to InTheSign failed: %s" % safe_error(e)(原为 "upstream request failed")
- `fetch_sign_in_info`:getSignInInfo 响应过 `check_response`;新增 `sess`/`token` 参数注入(auto 模式请求内复用登录态,省去每活动一次 SSO);补齐 "Malformed getSignInInfo response" 非 dict 校验
- 验证:免凭据单测(发现/过滤正反用例)+ 假凭据 401 线路测试 + bj 真账密**只读**联调(登录 OK,notices=20/busi=18,发现 12 个活动含 1 个普通签到,类型/标题解析正确);未做真实提交(避免与运行中的 bot 轮询交叉签到)
**2026-09-23(早些)**
- 新增 **jm 漫画下载打包服务**:`source/jm/jm.py` + `jm.service`(`127.0.0.1:9004`,aiohttp + jmcomic,端点 `GET /jm/get_comic?id=xxx[&range=起-止]`) - 新增 **jm 漫画下载打包服务**:`source/jm/jm.py` + `jm.service`(`127.0.0.1:9004`,aiohttp + jmcomic,端点 `GET /jm/get_comic?id=xxx[&range=起-止]`)
- 移植自 qqbot 的 jm 模块(bj `C:/call/bot/utility/jm.py`):保留章节范围解析 / 文件名清洗 / webp→jpg 打包逻辑;去掉 QQ 回复、代理轮换(hk 可直连源站)、上传 cf 文件服务器三部分,改为 HTTP 直返 ZIP - 移植自 qqbot 的 jm 模块(bj `C:/call/bot/utility/jm.py`):保留章节范围解析 / 文件名清洗 / webp→jpg 打包逻辑;去掉 QQ 回复、代理轮换(hk 可直连源站)、上传 cf 文件服务器三部分,改为 HTTP 直返 ZIP
+613 -73
View File
@@ -6,17 +6,27 @@ Ginkgo (DUFE) sign-in code query service.
Endpoint: GET|POST /dufe/ginkgostu_sign_in Endpoint: GET|POST /dufe/ginkgostu_sign_in
params: username, password(base64), url(base64) params: username, password(base64), url(base64)
[commit=true] optional, default true: also POST the code back [commit=true] optional, default true: also POST the code back
[mode=auto] optional: fetch the notification list, discover
sign-in activities and auto sign-in each one
(bot ginkgo parity, consecutive retries);
results returned as an aggregated list.
mode=auto 时 url 参数可省略。
Listen: 127.0.0.1:9005 (WSGI: waitress) Listen: 127.0.0.1:9005 (WSGI: waitress)
""" """
import base64 import base64
import binascii import binascii
import html as _html
import json import json
import logging import logging
import re import re
import sys import sys
import time
import urllib.parse import urllib.parse
import uuid import uuid
from datetime import datetime, timedelta, timezone
from email.utils import parsedate_to_datetime
from html.parser import HTMLParser
try: try:
import requests import requests
@@ -43,6 +53,17 @@ GINKGO = "https://ginkgostu.dufe.edu.cn"
SSO = "https://sso.dufe.edu.cn" SSO = "https://sso.dufe.edu.cn"
CUSTOM_CODE = "kb_dconline" CUSTOM_CODE = "kb_dconline"
NOTICE_PAGE = GINKGO + "/notice"
SYSTEM_PAGE = GINKGO + "/notice/system"
PAGE_SIZE = 20 # 单类单次抓取条数(与 bot ginkgo 一致)
# 自动签到重试参数(与 bot ginkgo 的 MAX_SIGNIN_ATTEMPTS / SIGNIN_BACKOFF 一致:
# 首次 + 最多 5 次重试,每次间隔 3 秒)
MAX_SIGNIN_ATTEMPTS = 6
SIGNIN_BACKOFF = (3, 3, 3, 3, 3)
TZ8 = timezone(timedelta(hours=8))
UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36") "(KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36")
@@ -65,13 +86,183 @@ class AuthError(Exception):
class NoSignInCodeError(Exception): class NoSignInCodeError(Exception):
"""The sign-in activity exists but has no code configured.""" """The numeric-code activity exists but has no code configured."""
class UnsupportedSigninType(Exception):
"""The activity requires a flow this service cannot safely automate."""
SIGNIN_TYPE_NAMES = {
0: "二维码签到",
1: "数字码签到",
2: "普通签到",
3: "直播签到",
}
def signin_type_label(value):
try:
signin_type = int(value)
except (TypeError, ValueError):
return "未知签到类型"
return SIGNIN_TYPE_NAMES.get(signin_type, "未知签到类型(%s)" % signin_type)
class UpstreamError(Exception): class UpstreamError(Exception):
"""Remote service returned an unexpected or erroneous response.""" """Remote service returned an unexpected or erroneous response."""
class RateLimitedError(Exception):
"""An upstream explicitly refused requests due to throttling."""
# ------------------------------------------------------------------------------
# Rate-limit detection & credential-safe errors
# (synced from bot ginkgostu.py 2026-09-23; bot is the baseline)
# ------------------------------------------------------------------------------
_RATE_MESSAGE = re.compile(
r"too\s+many\s+requests|rate[\s_-]*limit(?:ed|ing|\s+exceeded)?|"
r"request(?:s)?\s+throttl|throttl(?:ed|ing)|"
r"(?:请求|访问|操作|登录|登陆|接口|提交).{0,12}(?:过于频繁|太频繁|频繁|频率.{0,6}(?:超|限制))|"
r"(?:限流|请求次数.{0,8}(?:超限|超出|超过)|访问次数.{0,8}(?:超限|超出|超过))",
re.I,
)
_RATE_CODES = {"429", "4290", "40029", "90029", "RATE_LIMIT", "RATE_LIMITED", "RATE_LIMIT_EXCEEDED",
"TOO_MANY_REQUESTS", "REQUEST_TOO_FREQUENT", "THROTTLED",
"THROTTLING", "FREQUENT_REQUESTS"}
_FIELDS = ("code", "msg", "message", "error")
def _status(value):
try:
return int(value)
except (ValueError, TypeError):
return 0
def _explicit(value, depth=0):
if depth > 3:
return False
if isinstance(value, dict):
for key in _FIELDS:
item = value.get(key)
if key == "code" and isinstance(item, (str, int)):
if str(item).strip().upper().replace("-", "_") in _RATE_CODES:
return True
elif isinstance(item, dict):
if _explicit(item, depth + 1):
return True
elif isinstance(item, str) and _RATE_MESSAGE.search(item[:1024]):
return True
return False
class _LoginErrors(HTMLParser):
"""Extract only explicitly marked login error elements; ignore scripts/notices."""
def __init__(self):
super().__init__()
self.stack = []
self.messages = []
def handle_starttag(self, tag, attrs):
attrs = dict(attrs)
marker = (attrs.get("id", "") + " " + attrs.get("class", "")).lower()
explicit = bool(re.search(r"(?:^|[\s_-])(?:error|errors|errormsg|errormessage)(?:$|[\s_-])", marker))
ignored = tag in ("script", "style", "template") or bool(self.stack and self.stack[-1][2])
active = explicit or bool(self.stack and self.stack[-1][1])
if tag not in ("input", "br", "img", "meta", "link", "hr", "wbr", "source"):
self.stack.append((tag, active, ignored))
def handle_endtag(self, tag):
for index in range(len(self.stack) - 1, -1, -1):
if self.stack[index][0] == tag:
del self.stack[index:]
break
def handle_data(self, data):
if self.stack and self.stack[-1][1] and not self.stack[-1][2]:
self.messages.append(data)
def is_rate_limited(status, payload=None, text=""):
"""Check HTTP status and explicit error fields, never arbitrary payload data/HTML."""
status = _status(status)
if status == 429 or _explicit(payload):
return True
# Plain response bodies are evidence only on HTTP errors, and must be short.
if status >= 400 and isinstance(text, str):
plain = text.strip()
if len(plain) <= 1024 and not re.search(r"<[^>]+>", plain):
return bool(_RATE_MESSAGE.search(plain))
return False
def _valid_retry_after(value):
if not isinstance(value, str) or not value.strip():
return False
value = value.strip()
if re.fullmatch(r"[0-9]+", value):
return True
try:
return parsedate_to_datetime(value).tzinfo is not None
except (ValueError, TypeError, OverflowError):
return False
def check_response(response, what="upstream"):
"""Return response unchanged, or raise RateLimitedError before caller parsing.
`what` is deliberately not interpolated: URLs and upstream bodies may contain
credentials. Safe to use as a requests response hook through a small lambda.
"""
if response is None:
return response
status = _status(getattr(response, "status_code", 0))
message = "上游请求过于频繁,请稍后重试"
if status == 429:
raise RateLimitedError(message)
headers = getattr(response, "headers", {}) or {}
retry_after = next((v for k, v in headers.items() if str(k).lower() == "retry-after"), None)
if status >= 400 and _valid_retry_after(retry_after):
raise RateLimitedError(message)
try:
payload = response.json()
except (ValueError, TypeError, AttributeError):
payload = None
text = getattr(response, "text", "") or ""
if is_rate_limited(status, payload, text):
raise RateLimitedError(message)
# Login pages can return HTTP 200 with an explicit error element. Never scan
# the whole successful HTML document, inline scripts, or announcement text.
if payload is None and isinstance(text, str) and "<" in text:
parser = _LoginErrors()
try:
parser.feed(text[:262144])
except (ValueError, TypeError):
pass
if _RATE_MESSAGE.search(" ".join(parser.messages)[:1024]):
raise RateLimitedError(message)
return response
def safe_error(exception):
"""Propagate explicit throttling, otherwise return a credential-free diagnostic.
Never stringify arbitrary exceptions: requests URLs, query strings, tokens,
passwords and JSON decode snippets can all include secrets.
"""
if isinstance(exception, RateLimitedError):
raise exception
response = getattr(exception, "response", None)
if response is not None:
check_response(response, "request error")
status = _status(getattr(response, "status_code", 0))
if status:
return "upstream HTTP %d" % status
return "upstream request or response error (%s)" % type(exception).__name__
# ============================================================================== # ==============================================================================
# Part 1: Pure-python AES-128 # Part 1: Pure-python AES-128
# ============================================================================== # ==============================================================================
@@ -247,12 +438,20 @@ def new_session():
def parse_course_url(url): def parse_course_url(url):
m = re.search( parts = urllib.parse.urlsplit(str(url or ""))
r"/course/detail/(?P<courseId>\d+)/(?P<batchNo>[^/]+)/" m = re.fullmatch(
r"(?P<weekTimeId>\d+)/signin/(?P<signInId>\d+)/(?P<type>\d+)", url) r"/course/detail/(?P<courseId>[0-9A-Za-z]+)/(?P<batchNo>[^/]+)/"
if not m: r"(?P<weekTimeId>\d+)/signin/(?P<signInId>\d+)/(?P<type>\d+)/?",
parts.path)
try:
port = parts.port
except ValueError:
port = -1
if (parts.scheme != "https" or parts.username is not None or parts.password is not None
or port not in (None, 443)
or (parts.hostname or "").lower() != "ginkgostu.dufe.edu.cn" or not m):
raise ValueError( raise ValueError(
"URL is not a sign-in page (missing /signin/<signInId>/<type>): %s" % url) "URL is not a sign-in page (missing /signin/<signInId>/<type>)")
return m.groupdict() return m.groupdict()
@@ -321,18 +520,46 @@ def exchange_token(sess, cfg, code):
headers={"Content-Type": "application/x-www-form-urlencoded"}) headers={"Content-Type": "application/x-www-form-urlencoded"})
try: try:
j = r.json() j = r.json()
except ValueError as e: except ValueError:
raise UpstreamError("authLogin returned non-JSON: %s" % e) raise UpstreamError("authLogin returned non-JSON") from None
if j.get("code") != "2000": if not isinstance(j, dict) or j.get("code") != "2000":
raise AuthError("OAuth code exchange refused: %s" raise AuthError("OAuth code exchange refused")
% json.dumps(j, ensure_ascii=False))
return j["data"]["token"] return j["data"]["token"]
def login(username, password):
"""Complete SSO login flow. Returns (sess, token).
auto 模式一次登录后复用同一 session/token 逐个活动签到,
对齐 bot ginkgo 复用轮询缓存登录态的思路(请求内级)。"""
sess = new_session()
cfg = get_unify_config(sess)
csrf, page_url = open_authorize(sess, cfg)
slider = exploit_slider(sess, page_url)
resp = submit_login(sess, page_url, csrf, username, password, slider, page_url)
loc = resp.headers.get("Location", "")
m = re.search(r"[?&]code=([A-Za-z0-9_\-]+)", loc)
if not m:
raise AuthError(
"Login failed: no OAuth code in redirect (HTTP %s)" % resp.status_code)
token = exchange_token(sess, cfg, m.group(1))
return sess, token
# ============================================================================== # ==============================================================================
# Part 4: Sign-in API # Part 4: Sign-in API
# ============================================================================== # ==============================================================================
def _request(sess, method, *args, **kwargs):
"""Also guard fake sessions and HTTPError.response before any JSON parsing."""
try:
response = getattr(sess, method)(*args, **kwargs)
except requests.RequestException as exc:
raise UpstreamError(safe_error(exc)) from None
return check_response(response, "signin")
def api_headers(token, fingerprint=None): def api_headers(token, fingerprint=None):
return { return {
"User-Agent": UA, "User-Agent": UA,
@@ -345,8 +572,8 @@ def api_headers(token, fingerprint=None):
def call_api(sess, path, data, token): def call_api(sess, path, data, token):
return sess.post(GINKGO + path, data=data, timeout=20, return _request(sess, "post", GINKGO + path, data=data, timeout=20,
headers=api_headers(token)) headers=api_headers(token))
def commit_sign_in(sess, token, sign_in_id, batch_no, referer, code): def commit_sign_in(sess, token, sign_in_id, batch_no, referer, code):
@@ -368,14 +595,14 @@ def commit_sign_in(sess, token, sign_in_id, batch_no, referer, code):
headers["Referer"] = referer headers["Referer"] = referer
try: try:
r = sess.post( r = _request(sess, "post",
GINKGO + "/student/signIn/InTheSign", GINKGO + "/student/signIn/InTheSign",
data={"signInId": str(sign_in_id), "code": str(code)}, data={"signInId": str(sign_in_id), "code": str(code)},
timeout=20, timeout=20,
headers=headers, headers=headers,
) )
except requests.RequestException as e: except (requests.RequestException, UpstreamError) as e:
return False, "request to InTheSign failed: %s" % e, None return False, "request to InTheSign failed: %s" % safe_error(e), None
if r.status_code != 200: if r.status_code != 200:
return False, "InTheSign returned HTTP %d" % r.status_code, None return False, "InTheSign returned HTTP %d" % r.status_code, None
@@ -383,76 +610,349 @@ def commit_sign_in(sess, token, sign_in_id, batch_no, referer, code):
try: try:
j = r.json() j = r.json()
except ValueError: except ValueError:
return False, "InTheSign returned non-JSON: %s" % r.text[:200], None return False, "InTheSign returned non-JSON", None
if not isinstance(j, dict):
return False, "Malformed InTheSign response", None
api_code = str(j.get("code", "")) api_code = str(j.get("code", ""))
server_msg = (j.get("message") or j.get("msg") or "").strip()
if api_code == "2000": if api_code == "2000":
return True, server_msg or "accepted", j return True, "accepted", j
# Never return/log arbitrary upstream messages or bodies: they may contain secrets.
if server_msg: return False, "InTheSign refused the sign-in request", j
reason = server_msg
elif api_code:
reason = "upstream returned code %s" % api_code
else:
reason = ("unrecognized upstream response: %s"
% json.dumps(j, ensure_ascii=False)[:200])
return False, reason, j
def fetch_sign_in_info(username, password, course_url, do_commit=True): def fetch_sign_in_info(username, password, course_url, do_commit=True,
sess=None, token=None):
route = parse_course_url(course_url) route = parse_course_url(course_url)
sign_in_id = route["signInId"] sign_in_id = route["signInId"]
batch_no = route.get("batchNo") batch_no = route.get("batchNo")
sess = new_session() if sess is None or not token:
cfg = get_unify_config(sess) # 无外部登录态时自登录(保持向后兼容):直接复用上方 SSO 流程
csrf, page_url = open_authorize(sess, cfg) sess, token = login(username, password)
slider = exploit_slider(sess, page_url) # 否则:sess/token 由调用方提供(auto 模式请求内复用同一登录态),跳过 SSO 登录
resp = submit_login(sess, page_url, csrf, username, password, slider, page_url)
loc = resp.headers.get("Location", "")
m = re.search(r"[?&]code=([A-Za-z0-9_\-]+)", loc)
if not m:
raise AuthError(
"Login failed: no OAuth code in redirect (HTTP %s)" % resp.status_code)
auth_code = m.group(1)
token = exchange_token(sess, cfg, auth_code)
r = call_api(sess, "/student/signIn/getSignInInfo", r = call_api(sess, "/student/signIn/getSignInInfo",
{"signInId": sign_in_id}, token) {"signInId": sign_in_id}, token)
check_response(r, "getSignInInfo")
try: try:
j = r.json() j = r.json()
except ValueError as e: except ValueError:
raise UpstreamError("getSignInInfo returned non-JSON: %s" % e) raise UpstreamError("getSignInInfo returned non-JSON") from None
if not isinstance(j, dict):
raise UpstreamError("Malformed getSignInInfo response")
data = j.get("data") or {} data = j.get("data") or {}
if not isinstance(data, dict):
raise UpstreamError("Malformed getSignInInfo data")
model = data.get("signInModel") or {} model = data.get("signInModel") or {}
if not isinstance(model, dict):
raise UpstreamError("Malformed getSignInInfo model")
code = model.get("code") code = model.get("code")
raw_type = model.get("type")
if raw_type is None or raw_type == "":
raw_type = route.get("type")
try:
signin_type = int(raw_type)
except (TypeError, ValueError):
raise UpstreamError("Malformed sign-in type") from None
type_name = signin_type_label(signin_type)
if code is None or code == "": if signin_type == 1 and (code is None or code == ""):
raise NoSignInCodeError( raise NoSignInCodeError("数字码签到未配置签到码")
"Sign-in activity %s has no code configured" % sign_in_id) if signin_type == 2:
code = ""
result = { result = {
"signInId": sign_in_id, "signInId": sign_in_id,
"signInCode": code, "signInCode": code,
"type": model.get("type"), "type": signin_type,
"typeName": type_name,
"title": model.get("title"), "title": model.get("title"),
"status": data.get("signStatus"), "status": data.get("signStatus"),
} }
if do_commit: if do_commit:
ok, reason, _raw = commit_sign_in( if signin_type not in (1, 2):
sess, token, sign_in_id, batch_no, course_url, code result["commit"] = {
) "success": False,
result["commit"] = {"success": ok, "message": reason} "message": "%s暂不支持自动签到,需特定流程" % type_name,
}
else:
canonical_referer = GINKGO + urllib.parse.urlsplit(course_url).path
ok, reason, _raw = commit_sign_in(
sess, token, sign_in_id, batch_no, canonical_referer, code
)
result["commit"] = {"success": ok, "message": reason}
return result return result
# ==============================================================================
# Part 4b: Auto sign-in (mode=auto)
# 通知列表发现签到活动并逐个自动签到 —— 逻辑对齐 bot ginkgo 模块:
# fetch_all(通知+互动消息)→ _validated_signin_item(busi + 文本含"签到" +
# 可解析课程链接)→ 按 signInId 去重 → 每活动连续重试(首次+最多5次,间隔3s,
# token 失效重登录一次)→ 结果聚合为列表返回。
# ==============================================================================
_TAG_RE = re.compile(r"<[^>]+>")
def _html_to_text(raw):
if not raw:
return ""
text = re.sub(r"(?i)<br\s*/?>", "\n", raw)
text = re.sub(r"(?i)</(p|div|tr|li)>", "\n", text)
text = _TAG_RE.sub("", text)
text = _html.unescape(text)
text = re.sub(r"[ \t\u3000]+", " ", text)
text = re.sub(r"\n{2,}", "\n", text)
return text.strip()
def _parse_ts(value):
text = (value or "").strip()
if not text:
return 0.0
if text.endswith("Z"):
text = text[:-1] + "+00:00"
m = re.search(r"([+-]\d{2})(\d{2})$", text)
if m:
text = text[:m.start()] + m.group(1) + ":" + m.group(2)
try:
dt = datetime.fromisoformat(text)
except ValueError:
try:
dt = datetime.strptime(text[:19], "%Y-%m-%d %H:%M:%S")
except ValueError:
return 0.0
if dt.tzinfo is None:
dt = dt.replace(tzinfo=TZ8)
return dt.timestamp()
def _normalize_notice(raw):
title = (raw.get("title") or "").strip() or "(无标题)"
ts = _parse_ts(raw.get("createTime"))
return {
"kind": "notice",
"key": "N%s" % raw.get("id"),
"id": str(raw.get("id") or ""),
"title": title,
"text": _html_to_text(raw.get("content") or ""),
"ts": ts,
"link": NOTICE_PAGE,
"jump": "",
}
def _normalize_busi(raw):
text = _html_to_text(raw.get("content") or "")
jump = (raw.get("jumpUrl") or "").strip()
if jump.startswith("http"):
link = jump
elif jump.startswith("/"):
link = GINKGO + jump
else:
link = SYSTEM_PAGE
ts = _parse_ts(raw.get("createTime"))
return {
"kind": "busi",
"key": "B%s" % (raw.get("uuid") or ""),
"uuid": str(raw.get("uuid") or ""),
"id": "",
"title": text[:120] or "(无内容)",
"text": text,
"ts": ts,
"link": link,
"jump": jump,
}
def _api_headers(token):
return {
"User-Agent": UA,
"Origin": GINKGO,
"Referer": GINKGO + "/",
"Content-Type": "application/x-www-form-urlencoded",
"token": token,
"fingerprint": uuid.uuid4().hex,
}
def _call_json(sess, token, method, path, params=None, body=None, what="接口"):
"""消息类 API 调用(对齐 bot _call):check_response + code 必须为 2000。"""
headers = _api_headers(token)
if method == "GET":
r = sess.get(GINKGO + path, params=params, headers=headers, timeout=20)
else:
headers["Content-Type"] = "application/json"
r = sess.post(GINKGO + path, data=json.dumps(body if body is not None else {}),
headers=headers, timeout=20)
check_response(r, what)
try:
payload = r.json()
except ValueError:
raise UpstreamError("%s returned non-JSON (HTTP %s)" % (what, r.status_code))
if not isinstance(payload, dict):
raise UpstreamError("%s JSON must be an object" % what)
if is_rate_limited(None, payload):
raise RateLimitedError("%s 触发频率限制" % what)
if str(payload.get("code")) != "2000":
raise UpstreamError("%s returned code=%s" % (what, payload.get("code")))
return payload
def fetch_message_all(sess, token, page_size=PAGE_SIZE):
"""取两类消息(均为全部,含已读),返回 (notices, busi) 归一化列表。"""
j1 = _call_json(sess, token, "GET", "/student/message/getMessageAll",
params={"pageSize": page_size, "pageNo": 1, "readFlag": 0},
what="通知列表")
notices = [_normalize_notice(x) for x in ((j1.get("data") or {}).get("list") or [])]
j2 = _call_json(sess, token, "POST", "/student/busiMessage/list", body={},
what="互动消息列表")
busi = [_normalize_busi(x) for x in (j2.get("data") or [])]
return notices, busi
def _validated_signin_item(item):
"""Return an annotated copy only for a real interactive sign-in activity.
与 bot ginkgo 的 _validated_signin_item 同逻辑:仅互动消息、文本含"签到"、
跳转链接可解析为课程签到路由。query 参数可能携带 token,不入库不回显。"""
if (not isinstance(item, dict) or item.get("kind") != "busi"
or "签到" not in str(item.get("text") or "")):
return None
link = str(item.get("link") or "").strip()
try:
route = parse_course_url(link)
except (ValueError, TypeError, AttributeError):
return None
row = dict(item)
# Persist/display only the canonical path: query parameters may carry tokens
# and must never reach receipts or push logs.
row["link"] = GINKGO + urllib.parse.urlsplit(link).path
# signInId is the stable activity identity(同一活动的多条消息去重)
row["signin_activity_key"] = "S%s" % route["signInId"]
try:
row["signin_type"] = int(route["type"])
except (TypeError, ValueError, KeyError):
return None
row["signin_type_name"] = signin_type_label(row["signin_type"])
return row
def _signin_one_with_retry(username, password, row, sess, token):
"""对单个活动执行签到:失败连续重试(首次 + 最多 5 次,间隔 3s),
token 疑似失效(上游错误/登录错)时重新登录一次后立即重试(对齐 bot refresh 语义)。
与 bot complete_signin 一致:除限流(外层中断)外,任何未确认成功都消耗重试次数;
重试用尽记最终失败。返回 (entry, sess, token)。"""
link = row["link"]
base = {
"activityKey": row.get("signin_activity_key"),
"signInId": parse_course_url(link).get("signInId"),
"routeType": row.get("signin_type"),
"routeTypeName": row.get("signin_type_name"),
"link": link,
}
attempts = 0
refreshed = False
last_err = "unknown error"
last_data = None
while attempts < MAX_SIGNIN_ATTEMPTS:
attempts += 1
try:
data = fetch_sign_in_info(username, password, link, do_commit=True,
sess=sess, token=token)
last_data = data
commit = data.get("commit")
if isinstance(commit, dict) and commit.get("success") is True:
entry = dict(base)
entry.update({
"type": data.get("type"),
"typeName": data.get("typeName"),
"title": data.get("title"),
"signInCode": data.get("signInCode"),
"status": data.get("status"),
"success": True,
"message": "签到成功",
"attempts": attempts,
})
return entry, sess, token
message = ((commit or {}).get("message")
or "no commit result returned")
last_err = str(message)
except RateLimitedError:
raise
except (UpstreamError, AuthError) as e:
# 疑似 token 失效:重登录一次后立即重试(不消耗退避间隔,对齐 bot)
if not refreshed:
refreshed = True
try:
sess, token = login(username, password)
continue
except Exception:
pass # 重登录也失败:落入常规退避重试
last_err = str(e) or type(e).__name__
except NoSignInCodeError as e:
last_err = str(e)
except Exception as e:
last_err = safe_error(e)
if attempts < MAX_SIGNIN_ATTEMPTS:
time.sleep(SIGNIN_BACKOFF[min(attempts - 1, len(SIGNIN_BACKOFF) - 1)])
entry = dict(base)
if isinstance(last_data, dict):
entry.update({
"type": last_data.get("type"),
"typeName": last_data.get("typeName"),
"title": last_data.get("title"),
"signInCode": last_data.get("signInCode"),
"status": last_data.get("status"),
})
entry.update({"success": False,
"message": "签到失败(重试 %d 次): %s" % (attempts, last_err),
"attempts": attempts})
return entry, sess, token
def auto_sign_in_all(username, password):
"""mode=auto 主流程。返回聚合结果 dict(results 为列表)。"""
sess, token = login(username, password)
notices, busi = fetch_message_all(sess, token)
activities = {}
for item in notices + busi:
row = _validated_signin_item(item)
if row:
activities.setdefault(row["signin_activity_key"], row)
results = []
rate_limited = False
for key in sorted(activities, key=lambda k: activities[k].get("ts") or 0):
row = activities[key]
try:
entry, sess, token = _signin_one_with_retry(
username, password, row, sess, token)
except RateLimitedError:
# 对齐 bot:限流即中断本轮,已完成的结果照常返回
rate_limited = True
break
results.append(entry)
succeeded = sum(1 for r in results if r.get("success"))
return {
"mode": "auto",
"activities": len(activities),
"total": len(results),
"succeeded": succeeded,
"failed": len(results) - succeeded,
"rate_limited": rate_limited,
"results": results,
}
# ============================================================================== # ==============================================================================
# Part 5: HTTP service (WSGI app) # Part 5: HTTP service (WSGI app)
# ============================================================================== # ==============================================================================
@@ -496,17 +996,24 @@ def ginkgostu_sign_in():
url_b64 = args.get("url") or "" url_b64 = args.get("url") or ""
commit_arg = (args.get("commit") or "true").strip().lower() commit_arg = (args.get("commit") or "true").strip().lower()
do_commit = commit_arg not in ("0", "false", "no") do_commit = commit_arg not in ("0", "false", "no")
mode = (args.get("mode") or "").strip().lower()
if not (username and password_b64 and url_b64): if not (username and password_b64):
return _error(400, 400,
"Missing parameters: username, password(base64)")
if mode != "auto" and not url_b64:
return _error(400, 400, return _error(400, 400,
"Missing parameters: username, password(base64), url(base64)") "Missing parameters: username, password(base64), url(base64)")
try: try:
password = _b64_decode(password_b64, "password") password = _b64_decode(password_b64, "password")
course_url = _b64_decode(url_b64, "url") course_url = _b64_decode(url_b64, "url") if url_b64 else ""
except ValueError as e: except ValueError as e:
return _error(400, 400, str(e)) return _error(400, 400, str(e))
if mode == "auto":
return _auto_response(username, password)
log.info("query start user=%s commit=%s", username[:2] + "***", do_commit) log.info("query start user=%s commit=%s", username[:2] + "***", do_commit)
try: try:
data = fetch_sign_in_info(username, password, course_url, do_commit=do_commit) data = fetch_sign_in_info(username, password, course_url, do_commit=do_commit)
@@ -524,28 +1031,61 @@ def ginkgostu_sign_in():
"data": data, "data": data,
}) })
log.info("query ok user=%s code=%s", username[:2] + "***", data.get("signInCode")) log.info("query ok user=%s type=%s", username[:2] + "***", data.get("type"))
return jsonify({"code": 200, "msg": "ok", "data": data}) return jsonify({"code": 200, "msg": "ok", "data": data})
except AuthError as e: except AuthError:
log.warning("auth failed: %s", e) log.warning("authentication failed user=%s", username[:2] + "***")
return _error(401, 401, str(e)) return _error(401, 401, "Authentication failed")
except NoSignInCodeError as e: except NoSignInCodeError:
log.info("no code: %s", e) log.info("numeric code missing user=%s", username[:2] + "***")
return _error(404, 404, str(e)) return _error(404, 404, "数字码签到未配置签到码")
except UpstreamError as e: except UpstreamError:
log.error("upstream error: %s", e) log.error("upstream response error user=%s", username[:2] + "***")
return _error(502, 502, str(e)) return _error(502, 502, "Upstream response error")
except ValueError as e: except ValueError:
log.warning("bad request: %s", e) log.warning("bad request user=%s", username[:2] + "***")
return _error(400, 400, str(e)) return _error(400, 400, "Invalid request")
except requests.RequestException as e: except requests.RequestException:
log.error("upstream request failed: %s", e) log.error("upstream request failed user=%s", username[:2] + "***")
return _error(502, 502, "Upstream request failed") return _error(502, 502, "Upstream request failed")
except Exception: except Exception as exc:
log.exception("internal error") log.error("internal error type=%s", type(exc).__name__)
return _error(500, 500, "Internal error") return _error(500, 500, "Internal error")
def _auto_response(username, password):
"""mode=auto:发现 + 自动签到 + 聚合列表返回。"""
log.info("auto sign-in start user=%s", username[:2] + "***")
try:
data = auto_sign_in_all(username, password)
except AuthError:
log.warning("auto sign-in authentication failed user=%s", username[:2] + "***")
return _error(401, 401, "Authentication failed")
except RateLimitedError:
log.warning("auto sign-in rate limited user=%s", username[:2] + "***")
return _error(429, 429, "Upstream rate limited, please retry later")
except UpstreamError:
log.error("auto sign-in upstream error user=%s", username[:2] + "***")
return _error(502, 502, "Upstream response error")
except requests.RequestException:
log.error("auto sign-in upstream request failed user=%s", username[:2] + "***")
return _error(502, 502, "Upstream request failed")
except Exception as exc:
log.error("auto sign-in internal error type=%s", type(exc).__name__)
return _error(500, 500, "Internal error")
log.info("auto sign-in done user=%s total=%s succeeded=%s rate_limited=%s",
username[:2] + "***", data["total"], data["succeeded"],
data["rate_limited"])
if data["rate_limited"]:
msg = ("partial: upstream rate limited, %d/%d activities processed"
% (data["total"], data["activities"]))
elif data["failed"]:
msg = "ok with failures: %d/%d succeeded" % (data["succeeded"], data["total"])
else:
msg = "ok"
return jsonify({"code": 200, "msg": msg, "data": data})
@app.route("/healthz", methods=["GET"]) @app.route("/healthz", methods=["GET"])
def healthz(): def healthz():
return jsonify({"code": 200, "msg": "ok", "data": None}) return jsonify({"code": 200, "msg": "ok", "data": None})