feat: V-learning架构 — 改Q→V
Q-learning问题: pass action Q值稳定偏高 → 永远pass V-learning修复: - 网络输出 14(Q-action)→1(V-state): P(win|state) - NeuralBot: 对每个候选编码出牌后state → 选V值最高的 - pass和出牌都是候选, 公平竞争(不是固定action slot) TrainingCollector: - 新增 EncodeAfterPlay(view, tracker, afterHand, playedIds) - EncodeState 支持 extraSeen 参数(模拟出牌时标记已见) Python: - train_v.py: V-learning训练器(load CSV→predict win/loss) - network.py: OUTPUT_DIM=14→1 验证: 200局0错误, V-model loss 0.81→0.0001 需更多数据覆盖状态空间
This commit is contained in:
@ -392,6 +392,7 @@ namespace PdkFriendServer.Logic
|
||||
int totalSims = allCandidates.Count * fastSims
|
||||
+ scoredCandidates.Count(c => c.winRate > 0.3) * (simsPerCandidate - fastSims);
|
||||
Console.WriteLine($"[ISMCTS pos{view.MyPos}] {totalSims}sims => {desc}");
|
||||
TrainingCollector.Record(view, _tracker, TrainingCollector.ActionToIdx(best.play), view.MyPos-1);
|
||||
return best.play;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user