Ting BLOG🐈

论文 · Bayesian Inference of Contextual Bandit Policies via Empirical Likelihood

Ting published on 2026-07-01 included in 论文阅读

基础信息

会议/期刊：Journal of Machine Learning Research 27 (2026), 1-28
关键词：empirical likelihood, contextual bandit, off-policy evaluation, Bayesian inference, policy comparison

核心贡献：本文用 Bayesian empirical likelihood 构造 contextual bandit policy value 的联合后验和差值后验，使小样本 off-policy policy evaluation 与 policy comparison 能获得更稳健的不确定性量化。

论文 · Distributional Soft Actor-Critic With Three Refinements

Ting published on 2026-07-01 included in 论文阅读

基础信息

会议/期刊：IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 47, No. 5, May 2025
关键词：reinforcement learning, soft actor-critic, distributional RL, value overestimation, continuous control

核心贡献：本文在 DSACv1 基础上提出 DSAC-T/DSACv2，通过 expected value substitution、twin value distribution learning 和 variance-based critic gradient adjustment 降低 critic 梯度随机性与 Q-value 估计偏差，在连续控制基准和真实移动机器人任务中提升稳定性与性能。

论文 · Generative Agents: Interactive Simulacra of Human Behavior

Ting published on 2026-07-01 included in 论文阅读

基础信息

会议/期刊：UIST 2023
论文主题：用大语言模型构造能记忆、反思、规划并互动的 believable generative agents

核心贡献：本文提出 generative agents 架构，将 LLM 与 memory stream、reflection、planning 结合，使 25 个智能体在 Smallville 沙盒世界中表现出可信的个体行为和信息扩散、关系形成、活动协调等群体行为。

论文 · Hopfield-Fenchel-Young Networks: A Unified Framework for Associative Memory Retrieval

Ting published on 2026-07-01 included in 论文阅读

基础信息

会议/期刊：Journal of Machine Learning Research 26 (2025), 1-51
关键词：Hopfield networks, associative memory, Fenchel-Young loss, sparse transformations, structured prediction

核心贡献：本文提出 Hopfield-Fenchel-Young Networks，用两个 Fenchel-Young loss 的差构造统一能量框架，把 classical/modern Hopfield networks、sparse transformations、structured retrieval 和 post-transformations 纳入同一凸分析视角，并在多类记忆检索任务上验证稀疏与结构化检索能力。

论文 · MemGPT: Towards LLMs as Operating Systems

Ting published on 2026-07-01 included in 论文阅读

基础信息

会议/期刊：arXiv preprint, 2023
论文主题：用操作系统式分层记忆与 virtual context management 扩展固定上下文 LLM

核心贡献：本文提出 MemGPT，将 LLM 类比为操作系统中的处理器，通过 main context、external context、archival storage、recall storage 和 function calling 实现 virtual context management，使固定上下文模型能处理长对话和长文档任务。

论文 · Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents

Ting published on 2026-07-01 included in 论文阅读

基础信息

会议/期刊：Journal of Machine Learning Research 26 (2025), 1-40
关键词：deep reinforcement learning, memory, benchmark, Transformer-XL, GRU, PPO

核心贡献：本文把 Memory Gym 的三个有限 POMDP 任务扩展为 endless cumulative memory tasks，并提供 CleanRL 中基于 PPO 的 Transformer-XL baseline，证明有限任务上的 sample efficiency 不足以衡量长期 memory effectiveness，且 GRU 在 endless 任务中显著优于 TrXL。