cs.AI updates on arXiv.org 09月25日 13:44
LLMs自动恢复用户故事研究
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

本文研究了大型语言模型(LLMs)能否从源代码中自动恢复用户故事,并探讨了提示设计对输出质量的影响。通过1750个不同复杂度的C++代码片段,评估了五种最先进的LLMs在六种提示策略下的表现,结果表明,所有模型在200行代码以下的F1分数平均为0.8。研究发现,一个示例能够使最小模型(8B)的性能匹配70B的大模型,而结构化推理的Chain-of-Thought仅对大型模型有轻微提升。

arXiv:2509.19587v1 Announce Type: cross Abstract: User stories are essential in agile development, yet often missing or outdated in legacy and poorly documented systems. We investigate whether large language models (LLMs) can automatically recover user stories directly from source code and how prompt design impacts output quality. Using 1,750 annotated C++ snippets of varying complexity, we evaluate five state-of-the-art LLMs across six prompting strategies. Results show that all models achieve, on average, an F1 score of 0.8 for code up to 200 NLOC. Our findings show that a single illustrative example enables the smallest model (8B) to match the performance of a much larger 70B model. In contrast, structured reasoning via Chain-of-Thought offers only marginal gains, primarily for larger models.

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

大型语言模型 用户故事 代码分析 提示设计 性能评估
相关文章