cs.AI updates on arXiv.org 10月08日 12:15
图神经网络在软件测试中的视觉变化检测
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

本文提出一种基于图神经网络的软件测试中视觉变化检测方法,利用YOLOv5模型识别UI控件,构建图形模型,通过结构、视觉和文本线索进行相似度计算,有效检测UI元素变化。

arXiv:2405.00874v2 Announce Type: replace-cross Abstract: Automated software testing is integral to the software development process, streamlining workflows and ensuring product reliability. Visual testing, particularly for user interface (UI) and user experience (UX) validation, plays a vital role in maintaining software quality. However, conventional techniques such as pixel-wise comparison and region-based visual change detection often fail to capture contextual similarities, subtle variations, and spatial relationships between UI elements. In this paper, we propose a novel graph-based approach for context-aware visual change detection in software test automation. Our method leverages a machine learning model (YOLOv5) to detect UI controls from software screenshots and constructs a graph that models their contextual and spatial relationships. This graph structure is then used to identify correspondences between UI elements across software versions and to detect meaningful changes. The proposed method incorporates a recursive similarity computation that combines structural, visual, and textual cues, offering a robust and holistic model of UI changes. We evaluate our approach on a curated dataset of real-world software screenshots and demonstrate that it reliably detects both simple and complex UI changes. Our method significantly outperforms pixel-wise and region-based baselines, especially in scenarios requiring contextual understanding. We also discuss current limitations related to dataset diversity, baseline complexity, and model generalization, and outline planned future improvements. Overall, our work advances the state of the art in visual change detection and provides a practical solution for enhancing the reliability and maintainability of evolving software interfaces.

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

软件测试 视觉变化检测 图神经网络 YOLOv5 UI元素
相关文章