AI News 09月24日 16:28
自主AI的治理:平衡自主性与问责制
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

文章探讨了自主人工智能(agentic AI)的兴起及其带来的机遇与挑战。随着AI越来越多地融入各行各业,自主AI系统能够独立决策并与企业系统交互,预示着下一波AI应用的飞跃。然而,这种更高的自主性也伴随着新的风险,包括AI可能偏离预期目标或违反商业规则。因此,在设计和部署自主AI时,必须从一开始就建立强大的人工监督、治理框架和透明度。低代码平台被视为一种有效的解决方案,它们可以作为自主代理和企业系统之间的控制层,将治理和合规性嵌入开发过程,从而在推进战略目标的同时降低风险。

🤖 **自主AI的崛起与机遇**: 自主AI系统超越了简单的洞察提供和任务自动化,它们能作为独立的代理,适应变化,与其他系统连接,并影响关键业务决策。这预示着AI应用的新高度,能够主动解决客户问题或动态调整应用以满足业务优先级。

⚠️ **自主AI带来的风险与挑战**: 尽管自主AI潜力巨大,但其高度自主性也带来了新的风险。如果没有适当的保障措施,AI代理可能偏离其预期目标,或做出与商业规则、法规或道德标准相悖的选择。这可能导致合规性差距、安全漏洞和声誉损害。

⚖️ **透明度与控制的重要性**: 在自主AI时代,透明度和问责制至关重要。当AI系统行为不透明时,领导者难以理解或验证其决策,从而侵蚀内部和客户的信心。缺乏监控的代理可能模糊问责制,扩大攻击面,并导致不一致的决策,因此需要强有力的治理框架来维持信任和控制。

🏗️ **低代码平台作为解决方案**: 文章提出低代码平台是安全扩展自主AI的一种可行途径。它们提供了一个可靠、可扩展的框架,将安全、合规和治理融入开发过程。通过将应用程序和代理开发统一在一个环境中,并嵌入DevSecOps实践,组织可以确保AI驱动的流程符合战略目标,同时降低不必要的风险。

🧑‍💼 **开发者角色的转变**: 随着自主AI系统的成熟,开发者的角色正在从逐行编写代码转变为定义指导AI行为的保障措施。开发者和IT领导者需要承担更广泛的监督角色,引导技术和组织变革,确保AI驱动的决策可靠、可解释并与业务目标保持一致。

Author: Rodrigo Coutinho, Co-Founder and AI Product Manager at OutSystems

AI has moved beyond pilot projects and future promises. Today, it’s embedded in industries, with more than three-quarters of organisations (78%) now using AI in at least one business function. The next leap, however, is agentic AI: systems that don’t just provide insights or automate narrow tasks but operate as autonomous agents, capable of adapting to changing inputs, connecting with other systems, and influencing business-critical decisions. Although these agents will deliver greater value, agentic AI also poses challenges.

Imagine agents that proactively resolve customer issues in real-time or adapt applications dynamically to meet shifting business priorities. The greater autonomy inevitably brings new risks. Without the right safeguards, AI agents may drift from their intended purpose or make choices that clash with business rules, regulations, or ethical standards. Navigating this new era requires stronger oversight, where human judgement, governance frameworks, and transparency are built-in from the start. The potential of agentic AI is vast but so are the obligations that come with deployment. Low-code platforms offer one path forward, serving as a control layer between autonomous agents and enterprise systems. By embedding governance and compliance into development, they give organisations the confidence that AI-driven processes will advance strategic goals without adding unnecessary risk.

Designing safeguards instead of code for agentic AI

Agentic AI marks a steep change in how people interact with software. It’s indicative of a fundamental shift in the relationship between people and software. Traditionally, developers have focused on building applications with clear requirements and predictable outputs. Now, instead of fragmented applications, teams will orchestrate entire ecosystems of agents that interact with people, systems and data. 

As these systems mature, developers shift from writing code line by line to defining the safeguards that steer them. Because these agents adapt and may respond differently to the same input, transparency and accountability must be built in from the start. By embedding oversight and compliance into design, developers ensure AI-driven decisions stay reliable, explainable and aligned with business goals. The change demands that developers and IT leaders embrace a broader supervisor role, guiding both technological and organisational change over time. 

Why transparency and control matter in agentic AI

Greater autonomy exposes organisations to additional vulnerabilities. According to a recent OutSystems study, 64% of technology leaders cite governance, trust and safety as top concerns when deploying AI agents at scale. Without strong safeguards, these risks extend beyond compliance gaps to include security breaches and reputational damage. Opacity in agentic systems makes it difficult for leaders to understand or validate decisions, eroding confidence internally and with customers, leading to concrete risks.

Left unchecked, autonomous agents can blur accountability, widen the attack surface and create inconsistency at scale. Without visibility into why an AI system acts, organisations risk losing accountability in critical workflows. At the same time, agents that interact in sensitive data and systems expand the attack surface for cyber threats, while un-monitored “agent sprawl” can create redundancy, fragmentation and inconsistent decisions. Together, these challenges underscore the need for strong governance frameworks that maintain trust and control as autonomy scales. 

Scaling AI safely with low-code foundations

Crucially, adopting agentic AI need not involve rebuilding governance from the ground up. Organisations have multiple approaches available to them, including low-code platforms, which offer a reliable, scalable framework where security, compliance and governance are already part of the development fabric.

Across enterprises, IT teams are being asked to embed agents into operations without disrupting what already works. With the right frameworks, IT teams can deploy AI agents directly into enterprise-wide operations without disrupting current workflows or re-architecting core systems. Organisations have full control over how AI agents operate at every step, ultimately building trust to scale confidently in the enterprise.

Low-code places governance, security and scalability at the heart of AI adoption. By unifying app and agent development in a single environment, it is easier to embed compliance and oversight from the start. The ability to integrate seamlessly in enterprise systems, combined with built-in DevSecOps practices, ensures that vulnerabilities are addressed before deployment. And with out-of-the-box infrastructure, organisations can scale confidently without having to reinvent foundational elements of governance or security.

The approach lets organisations pilot and scale agentic AI while keeping compliance and security intact. Low-code makes it easier to deliver with speed and security, giving developers and IT leaders confidence to progress.

Smarter oversight for smarter systems

Ultimately, low-code provides a dependable route to scaling autonomous AI while preserving trust. By unifying app and agent development in one environment, low-code embeds compliance and oversight from the start. Seamless integration in systems and built-in DevSecOps practices help address vulnerabilities before deployment, while ready-made infrastructure enables scale without reinventing governance from scratch. For developers and IT leaders, this shift means moving beyond writing code to guiding the rules and safeguards that shape autonomous systems. In a fast-changing landscape, low-code provides the flexibility and resilience needed to experiment confidently, embrace innovation early, and maintain trust as AI grows more autonomous.

Author: Rodrigo Coutinho, Co-Founder and AI Product Manager at OutSystems

(Image by Alexandra_Koch)

See also: Agentic AI: Promise, scepticism, and its meaning for Southeast Asia

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Governing the age of agentic AI: Balancing autonomy and accountability   appeared first on AI News.

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

Agentic AI 自主AI AI治理 低代码 问责制 透明度 Low-code AI Governance Accountability Transparency
相关文章