ZyVOP Logo
Content That Connects
SeriesAI NewsWhy ZyVOPJoin Discord
ZyVOP Logo
Content That Connects

Empowering developers and creators with cutting-edge insights, comprehensive tutorials, and innovative solutions for the digital future.

Content

  • Categories
  • Tags
  • Badges
  • Leaderboard
  • Write Article
  • Newsletter

Company

  • About Us
  • Why ZyVOP
  • API Documentation
  • Write for Us
  • Contact

Connect

  • Privacy Policy
  • Terms of Service
  • Cookie Policy
  • DMCA Policy
  • Code of Conduct

© 2026 ZyVOP. Crafted with care for the developer community.

Made with ❤️ by the ZyVOP team
All systems operational
HomeWisdom War 2025: A Complete Review of the Tencent Cloud AI Penetration Competition

Wisdom War 2025: A Complete Review of the Tencent Cloud AI Penetration Competition

I0veD
I0veDcyber security researcher
August 12, 2026
31 min read
Wisdom War 2025: A Complete Review of the Tencent Cloud AI Penetration Competition
Article

Actual review based on the 2025 “1129” AI Penetration Competition


author: cdxiaodong
date: 2025/11/29
Competition name:Tencent Yunding AI Penetration Competitiontype:Technical review and future expectations


The review was basically completed on December 3, 2025, and the follow-up is the iteration of each version. AI is making rapid progress and version iterations are also rapid. I hope this article will be of some help to those who are interested.

Table of contents

  • Competition background and basic introduction

  • Panoramic analysis of participating teams

  • core glossary

  • In-depth comparison of technical routes

  • Comparison of core problem solutions

  • Panoramic comparison table of technical solutions of the nine major teams

  • Comprehensive analysis and summary

  • Future trends and recommendations


Competition background and basic introduction

Competition overview

This competition is the world's first practical competition focusing on AI automated penetration testing. It brings together security teams from top universities such as Tsinghua University, Xi'an Jiaotong University, City University of Hong Kong, Chengdu University, and Guangzhou University, as well as enterprise security teams such as NSFOCUS and Changting Technology.

Core goals of the competition:

  • Explore the boundaries of AI’s actual capabilities in automated penetration testing

  • Compare the actual combat effects of different AI architectures and security strategies

  • Promote the engineering implementation of AI technology in the field of network security

Competition question setting analysis

Test question bank overview:

  • Total number of questions:104 network security challenge questions

  • Difficulty distribution:

Vulnerability type coverage:

  • High frequency vulnerabilities: XSS, default credentials, IDOR, privilege escalation

  • Medium frequency vulnerabilities: SSTI, command injection, SQL injection

  • Low frequency but high risk: deserialization, code execution, file inclusion

Feature analysis:This is a comprehensive exercise question bank covering everything from basic logic vulnerabilities to complex code execution vulnerabilities. It includes both traditional Web vulnerabilities and security threats of modern Web applications.

XBOW data set analysis (actual game data)

Image

XBOW data set analysis

Data source:Tencent Yunding AI Penetration Competition Live Demonstration

Key findings:

  • Map the vulnerabilities involved in all environments to the OWASP TOP 10

  • Injection vulnerabilities (A01) ranked first with 59 times, accounting for the highest proportion

  • Access control vulnerabilities (A02) 29 times, ranking second

  • Unsafe deserialization (A03) 18 times, ranked third

OWASP Top 10 coverage statistics:

category

quantity

Proportion

A01: Injection

59

39.6%

A02: Broken Access Control

29

19.5%

A03: Insecure Deserialization

18

12.1%

A04: Insecure Design

7

4.7%

A05: Security Misconfiguration

7

4.7%

A06: Vulnerable Components

5

3.4%

A07: Authentication Failures

3

2.0%

A08: Integrity Failures

3

2.0%

A09: Logging Failures

3

2.0%

A10: SSRF

3

2.0%

Practical insights:Injection vulnerabilities and access control vulnerabilities account for nearly 60%, which are areas that AI penetration testing needs to focus on.

Comparison of existing AI assessment tools

Image

AI Assessment Tools Comparison Table

Comparative analysis:

  • RAVEN:NYU CTF benchmark accuracy is 22%, but the test range is limited

  • CTFKnow:InterCode - CTF increased by 85%, based on 1084 Whitepaper

  • Cybench:Maximum success rate 89.4%, context 129k tokens

  • CABench:Improve CTF success rate to 67-75%, sample size >20,000

  • CAI:The convergence speed is increased by 197-900%, but there are experimental deviations

  • Automation Gap:Based on CAI 2026, the analysis perspective is limited

Each of these tools has its own advantages and disadvantages, which provide an important reference for the selection of technical routes for this competition.

Six-stage evolution of automated penetration testing

Image

Automation evolution path - two-tier process architecture

Image

Generate Picture - Automated Penetration Testing Evolution Timeline (Chinese)

stage

level

describe

Analogy to autonomous driving

P0

artificial infiltration

Complete reliance on human experts

L0 - No automation

P1

Tool enhancements

Tools perform fixed steps, humans make decisions

L1 - Driving Assistance

P2

Partially automated

Automated independent steps, unable to run the complete attack chain

L2 - Partially automated

P3

Conditional automation

Automatically complete the limited utilization chain, key nodes need to be confirmed

L3 - Conditional autonomous driving

P4

Highly automated

Automatically run the complete attack chain, and humans are responsible for verification

L4 - Highly automated driving

P5

Fully automated

The entire link operates autonomously without human intervention.

L5 - Fully autonomous driving

Current situation analysis:Most of the top teams currently participating are in theP3-P4It can automatically complete penetration testing under specific conditions, but human experts are still required to intervene in complex scenarios and unknown vulnerabilities.

TOKEN consumption and cost analysis

Cost comparison of each team:

team

average cost/question

Cost description

main model

Xi'an Jiaotong University

About 1,000 yuan/day

Training phase cost

N/A

Antix(Chengxinda)

19.2 yuan

Kimi consumes more

Kimi

Green Alliance

13 yuan

High cost performance

DeepSeek

yhy0

13.7 yuan

All with DeepSeek

DeepSeek

Simple question reference

0.8 yuan

Take XBow-039-24 as an example

N/A

Image

Cost comparison analysis - Token consumption of each team

Key points of cost analysis:

  1. Problem costs skyrocket- Failure to call tools and deviation from intentions will cause costs to rise exponentially

  1. Model selection impact- DeepSeek has the best price/performance ratio, and Kimi has the advantage in solving difficult problems.

  1. Optimization direction- Reduce invalid rounds, optimize context compression, and accurately call tools


Panoramic analysis of participating teams

Important note:The contents of each team's report are mostly future plans and expectations and do not represent their current complete ability level.

Outside Changting - Champion Team (NO.1)

Core features:Coded Agent interface | Programming instead of prompts | Code planning | DSPy frameworkNote: This article uses MPI to refer to "Model Programming Interface" (Model Programming Interface)

Team motto: “Programming not Prompting”

Championship achievements

Final ranking:First place (championship)

Obvious advantages:

There is a big gap between the third place and the champion and runner-up

Cost input:

  • On average each team spends approx.30,000 yuan

  • Get the highest return at high costs

  • Proved the effectiveness of the "high input, high output" strategy

Core members:Doctoral team of Xi'an Jiaotong University

Core innovation: MPI (Model Programming Interface)

Image

Outside Changting - MPI architecture diagram

Image

Generate pictures - How MPI works (Chinese)

Core Paradigm: Programming not Prompting

Comparison of design concepts:

Traditional solution (Prompting):

用长篇大论的Prompt去"哄"模型干活
→ 累赘、不稳定、难以维护

Changting Programming:

使用Python代码显式定义业务逻辑
→ 精确、稳定、易维护

achieve comparison

Advantage comparison:

Dimensions

Prompt method

code method

Accuracy

Vague

Accurate ✅

stability

unstable

Stable ✅

Maintainability

Difficult to maintain

Easy to maintain ✅

logical expression

Difficult to be precise

Full control ✅

Execution and suspension mechanisms

How it works:

1. 脚本在容器中运行
2. 遇到自定义工具调用
3. "暂停" (Script pauses)
4. 执行工具并获取结果
5. "恢复" (Resume script)
6. 继续执行后续代码

value:Make long process control very precise and every step controllable.

DSPy framework concept

DSPy = Programmatic LLM Interactions

Core values:

  • Automatically optimize Prompt (no need to manually adjust parameters)

  • Programmatically define the LLM calling process

  • Typed input and output guarantees

"Let AI write prompts by itself instead of humans guessing"

Programming vs Prompt Words:

Prompt word project:

  • Advantages: Simple and easy to use

  • Disadvantages: unstable, difficult to debug, difficult to maintain, difficult to expand

Code planning:

  • Advantages: Stable, easy to debug, easy to maintain, easy to expand

  • Disadvantages: Requires programming skills

Success factor analysis

  1. The advancement of technical concepts ✅

  1. Engineered Integrity ✅

  1. Depth of expertise in the team ✅

  1. Effective use of high investment ✅

Lessons to learn from

  1. Code over hints⭐⭐⭐⭐⭐ - This is the clear direction

  1. Execution pause mechanism⭐⭐⭐⭐⭐ - Precise process control

  1. Engineering thinking⭐⭐⭐⭐⭐ - Complete frame, ready to stand

  1. DSPy framework⭐⭐⭐⭐⭐ - Worth studying and ready to use


Xi'an Jiaotong University - Runner-up Team (NO.2)

Core features:Distributed cluster architecture | XML strongly typed protocol | Three-stage pipeline | Bottom-level tool encapsulation

Runner-up achievements

Final ranking:Second place (runner-up)

The gap with the champion:

  • Second only to Changtingwai

  • Clearly ahead of third place

  • Demonstrated strong comprehensive strength

Core team:Doctoral team of Xi'an Jiaotong University

Core innovation: distributed cluster architecture

Image

Xi'an Jiaotong University - Distributed Master-Worker Architecture

Image

Generate pictures - Distributed Master-Worker cluster architecture (Chinese)

Master-Worker mode

Design concept:

Problems with traditional stand-alone architecture:

  • CPU limitations, memory limitations, network bandwidth limitations, computing power bottlenecks

XJTJ’s distributed solution:

Master节点 (任务管理)
    ↓
    ├─ Worker节点1 ─┐
    ├─ Worker节点2 ─┤
    ├─ Worker节点3 ─┼→ 并发执行
    └─ Worker节点N ─┘

Advantage analysis

  1. horizontal scalability- Need more computing power? Just add Worker nodes

  1. High availability-A worker fails? Other workers continue to work

  1. load balancing- Master intelligent allocation to avoid overloading of a certain Worker

  1. Task decoupling- Task management and execution are separated and optimized independently

Core innovation: XML strongly typed protocol

Why use XML?

Problems with traditional JSON:

  • Special characters need to be escaped

  • Nested structures can be confusing

  • Ambiguous type

XML solution:

<payload>
    <
![CDATA[
    <script>alert(1)</script>
    ]]>
</payload>

Advantages:

  • ✅ Special characters do not need to be escaped

  • ✅ Clear structure

  • ✅Strong type constraints

  • ✅ Perfectly resolve Payload conflicts

Trade-offs to consider:

  • ⚠️ Larger file size (Token consumption increases by 20-30%)

  • ⚠️ Parsing speed is slightly slower than JSON

  • ⚠️ Modern toolchains have better support for JSON

in conclusion:In the attack scenario, Payload correctness > transmission efficiency, so choose XML

communication process

Master → Worker:
<task>
  <target>http://example.com</target>
  <port>80</port>
  <payload><
![CDATA["' OR 1=1--]]></payload>
</task>

Worker → Master:
<result>
<status>success</status>
<data>flag{xxx}</data>
</result>

Clear, stable, and less error-prone!

Core innovation: three-stage assembly line

Image

Generate images - three-stage pipeline processing (Chinese)

Stage 1: Page Exploration

  • Crawl website structure

  • Identify all URLs

  • Analyze page type

  • Extract form and parameters

Stage 2: Scene Modeling

  • Analyze business logic

  • Identify data flows

  • Build attack graph

  • Plan attack path

Stage 3: Vulnerability Detection

  • Send attack request

  • Analyze response

  • Verify that the vulnerability exists

  • GetFlag

Pipeline value:

  1. Standardized process - clear responsibilities at each stage

  1. Reusability - each stage can be optimized independently

  1. Scalability - easy to add new stages

Core innovation: underlying tool encapsulation

urllib underlying library

Why not use requests?

  • requests: Automatic encoding may interfere with attack traffic, automatic processing may change the payload, and uncontrollable automatic redirection

  • urllib: Completely control requests, accurately construct attack traffic, and avoid automatic interference

Complex coding templates

jsfuck and other encodings are integrated into Prompt:

  • Avoid automatic encoding interference

  • Precisely control attack traffic

  • Improve attack success rate

Ranking analysis

Advantages ✅

  1. Distributed architecture- Powerful computing power, good scalability, suitable for large-scale tasks

  1. Engineering perfection- The XML protocol is stable, the pipeline is clear, and the tool packaging is professional

  1. Technical depth- Precise bottom-level control, avoid automatic interference, and high attack quality

Disadvantages ⚠️

  1. High complexity- Distributed architecture is difficult to maintain and XML protocol learning costs are high

  1. Not enough flexibility- Standardized processes may not be flexible enough to cope with unknown scenarios

  1. The gap with the champion- Chaiting’s “code planning” is more precise, the control granularity is finer, and the DSPy framework is more advanced

vs Changtingwai (Champion) comparison

Dimensions

Xi'an Jiaotong University

outside the pavilion

core innovation

Distributed+XML

Code Planning (DSPy)

Architecture complexity

extremely high

high

Control granularity

thin

extremely fine

flexibility

Low

middle

Scalability

Extremely strong

powerful

Engineering

Complete

Complete

Gap analysis:

  1. Accuracy of code planning- Python is more flexible and intuitive

  1. Pipeline vs code generation- Code generation is more dynamic

  1. Distributed vs stand-alone high performance- A single machine is sufficient in CTF scenarios, precise control is more important

Applicable scenarios

The most suitable scenario ✅✅✅:

  • Large-scale penetration testing (needs to test a large number of targets simultaneously)

  • Enterprise-level SaaS platform (requires high availability and horizontal expansion)

  • Long-term operational projects (maintenance costs of complex architecture are acceptable)

Scenarios that may not be suitable ⚠️:

  • Small CTF competition (distributed architecture is not needed, single machine is enough)

  • Rapid prototyping (steep learning curve, long development cycle)

  • Cost-sensitive scenarios (requires multiple servers and high maintenance costs)

Lessons to learn from

  1. Distributed architecture ideas⭐⭐⭐⭐⭐ - Decoupling of task management and execution, horizontal scalability

  1. XML strongly typed protocol⭐⭐⭐⭐ - Payload conflict resolution, type safety

  1. Three-stage pipeline⭐⭐⭐⭐⭐ - Standardized process, reusable design

  1. Low-level tool encapsulation⭐⭐⭐⭐ - Precisely control attack traffic and improve attack quality


BinX (Guangzhou University) - Third place team (NO.3)

team:Guangzhou UniversityCore features:Three-person special team structure | Sandbox cleaning | Meta-tool orchestration | Intent map

Three core issues facing

01 Environmental complexity leads to perceived status distortion (Status Distortion)

Problem description:

  • Long-term tasks lead to context loss

  • Complex environment causes misjudgment of status

  • Ineffective noise drowns out critical attack signals

  • Action history cannot be tracked over the long term

Core pain points:

#model doesn’t know who he is, where he is, and what he has done#

02 Weak feedback from the environment leads to extremely low exploration efficiency (Low Efficiency)

Problem description:

  • Effective signals are extremely scarce

  • Intermediate feedback is highly ambiguous or even misleading

  • Huge search space leads to “blind testing”

Core pain points:

#The model is like groping in the fog, without clues to guide it, so it can only try blindly#

03 Limitations in structural cognition lead to unreliable decision-making (Unreliable Decision-Making)

Problem description:

  • Field illusion, misattribution

  • The chain of reasoning is fragile and easily interrupted

  • Solid strategy and lack of flexibility

Core pain points:

#Models can speak but cannot reason, can generate but cannot reflect, can imitate but lack creative strategies#

core solution

Image

BinX - Three-person special team structure

Solution 01: Sandbox Cleaning

Image

Generate images - sandbox cleaning mechanism

#Core Concept:## "Only look at the results, not the process noise"#

Implementation mechanism:

  1. Isolation sandbox preprocessing- The original massive output of the tool enters the isolation sandbox first

  1. Key conclusion extraction- Only the key conclusions after cleaning are sent back to the Agent

  1. Effect- Completely eradicate context pollution and significantly reduce Token waste

Advantages:

  • Reduce contextual pressure

  • Improve decision quality

  • Reduce cost consumption

Solution 02: Python Meta-Tooling (meta-tool orchestration)

#Core Concept:## “Use code execution to replace conversation idling”#

Implementation mechanism:

  1. Give Agent programming capabilities- Agent can write Python code

  1. control flow transformation- Convert "multi-turn dialogue simulation control flow" into "sandbox code execution flow"

  1. Complex logic self-closed loop- Realize instant self-closing loop of complex logic

Advantages:

  • Execution efficiency is greatly improved

  • Reduce conversation turns

  • More precise logic control

Solution 03: Agent Pattern Graph (APG/Intent Graph)

#Core Concept:## “Equip AI with expert brain circuits”#

Implementation mechanism:

  1. Declarative graph structure- In YAML format

  1. Standard Operating Procedure (SOP)- Codify professional attack and defense processes

  1. Effect- Eliminate the illusion of reasoning, avoid blind temptation, and standardize the attack process

Architectural highlights: "Three-person special team" that can think

Image

Generate Image - Three-Man Special Forces Structure (Chinese)

This architecture basically solves the problem of traditional AI being prone to "random running" and "random guessing":

1. Brain planning layer

Responsibilities:Responsible for tactical formulation and overall strategy

2. Hands and feet execution layer

Responsibilities:Responsible for specific operations and execution

3. Post-review layer

Responsibilities:Responsible for experience summarization and learning

Two core pictures:

  1. task map- Streamline the process, support multi-tasking in parallel, and automatically fuse if the front fails

  1. cause and effect diagram- Get the logic right, don’t rely on probabilities, and use evidence to speak like an experiment.

Let AI evolve from "rigid script running" to a practical expert with "memory, error correction, and understanding of tactics."

Ranking analysis

Final ranking:Second runner-up (NO.3)

Success factors:

  1. Systematic methodology - three major issues clearly identified

  1. Innovative solutions - sandbox cleaning, meta-tool orchestration, intent mapping

  1. Engineering architecture - clear division of labor among three layers

  1. Combination of theory and practice - both in-depth thinking and practical implementation

Analysis of the gap with first place:

  • May be slightly inferior in execution efficiency

  • Cost control may not be optimal

  • But the overall architecture design has great reference value

Lessons to learn from

  1. problem driven design- Clearly identifying the core problem is the first step

  1. The value of multi-tier architecture- A clear division of labor is more effective than working alone

  1. The importance of noise management- Sandbox cleaning is an idea worth learning

  1. Code over conversation- Using code to implement logic is more efficient than multiple rounds of dialogue


Antix (Shengxin University) - 4th place

team:Chengxin UniversityCore features:Meta-Tooling | Jupyter Kernel | Minimalist Architecture | Code as ActionTeam motto:“Simple is always best”

Core concept: minimalism

Design philosophy:The Antix team advocates minimalist design, giving full play to core model capabilities and rejecting excessive complexity.

Core philosophy:

Let AI directly generate intent and execute code
Clean context and less noise
High efficiency

Code size:

  • The code of Agent itself is only100-200 lines

  • Called "Baby Runtime"

  • Extremely streamlined and powerful

Core solution: Meta-Tooling (meta-tool mode)

Image

Generate images - Meta-Tooling workflow

Image

Antix - Jupyter Kernel state management

single interface principle

Image

Generate pictures - MCP protocol workflow

Through the MCP protocol, the system is only exposed to the Agenta core function:

execute_code(code: str) -> result

Workflow:

传统方式:
Agent → tool_a() → tool_b() → tool_c()...
上下文爆炸,参数混乱

Antix方式:
Agent → execute_code(python_script)
代码内部调用各种工具
上下文清爽,逻辑清晰

Code is action

Image

Antix - Meta-Tooling tool calling mode

Agent no longer calls the scanner directly, but writes a Python script:

import requests
from bs4 import BeautifulSoup

扫描目标

url = "http://target.com"
response = requests.get(url)

分析响应

if "admin" in response.text:
print("Found admin panel!")

利用漏洞

payload = {"username": "admin", "password": "' OR 1=1--"}
exploit = requests.post(url + "/login", data=payload)

Ability improvement:

  • ✅ Combined tools - scripts can call multiple tools at the same time

  • ✅ Processing data logic - complex data processing is done in code

  • ✅ Write Exploits on the spot - you can immediately write exploit code when encountering new vulnerabilities

  • ✅ Unlimited flexibility - not limited by predefined toolsets

Antix Sandbox

Customized Ubuntu Docker

Core features:An infiltration environment designed specifically for AI, not traditional Kali.

Pre-installed tools:

  • sqlmap - SQL injection automation

  • CAIDO - Traffic Brokering and Analysis

  • Other commonly used penetration tools

Traffic monitoring mechanism:

浏览器/工具 → 自动经过CAIDO代理 → 目标网站
                      ↓
                记录所有请求
                      ↓
                AI可见所有细节

VNC visualization:

  • Although it is for AI, humans can connect in real time through VNC.

  • "Looking at" AI operating terminals and browsers

  • Achieve good "human-machine alignment"

Execution layer: Stateful Jupyter Kernel

Session persistence mechanism

Core innovation:Python Executor is essentially aStateful Jupyter Kernel。

How it works:

# 第一轮对话
session = requests.Session()
session.post("http://target.com/login", data=credentials)

第二轮对话

session变量仍然存在!

response = session.get("http://target.com/admin")

Cookie和登录态保持完整

第三轮对话

继续使用session

session.get("http://target.com/flag")

Problems solved:

  • ✅ Perfect solution to cookie and login status retention

  • ✅ Variables persist across multiple conversations

  • ✅ In line with programming logic and more natural

Compared with traditional solutions:

传统方案:
每轮都需要序列化状态 → 传回Prompt → 下一轮反序列化
效率低,容易出错

Antix方案:
变量留在内存中 → 直接使用
高效,符合直觉

Potential risks and mitigations:

  • ⚠️ Kernel crash leading to state loss → Periodic checkpoint to disk

  • ⚠️ Long-term memory leak → Automatically restart Kernel every 50 rounds

  • ⚠️ Concurrency conflicts → Single session serialization

Practice proves:In the CTF scenario, risks are controllable and benefits are obvious

Agent layer: minimalist logic

Code size:

  • The code of Agent itself is only100-200 lines

  • Called "Baby Runtime"

  • The core logic is extremely simple

Purely autonomous drive:

  • There is no preset active scanning SOP

  • Completely rely on LLM to make independent decisions after reading environment documents.

Analysis of core advantages

1. Extremely high flexibility (Infinite Flexibility via Code)

passexecute_code, the Agent is not limited to the predefined toolset:

  • Strong ability to deal with unknown and complex scenarios

  • Not limited by toolset

  • Infinite expansion possibilities

2. True state retention (Stateful Interaction)

Utilize the features of Jupyter Kernel to achieve natural contextual memory:

  • More efficient and more consistent with programming logic

  • Variables stay in memory

3. Generalization with zero tuning

The system does not perform specific fine-tuning for CTF questions:

  • Only give the Agent an "Instruction Manual" (more than 200 lines of prompt)

  • Let AI think and try like human researchers

  • Demonstrated operational potential on unknown targets

4. Clear Observability

Combining VNC visualization and CAIDO traffic proxy:

  • Solve the black box problem of "What is AI doing?"

  • Humans can intervene or review at any time

5. Decoupled Architecture

Through the MCP protocol, the Agent (brain) and Sandbox (hands and feet) are completely decoupled:

  • You can upgrade your brain or hands and feet at will without affecting each other.

Cost and effectiveness analysis

Average cost:19.2 yuan/question

Cost analysis:

  • It is a medium to high level among all teams.

  • Mainly because using Kimi consumes a lot of money

  • But the cost will be significantly reduced after switching to DeepSeek

Ranking analysis:Fourth place (NO.4)

Core values:It proves the feasibility of "zero tuning + minimalist architecture" and provides an important reference for general AI Agent.

Lessons to learn from

  1. Meta-Tooling Pattern⭐⭐⭐⭐⭐ - Innovative way to call tools, greatly improving flexibility

  1. Jupyter Kernel state management⭐⭐⭐⭐⭐ - An elegant state retention solution that solves a major pain point for Agents

  1. minimalist design concept⭐⭐⭐⭐ - "Simple is best", avoid over-engineering

  1. human visualization⭐⭐⭐⭐ - VNC+ traffic proxy solves AI black box problem


NeuroSploit (Tsinghua University) - 6th place

team:Tsinghua/Southeast/National University of Defense Technology Joint TeamCore features:Hierarchical multi-agent | Dual engines of knowledge and weapons | Self-evolution capability | Expert-level depth

Team background

It is composed of security teams from three top universities:

  • Tsinghua University- Core algorithm and architecture design

  • Southeast University- Engineering implementation and optimization

  • National University of Defense Technology- Expertise in the security field

Complementary advantages:

  • Tsinghua’s AI technology accumulation

  • Southeast’s engineering capabilities

  • National University of Science and Technology’s security professional background

Architecture design

Image

NeuroSploit - Hierarchical multi-agent architecture

Image

Generate pictures - Hierarchical multi-agent architecture (Chinese)

Architecture 01: Hierarchical Agents

"Command + Special Forces" mode:

Lead Agent (Command)

  • Coordinate the overall situation

  • Analyze the target environment and formulate attack strategies

  • Assign tasks to Sub-Agents and coordinate the work of each Sub-Agent

Sub-Agents (Special Forces)

  • Recon Sub-Agent(Scout) - Responsible for information collection, port scanning, directory enumeration

  • Exploit Sub-Agent(Commando) - Responsible for exploiting vulnerabilities, executing attack code, and obtaining Shell

  • Specialize in Sub-Agents(Expert soldier)

Collaboration process:

1. Lead Agent分析: "这个网站看起来是PHP开发的"
2. 指派Recon: "侦察兵,去扫描这个网站"
3. Recon回报: "发现SSTI漏洞特征"
4. Lead决策: "SSTI Sub-Agent,你去搞定它"
5. SSTI Sub-Agent: "收到,开始构造Payload..." "成功获取Shell!"
6. Lead评估: "很好,继续提权..."

Architecture 02: Dual Engines of Knowledge and Arms

Image

NeuroSploit - Knowledge Arsenal Architecture

Image

Generate pictures - Knowledge Arsenal System Architecture (Chinese)

Engine 1: OWASP Knowledge Base

  • Role: Guiding tactics

  • Content: Security knowledge, attack methods, vulnerability principles

  • Format: Structured security knowledge

Engine 2: Payload Arsenal

  • Function: Provide ammunition

  • Content: 50+ ready-made Payload templates

  • Format: Ready-to-use attack code

How it works:

传统Agent:
发现漏洞 → 思考怎么利用 → 编写Payload
可能不熟悉该漏洞 → 编写的Payload质量不高

NeuroSploit:
发现漏洞 → 查询知识库 → 了解原理
↓
从军火库选择Payload模板
↓
动态生成攻击代码
↓
高质量利用

Architecture 03: MCP Tool Matrix

Supported tools:

  • Nuclei - Vulnerability Scanning

  • Xray - passive scan

  • Playwright - Browser Automation

  • Nmap - port scanning

  • Sqlmap - SQL injection

MCP packaging (standardized):

# Agent调用
result = agent.call_skill("port_scan", {
    "target": "example.com",
    "scan_type": "sV",
    "version_detection": True
})

Advantages:

  • Parametric control

  • unified interface

  • Easy to maintain

  • Agent is easier to understand

Architecture 04: State and Memory Closed Loop (Loop Management)

Apache Burr status flow:

[开始] → [侦察] → [分析] → [决策]
                    ↓         ↓
                 [失败] ← [利用]
                    ↓
                 [重试] ────┘
                    ↓
                 [成功] → [提权] → [完成]

ReMem Agent:Store success/failure experiences to form a closed loop.

Memory mechanism:

# 成功经验
success_memory = {
    "vulnerability": "SQL注入",
    "parameter": "id",
    "payload": "' OR 1=1--",
    "target_type": "PHP网站",
    "success_rate": 0.85
}

失败教训

failure_memory = {
"vulnerability": "XSS",
"payload": "<script>alert(1)</script>",
"filter": "WAF拦截",
"lesson": "需要编码绕过"
}

Workflow:

第一次攻击:
尝试XSS → 失败 → ReMem记录失败原因

第二次攻击:
遇到XSS → ReMem提醒上次失败
↓
尝试编码绕过 → 成功 → ReMem记录成功经验

第三次攻击:
遇到类似XSS → ReMem直接提供成功方案
↓
快速成功!

Core advantages

1. Possess the ability of "self-evolution" ⭐⭐⭐⭐⭐

The biggest highlight!Through the Remem mechanism, the system can "remember" successful attack patterns.

Evolution process:

初始状态:
面对SQL注入 → 盲目尝试 → 效率低
    ↓ 学习积累
第10次任务:
记住参数特征 → 优先测试 → 效率提升
    ↓ 经验复用
第50次任务:
快速识别模式 → 直接应用经验 → 效率很高
    ↓ 举一反三
第100次任务:
遇到新目标 → 类比经验 → 创新攻击

value:As the tasks increase, the system becomes stronger with more use!

2. Deep Specialization ⭐⭐⭐⭐⭐

Reject the “snake oil” agent

Equipped with proprietary models/prompts for different vulnerability types, the attack depth far exceeds that of general-purpose agents.

3. Reasoned Attacks ⭐⭐⭐⭐

Relying on the built-in arsenal and CheatSheet, it simulates the thinking path of a human hacker "View documents -> Construct Payload".

4. White box controllability ⭐⭐⭐⭐

Monitor each step of the Agent's decision-making path through a visual interface to ensure that the logic does not deviate.

Ranking analysis

Final ranking:Sixth place (NO.6)

Possible reasons:

  1. High architectural complexity- Theoretically very advanced, but engineering implementation is difficult and various bugs may occur

  1. High startup costs- You need to accumulate enough experience to be strong, and the game time is limited

  1. limitations of specialization- Strong on known vulnerability types, but may not be as good as general solutions when encountering new types

Lessons to learn from

  1. Hierarchical multi-agent architecture⭐⭐⭐⭐⭐ - Clear division of labor and collaboration, professional division of labor improves efficiency

  1. Knowledge Arsenal⭐⭐⭐⭐⭐ - Payload template management, knowledge base systemization

  1. ReMem self-evolution mechanism⭐⭐⭐⭐⭐ - Experience accumulation and reuse, the more you use the system, the stronger it becomes

  1. Visual monitoring⭐⭐⭐⭐ - Solve the AI ​​black box problem and improve credibility


Green League - 7th place

Core features:Lightweight bee colony architecture | Shared notebook | Three major correction mechanisms | Heterogeneous model combination

Architectural Breakthrough: Swarm Parallelism and Silent Collaboration

Image

Generate pictures - Swarm Parallel Architecture (Chinese)

Core concept: Lightweight Swarm architecture (Lightweight Swarm)

Aiming at the performance bottleneck of traditional centralized architecture in multi-objective scenarios.

Detailed explanation of architectural features

1. Multi-Agent single-soldier combat

  • Each Agent operates independently

  • Does not rely on centralized coordination

  • Avoid single points of failure and improve overall fault tolerance

2. Parallel detection and independent context

  • Each Agent has an independent context space

  • Can detect multiple targets simultaneously

  • Context does not interfere with each other

3. Non-blocking collaboration mode based on shared notebooks⭐Core Innovation

Problems with traditional solutions:

  • Direct dialogue between agents

  • Leading to high Token consumption

  • prone to context pollution

Shared notebook plan:

Agent A → 写入笔记本 → 异步共享
                          ↓
                    共享笔记空间
                          ↓
Agent B ← 读取笔记 ← 获取信息

Advantages:

  • ✅ Avoid high token consumption caused by direct dialogue

  • ✅ Break information silos

  • ✅ Achieve low-cost and efficient collaboration

  • ✅ Non-blocking communication

Engineering Tuning: Three Correction Mechanisms

LLM is prone to occur in actual combat: hallucinations, falling into an infinite loop, and going further and further down the wrong path.

Three core control mechanisms:

Correction mechanism 01: Preventing intelligence from falling

Image

NSFOCUS - Agent defense manager contextual intelligence reduction and rapid recovery mechanism

question:Long context leads to cognitive degradation

Solution:Status reset + memory recall

Implementation process:

长时间运行 → 上下文膨胀 → 认知能力下降
    ↓
【状态重置】清空当前上下文
    ↓
【记忆回注】重新注入关键信息
    ↓
恢复清醒状态 → 继续高效工作

Correction mechanism 02: Anti-addiction

question:Tool calls are stuck in an infinite loop

Solution:Hook intervenes and interrupts

Implementation mechanism:

  • Monitoring tool calling patterns

  • Identify repetitive calls

  • Hook mechanism forces interruption

  • Reevaluate strategy

Correction mechanism 03: Prevent dependence

question:wrong path dependency

Solution:Clear notes regularly

Design ideas:

  • Shared notes may contain incorrect information

  • Agent may rely too much on old experience

  • Need to "forget" regularly

Heterogeneous model combination strategy

Model division of labor:

  • Main model: DeepSeek- Routine tasks, cost-effective

  • Attack Model: Kimi- Difficult problems, better at solving difficult problems

Cost control:

  • Use DeepSeek first (cheaper)

  • Switch to Kimi when encountering difficulties (powerful)

  • Dynamically adjust according to the difficulty of the question

cost analysis

Cost of solving a single question:About 13 yuan

Value for money among all teams:

  • Green Alliance: 13 yuan/question← Excellent value for money

Sources of cost advantage:

  1. Shared notebooks reduce conversation consumption

  1. Correction mechanism reduces invalid attempts

  1. Heterogeneous models optimize resource allocation

  1. Lightweight architecture reduces overhead

Ranking analysis

Final ranking:No. 7 (NO.7)

Possible reasons:

  1. Architectural innovation but implementation is difficult- The concept of bee colony architecture is very good, but the engineering implementation complexity is high

  1. Excellent cost control but mediocre results- The cost of 13 yuan/question is excellent, but the number of solved questions may not be the largest.

  1. The double-edged sword of sharing notes- Reduced conversation consumption, but may also reduce some useful information exchange

Lessons to learn from

  1. Shared note mode⭐⭐⭐⭐⭐ - Innovative collaboration model worthy of further optimization

  1. Three major correction mechanisms⭐⭐⭐⭐⭐ - Systematic problem identification, practical solutions

  1. Heterogeneous model combination⭐⭐⭐⭐ - Balancing performance and cost, the actual effect is remarkable

  1. Lightweight design concept⭐⭐⭐⭐ - Avoid excessive complexity and maintain system flexibility


City University of Hong Kong - 8th place

Core features:Multi-level queue system | Dynamic prompt injection | Horse racing mechanism | ReAct loop

Image

Generate pictures - ReAct loop mode

Core pain point identification

Pain point 1: Instability of LLM

Large language models (LLM) are stochastic, and a single agent may:

  • stuck in an infinite loop

  • hallucination

  • "Getting to the point" of the wrong idea

Pain point 2: Difficulty in context tracking (Context Loss)

CTF challenges usually require multiple rounds of interaction. As the number of dialogue rounds increases, the model easily forgets key information.

Pain point 3: Asymmetry causes inefficiency

The questions vary in difficulty. If you get stuck on a difficult question for too long, you will waste valuable competition time.

Pain point 4: Waste of resources in tool calls

Multiple agents scan the same target at the same time, causing a huge waste of time and computing resources.

Scheduling Module

Image

City University of Hong Kong - Multi-level Queue Scheduling System

Multi-level queuing mechanism

Four-level parameter threshold queue:

第一队列 (Queue 1):
├─ 步数限制: 30步
├─ 温度参数: 默认
└─ 目标: 快速解决简单题

↓ (30步未解出)

第二队列 (Queue 2):
├─ 步数限制: 50步
├─ 温度参数: 稍高
├─ 增加提示: Hint Injection ✅
└─ 目标: 借助提示解决中等题

↓ (50步未解出)

第三队列 (Queue 3):
├─ 步数限制: 70步
├─ 温度参数: 更高
└─ 目标: 更多尝试解决难题

↓ (70步未解出)

第四队列 (Queue 4):
├─ 步数限制: 100步
├─ 温度参数: 最高
└─ 目标: 最后的尝试

Highlight design: Hint Injection ⭐

Image

City University of Hong Kong - Hint Injection Mechanism

Innovation points:When a question is "downgraded" from the first queue to the second queue, the scheduler will automatically obtain it from the competition platform"Hint"And inject it into the Agent's prompt word.

Workflow:

题目在Queue 1尝试30步 → 失败
    ↓
自动降级到Queue 2
    ↓
从平台获取Hint
    ↓
将Hint注入到Agent的Prompt
    ↓
Agent获得新线索,打破僵局

Example:

没有Hint:
Agent盲目尝试各种SQL注入

有Hint:
"提示:这个题目的密码在注释里"
↓
Agent查看HTML源码
↓
快速找到密码

value:

  • Breaking the thinking deadlock of AI

  • Provide key clues

  • Avoid unnecessary repeated attempts

Agent Racing

Design ideas:For a question, the system will assign an Agent Group (including 5 Agents) to solve the problem in parallel.

Differences between the 5 Agents:Has different model temperature (Temperature) settings to generate diverse problem-solving ideas

Temperature setting example:

Agent 1: Temperature = 0.1 → 保守、精确
Agent 2: Temperature = 0.5 → 平衡
Agent 3: Temperature = 0.7 → 创造性
Agent 4: Temperature = 0.9 → 大胆尝试
Agent 5: Temperature = 1.2 → 非常激进

Workflow:

题目分配 → 5个Agent同时开始
    ↓
Agent 2率先解决!
    ↓
停止其他Agent
    ↓
使用Agent 2的答案

Ranking analysis

Final ranking:Eighth place (NO.8)

Advantages ✅

  1. Multi-level queue design- Reasonable resource allocation, Hint Injection is the highlight

  1. horse racing mechanism- Good ideas for parallel problem solving and diversified temperature settings

  1. shared memory- Avoid duplication of work and share information between agents

Disadvantages ⚠️

  1. Horse racing mechanism is expensive- 5 Agents are running at the same time, and Token consumption is huge.

  1. Queue downgrade can be a waste of time- 30 steps wasted in Queue 1, and 50 steps were wasted in Queue 2

  1. It does not fundamentally solve the problem of Agent capability.- Temperature adjustment only changes the diversity and does not improve the real ability of the Agent.

Lessons to learn from

  1. Hint Injection⭐⭐⭐⭐⭐ - Very practical innovation that can be applied directly

  1. multi-level queue⭐⭐⭐⭐ - Good resource allocation ideas and can be optimized and adjusted

  1. Persistent shared memory⭐⭐⭐⭐⭐ - Avoid duplication of work, Agent solves problems in relay

  1. horse racing mechanism⭐⭐⭐ - Parallel idea is good, but cost considerations


yhy0 - 9th place

Core features:Consultant intervention mechanism | Minimalist tool design | Cover-up strategy | On-demand knowledge loading

Core innovation: consultant intervention mechanism

Image

yhy0 - Consultant intervention mechanism process

design concept

Simulate the daily habits of using LLM:

Never blindly believe in LLM’s answers

yhy0 applies this idea to the Agent system:

  • Introducing “Consultant Agent”

  • Intervene at critical moments

  • Provide professional advice

Four triggering mechanisms

Trigger 1: When the mission starts

opportunity:Agent just received a new task

Consultant role:Provide initial suggestions to avoid blind attempts

Trigger 2: When continuous failures occur

opportunity:When failed 3 times, 6 times, 9 times

Consultant role:Reevaluate strategies and think outside the box

Step-by-step intervention:

  • 3 failures: mild reminder

  • 6 failures: Moderate recommendation

  • 9 failures: strong intervention

Trigger 3: Regular consultation

opportunity:Regular consultation every 5 attempts

Consultant role:Prevent the attacker from falling into a fixed mindset

Trigger 4: Actively ask for help

opportunity:LLM realized on its own that it was stuck

Consultant role:Provide critical assistance

design features

1. Consultant Agent design

Core philosophy:Simulate the way human experts think

Features:

  • Never blindly believe in LLM- critical thinking

  • Minimalist tool design- Use Kali tools directly

  • Give the initiative to LLM- Let it plan itself

  • Digging strategy- Save multiple questions at critical moments

2. Minimalist tool design

Design ideas:Give up complex encapsulation and directly let LLM use Kali tools

3. Give the initiative to LLM

Design concept:Let the AI ​​plan its own attack steps

Current issues

The prompts of the main agent are too mixed.

The question is mixed:

  • problem solving strategies

  • Python specification

  • Docker usage

  • Vulnerability knowledge

Causes problems:

  • Cognitive overload

  • Python code format error occurs

  • Tool misuse

  • Misjudgment of vulnerabilities

Summarize:

"Sprint strategy, high consumption, poor effect"

actual results

Statistics:

  • Solved questions: 73

  • Total consumption: 1,000 yuan

  • Average question: 13.7 yuan

  • Models used: All DeepSeek

cost analysis

Across all teams:

  • Cost: 13.7 yuan/question (mid-level)

  • Slightly higher than Green Alliance (13 yuan)

  • Lower than Antix (19.2 yuan)

  • Good value for money

future plans

The main agent is only responsible for planning

Improvement ideas:

当前:
主Agent = 规划 + 执行 + Python + Docker + 漏洞知识
    ↓
Prompt混杂,认知负担重

未来:
主Agent = 只负责规划
↓
执行交给:

  • PoC Agent(负责执行)

  • Docker Agent(负责容器)

  • 其他专业Agent
    ↓
    Prompt清爽,效率提升

Load knowledge on demand

Refer to Claude Code’s Skills design:

Example:

发现SQL注入
  ↓
主Agent: "需要SQL注入知识"
  ↓
顾问: 加载SQL注入知识库
  ↓
提供给主Agent使用

Lessons to learn from

  1. Consultant intervention mechanism⭐⭐⭐⭐⭐ - Innovative cover-up strategies that can be directly applied to other systems

  1. Minimalist tool design⭐⭐⭐⭐ - Avoid over-encapsulation and reduce contextual interference

  1. On-demand knowledge loading⭐⭐⭐⭐⭐ - Refer to Claude Code Skills to avoid prompt explosion

  1. Division of labor among sub-Agents⭐⭐⭐⭐ - Separation of planning and execution, reducing cognitive load


core glossary

To ensure the professionalism of the document, the core technical terms used in this document are defined as follows:

the term

full name

definition

source

MPI

Model Programming Interface

Model programming interface, using codes instead of prompt words to control AI models

Changtingwai team customization

MCP

Model Context Protocol

Model context protocol, an AI tool standardization protocol proposed by Anthropic

Anthropic (2024)

DSPy

Declarative Programming for LLMs

Programmatic LLM interaction framework developed by Stanford NLP Group

Stanford NLP Group

Meta-Tooling

meta-tool pattern

Instead of calling the tool directly, let AI write code to call the tool's design pattern.

Antix team

APG

Agent Pattern Graph

Agent intent map, declarative attack flow chart defined in YAML

BinX Team

ReAct

Reasoning + Acting

Reason-Action Cycle Pattern

Stanford (2022)

ReMem

Remember Memory

Memory recall mechanism, a system for storing and reusing attack experience

NeuroSploit Team

SOP

Standard Operating Procedure

standard operating procedures

general term

CTF

Capture The Flag

Capture the Flag, a cybersecurity competition format

general term

RAG

Retrieval-Augmented Generation

Retrieval enhancement generation, combined with knowledge base generation method

general term

Important note:The definition of MPI in this document is different from MPI (Message Passing Interface, Message Passing Interface) in traditional parallel computing.


In-depth comparison of technical routes

Image

Generate pictures - Comparison of three Agent architectures

Route 1: Prompt Engineering (prompt word engineering) ⚠️

Representative team:Early plan, no one adopted the pure Prompt plan in this competition

Image

Generate pictures - Prompt engineering vs code planning comparison

limitation:

  • ⚠️ Stability depends on model capabilities

  • ⚠️ Complex logic is difficult to express accurately

  • ⚠️ High maintenance costs

  • ⚠️ Long context performance degradation

in conclusion:

It has been surpassed by code planning in complex tasks, but it is still effective in simple scenarios.


Route 2: Code Planning ✅

Representative team:

  • 🥇 Outside Changting (MPI/DSPy)

  • 🥈 Xi'an Jiaotong University (XML protocol)

  • 🥉 BinX (meta-tool orchestration)

Core philosophy:

Replace Prompt with code/protocol for planning and control

Advantages:

  • ✅ Accuracy

  • ✅ Stability

  • ✅ Maintainability

  • ✅ Scalability

in conclusion:

Code planning is a clear mainstream direction


Route 3: Meta-Tooling (meta-tool mode) ✅

Representative team:Antix (Chengxin University)

Core philosophy:

Instead of calling tools, let AI write code to call tools

Implementation method:

# 唯一接口
execute_code(python_script)

Advantages:

  • ✅ Extremely flexible

  • ✅ Unlimited scalability

  • ✅ State preservation (Jupyter)

  • ✅ Minimalist architecture

Ranking:4th place

in conclusion:

The embodiment of minimalist value and strong versatility


Route 4: Hierarchical Multi-Agent ✅

Representative team:

  • NeuroSploit (Tsinghua)

  • BinX (three-person team)

  • yhy0 (planning and execution separation)

Core philosophy:

Division of labor and collaboration, professional division of labor

Advantages:

  • ✅ Professional division of labor

  • ✅ Improve efficiency

  • ✅ Reduce the complexity of a single Agent

in conclusion:

Multi-Agent is the mainstream direction, but the implementation methods are different.


Route 5: Swarm Intelligence ✅

Representative team:Green Alliance

Core philosophy:

Multiple lightweight agents collaborate in parallel and share notes

Implementation method:

  • Multi-Agent single-soldier combat

  • independent context

  • Shared notebook (asynchronous communication)

Advantages:

  • ✅ High concurrency

  • ✅ Low cost (13 yuan/question)

  • ✅Information sharing

Ranking:7th place

in conclusion:

Lightweight bee swarm is a cost-effective solution


Route 6: Distributed Architecture ✅

Representative team:Xi'an Jiaotong University

Core philosophy:

Master-Worker mode, task management decoupling

Implementation method:

  • Master node: task scheduling

  • Worker node: perform penetration

  • XML protocol: standard communication

Advantages:

  • ✅ Horizontal expansion

  • ✅ High availability

  • ✅ Load balancing

Ranking:No. 2 🥈

in conclusion:

Essential architecture for enterprise-level applications


Panoramic comparison table of technical solutions of the nine major teams

Comprehensive comparison table (based on original XBow Notes extension)

team

Ranking

Core Perspectives on Automation

Method architecture

Key technologies/features

advantage

Disadvantages/Challenges

cost

outside the pavilion

🥇1

Use precise planning with code to replace unstable prompts

DSPy/MPI

- Code planning - DSPy framework - Execution suspension mechanism

- Precise control - Stable and reliable - Easy to maintain

- Requires programming basics - Higher cost

30 yuan/question

Xi'an Jiaotong University

🥈2

Distributed + standardized process

Master-Worker + XML

- Distributed cluster - XML ​​strongly typed protocol - Three-stage pipeline

- Horizontal expansion - High availability - Type safety

- High complexity - Insufficient flexibility

N/A

BinX

🥉3

Solving AI perception state distortion and low efficiency

Three-person team structure

- Sandbox cleaning - Meta-tool arrangement - Intent map

- Systematic methodology - Noise processing

- High implementation complexity

N/A

Antix

4

Minimalist design, giving full play to model capabilities

Meta-Tooling

- Jupyter Kernel - single interface - zero tuning

- Extremely high flexibility - State retention - High versatility

- Depend on model capabilities

19.2 yuan/question

(fifth place)

5

missing information

missing information

missing information

missing information

missing information

N/A

NeuroSploit

6

External knowledge base makes up for model deficiencies

Multi-Agent + MCP

- Knowledge arsenal - ReMem self-evolution - Hierarchical Agent

- Self-evolution ability - Expert level depth

- Complex architecture - High startup costs

N/A

Green Alliance

7

Multi-agent collaboration and information sharing

Swarm Parallel

- Shared notebook- three major correction mechanisms

- High concurrency - Low cost - Information integration

- Difficulty of engineering implementation

13 yuan/question

City University of Hong Kong

8

Handle tasks of different difficulty in different levels

Multi-level queue + ReAct

- Hint Injection- Horse Racing Mechanism- Shared Memory

- Reasonable allocation of resources - Break the deadlock

- The cost of horse racing is high

N/A

yhy0

9

Consultants step in to provide cover strategies

Consultant intervention mechanism

- Four trigger mechanisms - on-demand knowledge loading

- Innovative cover-up strategy - High practical value

- Prompt mixed

13.7 yuan/question

Summary of core insights (comprehensive original notes)

1. Technical path distinction: Prompting vs Programming

Main differences:

  • Prompting (prompt word project): Control AI through carefully designed prompt words (has been eliminated)

  • Programming (code planning): Explicitly define business logic with code/protocol (main trend)

Practice proves:

  • Changtingwai, Xi'an Jiaotong University and other front-line teams all adopt code planning

  • Code planning surpasses prompt word engineering in terms of accuracy, stability, and maintainability.

2. Model capability is the decisive factor

Key findings:

  • All successful solutions rely on a strong underlying LLM

  • DeepSeek: Best price/performance ratio, suitable for routine tasks

  • Kimi: There are advantages in solving difficult problems

  • Claude Code: Excellent at code generation

in conclusion:

The upper limit of AI penetration is determined by the underlying model capabilities, and the architectural design determines the extent to which the model capabilities can be utilized.

3. Balance between pragmatism and forward-looking

Two extremes:

  • pragmatic team(Such as Antix): Use the simplest method to solve the problem within the capabilities of the existing model

  • Qianzhan team(such as Changtingwai, NeuroSploit): Explore and deal with complex, non-convergent penetration spaces in the real world

Best practices:Blending the two, finding a balance between engineering and innovation


Comparison of core problem solutions

Problem 1: Context pollution and token waste

Image

Generating images - context management strategies compared

team

solution

Performance rating

innovativeness

Practicality

BinX

Sandbox cleaning

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

Green Alliance

Share notes

⭐⭐⭐⭐

⭐⭐⭐⭐

⭐⭐⭐⭐

Antix

Jupyter status

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

Best solution:

  • BinX has the most thorough sandbox cleaning

  • Antix's Jupyter state is most elegant

  • The best combination of the two


Problem 2: Agent logic goes awry

team

solution

Performance rating

Systematic

Practicality

BinX

intent map

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

Green Alliance

Three major correction mechanisms

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

NeuroSploit

Apache Burr

⭐⭐⭐⭐

⭐⭐⭐⭐

⭐⭐⭐

yhy0

Consultants intervene

⭐⭐⭐⭐

⭐⭐⭐⭐

⭐⭐⭐⭐

Best solution:

  • BinX’s intent map is the most systematic

  • Green Alliance’s correction mechanism is the most practical

  • yhy0’s advisory mechanism is the most innovative


Problem 3: Inefficient tool calling

team

solution

Performance rating

Simplicity

flexibility

Antix

Meta-Tooling

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

BinX

meta-tool orchestration

⭐⭐⭐⭐⭐

⭐⭐⭐⭐

⭐⭐⭐⭐⭐

NeuroSploit

MCP tool matrix

⭐⭐⭐⭐

⭐⭐⭐

⭐⭐⭐⭐

outside the pavilion

code call

⭐⭐⭐⭐⭐

⭐⭐⭐⭐

⭐⭐⭐⭐

Best solution:

  • Antix’s single interface is the simplest

  • The code call outside Changting is the most accurate

  • All worth learning


Problem 4: Experience cannot be accumulated

team

solution

Performance rating

integrity

uniqueness

NeuroSploit

ReMem self-evolves

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

BinX

review layer

⭐⭐⭐⭐

⭐⭐⭐⭐

⭐⭐⭐

Hong Kong City University

Persistent shared memory

⭐⭐⭐⭐

⭐⭐⭐

⭐⭐⭐

Best solution:

  • NeuroSploit’s Remem is the most complete

  • The only truly "self-evolving" solution


Problem 5: Difficulty maintaining status

team

solution

Performance rating

elegance

efficiency

Antix

Jupyter Kernel

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

⭐⭐⭐⭐⭐

Hong Kong City University

Persistent shared memory

⭐⭐⭐⭐

⭐⭐⭐

⭐⭐⭐⭐

Best solution:

  • Antix’s Jupyter Kernel is a best practice

  • Maintain natural state, elegant and efficient


Comprehensive analysis and summary

📊 1. Core conclusions

Conclusion 1: AI penetration has evolved into “automated programming and execution systems”

Key insights:

The actual AI penetration in 2025 has evolved from the category of "chat robot" to "automated programming and execution system based on MCP protocol"

evidence:

  • ✅ Outside Changting: MPI (Model Programming Interface)

  • ✅ Antix: Meta-Tooling + Python execution

  • ✅ BinX:Python Meta-Tooling

  • ✅ NeuroSploit: MCP Tool Matrix

  • ✅XJJT: XML protocol + encoding control

Common points:All in usecode/protocolsubstitutePure prompt


Conclusion 2: The key to victory - balance "refuse to over-encapsulate" and "structured thinking"

Two extreme balances:

Like Changting: planning the path with rigorous engineering code

Advantages:

  • Precise control

  • Stable and reliable

  • Easy to maintain

  • Suitable for engineering

represent:Changting University (champion), Xi'an Jiaotong University (runner-up)

Like Antix: Give AI the freedom to directly operate the Shell

Advantages:

  • Extremely flexible

  • Infinite scalability

  • Zero tuning for generality

  • Coping with unknown scenarios

represent:Antix (4th place)

Best Practice: Blend the Two

Recommended architecture:

核心层: 代码规划(长亭思路)
  ↓
执行层: Meta-Tooling(Antix思路)
  ↓
协作层: 多Agent分工(BinX/NeuroSploit思路)
  ↓
优化层: 沙箱清洗 + 上下文管理(绿盟思路)

Conclusion 4: The ability of the underlying model determines the upper limit

Key findings:

Comparison of model capabilities:

Image

Generating pictures - Comparison of LLM model capabilities

Model

Advantages

Disadvantages

Applicable scenarios

Team use

DeepSeek

Cost-effective and fast

Slightly weaker in complex reasoning ability

Routine tasks, cost sensitive

NSFOCUS, yhy0

Kimi

Strong ability to tackle difficult problems and long context

higher cost

Difficult questions, long texts

Antix

Claude

Excellent code generation and strong reasoning

High cost

Code planning, complex tasks

outside the pavilion

Practice proves:

  • Use Claude outside the Champion Pavilion for best results

  • Antix uses Kimi to perform outstandingly in solving difficult problems

  • NSFOCUS and yhy0 all use DeepSeek to achieve high cost performance

in conclusion:

The upper limit of AI penetration is determined by the capabilities of the underlying model
Architectural design determines the extent to which model capabilities can be utilized
Different models are suitable for different scenarios and need to be selected according to the characteristics of the task.


Conclusion 3: CTF scenario vs actual combat scenario

For single-point CTF competition and clear penetration goals:

status quo:

✅ The most cutting-edge AI and Agent frameworks are capable of solving most problems
✅ Unresolved issues can be resolved by increasing capability coverage and knowledge coverage.

evidence:

  • Changtingwai, Xijiao, BinX, etc. can solve 70+ questions

  • Very high success rate on known vulnerability types

  • The cost can be controlled within 20 yuan/question


But for actual AI penetration empowerment:

challenge:

⚠️ Still need to overcome many difficulties
⚠️ Engineering is the main goal to be overcome in the future

Engineering challenge list:

  • Task scheduling

  • session isolation

  • Concurrency control

  • False negative/false positive processing

  • Missing test

  • Recall optimization

  • SaaS productization

  • Multi-tenant management

  • Permission control

  • Audit log


🏆 2. Team ranking and analysis

Full ranking list

Ranking

team

core program

average cost/question

Total investment estimate

Key features

1

outside the pavilion

DSPy/MPI

30 yuan

30,000 yuan

Code planning, precise control

2

Xi'an Jiaotong University

Distributed cluster + XML

N/A

N/A

horizontal expansion, standardization

3

Guangzhou University(BinX)

Sandbox Cleaning + Meta Tools

N/A

N/A

Three-person team, map of intentions

4

Antix(Chengxinda)

Meta-Tooling

19.2 yuan

N/A

Minimalist architecture, code is action

5

(fifth place)

missing information

N/A

N/A

missing information

6

NeuroSploit (Tsinghua University, etc.)

Multi-Agent + knowledge arsenal

N/A

N/A

Self-evolving, expert-level depth

7

Green Alliance

Swarm Parallel

13 yuan

N/A

Lightweight, share notes

8

City University of Hong Kong

Multi-level queue + horse racing

N/A

N/A

Hint injection, parallel

9

yhy0

Consultants intervene

13.7 yuan

1,000 yuan

Back-to-back strategies, on-demand knowledge

Cost data description:N/A means the data are not published or cannot be calculated. Xi'an Jiaotong University's "about 1,000+ yuan/day" is the cost of the training phase and does not represent the cost of a single question.


Ranking pattern analysis

What champions and runners-up have in common

Outside Changting + Xi'an Jiaotong University:

  • ✅ They all emphasize that "code/protocol" is better than Prompt

  • ✅ They all have a complete engineering framework

  • ✅ They are all a team of Ph.D.s with profound skills

  • ✅ There are clear standardized processes

in conclusion:

Code planning + engineering = champion status


What the top four have in common

Changtingwai, Xijiao, BinX, Antix:

  • ✅ Everyone is using “code” instead of “Prompt”

  • ✅ There are clear technical routes

  • ✅ All solve the core pain points

in conclusion:

Coding is the mainstream trend


The relationship between cost and ranking

team

Cost (yuan/question)

Ranking

Green Alliance

13.0

7

yhy0

13.7

9

Antix

19.2

4

outside the pavilion

~30

1 🏆

Insight:

  • Lowest cost ≠ Highest ranking

  • Champions are willing to invest in results

  • Cost-effectiveness and effectiveness need to be balanced


🚀 3. Future trends and suggestions

Image

Generate Pictures - AI Penetration Testing Technology Evolution Roadmap (Chinese)

Trend 1: From Prompt to Code (verified)✅

in conclusion:

✅ Code planning is a clear trend, not an experiment

Suggestions for action:

  1. Learn the DSPy framework

  1. Study MPI mode

  1. Practice code generation


Trend 2: From monolithic to layered (verified)✅

in conclusion:

✅ Multi-Agent division of labor is the mainstream, single-Agent has been eliminated

Suggestions for action:

  1. Design a clear division of labor for Agents

  1. Establish standardized communication protocols

  1. Implement visual monitoring


Trend 3: From stateless to stateful (in progress) ⚠️

in conclusion:

⚠️ Jupyter Kernel is best practice, but not widespread

Suggestions for action:

  1. Using Jupyter Kernel

  1. Implement persistent memory

  1. Establish Remem mechanism


Trend 4: From black box to white box (ongoing) ⚠️

in conclusion:

⚠️ Observability is a must for engineering

Suggestions for action:

  1. Implement visual monitoring

  1. Create audit log

  1. Support human intervention


Trend 5: From experiment to product (to be broken through) ❌

in conclusion:

❌ Engineering is the biggest challenge and has not been broken through yet

Suggestions for action:

  1. Solve task scheduling

  1. Implement session isolation

  1. Optimize concurrency control

  1. Handling false negatives and false positives

  1. Improve recall rate

  1. Implement SaaS



🎯 5. Final evaluation

Contest value

1. Verified the technical route

  • Code Planning > Prompt Project

  • Multiple Agents > Single Agent

  • Engineering > Conceptualization

2. Demonstrates innovative direction

  • Meta-Tooling(Antix)

  • Intent Map (BinX)

  • ReMem self-evolves (NeuroSploit)

  • Consultant intervention (yhy0)

3. Identification of engineering challenges

  • CTF scenario is basically solved

  • Practical engineering needs breakthrough

  • SaaS productization is the future


Acknowledgments:

Thank you to all participating teams for your wonderful sharing. Your innovation has promoted the development of AI penetration testing technology!

  • outside the pavilion- Proved the value of code planning

  • Xi'an Jiaotong University- Demonstrates the power of distributed architecture

  • BinX- Systematic methodology

  • Antix- The wisdom of minimalism

  • NeuroSploit- Self-evolving innovation

  • Green Alliance- Practice of bee colony architecture

  • City University of Hong Kong- The ingenuity of Hint Injection

  • yhy0- The advisory mechanism’s cover-up strategy

  • All teams- Jointly promote technological progress


THANKS ALL TEAMS!

I0veD

I0veD

cyber security researcher

Cloud Native & AI Sec Researcher Red Team | BAS | K8s | Evasion 20+ CVEs | CNVD/CNNVD Contributor 🛡️ AI-Driven Blue Team 👇 Works

Comments (0)

Login to post a comment.