How can people retain effective control over hierarchical VLA robots during long-running physical tasks without continuously supervising them?
MIT Media Lab Research Portfolio
Chenchen
Zheng.
I build intelligent systems whose interpretations stay visible and whose actions remain open to human judgment.
HCI · HRI · Interactive AI · Robotics
Intent travels. A system responds. Judgment returns.
About me / Researcher identity
Build systems.
Keep people in the loop.

I’m Chenchen Zheng, a UW–Madison undergraduate in Computer Science and Mathematics and a research assistant in the People and Robots Lab. My work spans HCI, HRI, robotics, and interactive AI, bringing system building and interaction design into questions of human understanding and control.
Legible autonomy. Before a system acts on human intent or context, people should be able to inspect its interpretation, correct it, and decide when action is justified.
What should a system decide—and when should a decision return to the person?
- UW–MadisonUndergraduate
- CS + MathematicsTechnical foundation
- People and Robots LabResearch Assistant
- Art Studio + PhotographyMaking system feedback perceptible
Research lenses · MIT Media Lab application
One body of work.
Three ways to read it.
Across robotics, interactive AI, evaluation, and creative computing, I make interpretation and human control part of the system design.
Choose a lens to reframe the portfolio.
Hover or focus to preview · Click to keepTap a lens to keep
The work stays the same. A lens makes its strongest connections visible.
How can users inspect and refine a robot’s plan before it acts?
When should an intelligent system respond, how strongly should it respond, and when should it remain quiet?
How reliable are automated jailbreak evaluators compared with human judgment?
Prepare the manuscript. Keep judgment visible.
Context · evidence · representation
Selected work / 01–05
The loop, in practice.
Five approaches to interpretation, authority, and human judgment.
01 / Honors Thesis · HRI / HCI × EMBODIED AI / VLA
Visual lens / All
Remote Intervention for VLA Robots
Robot stack implemented · intervention research ongoingHow can people retain effective control over hierarchical VLA robots during long-running physical tasks without continuously supervising them?
I built a real-robot VLA manipulation stack and am developing remote, asynchronous intervention around it—connecting embodied AI capability with lightweight human control and recovery.
Return selectively.Research concept · interface developing
Most of the interaction happens apart; control reconnects only when needed.
What I built
A working embodied-AI testbed.
Wrist viewLocal manipulation
External viewGlobal scene context- 01 / Collect
SpaceMouse + D435
2 RealSense views · wrist + external
86 valid trajectories · aligned dataset - 02 / Train
π0.5-DROID
30,000 steps · CHTC
Checkpoint evaluation - 03 / Serve
OpenPI policy server
Desktop inference · synced views / reconnects
- 04 / Control
Polymetis / libfranka
Robot-state feedback
- 05 / Execute
Panda + Robotiq
1 real robot · 7-DoF
Language → target → grasp → lift
Real language-conditioned grasp/lift demonstrates technical feasibility, not robust-policy performance or a completed thesis study.
Dataset & deployment details
The two camera streams pair local manipulation context with a global scene view. I aligned images with robot state, converted the recordings, and evaluated checkpoints before deployment.
The locally documented acquisition run used checkpoint 15,000; fine-tuning reached 30,000 steps. One real grasp/lift establishes feasibility, while repeated attempts and an invalid start-state trial prevent a robustness claim.
Current placement code uses an operator-calibrated fixed target and fake-tested motion stages. Adaptive release-height calculation and physical placement validation remain in progress.
Hybrid execution
Keep adaptation. Make continuation predictable.
- 01 / Learned policy
π0.5
Perception-dependentPerception uncertainty
Object-dependent adaptation- Approach
- Grasp
- Lift
- 02 / Handoff condition
Handoff gate
Handoff logic implemented- Stable grasp
- Robot-state check
- Contact-height estimate
- 03 / Deterministic control
Structured placement
Geometry-dependentPrecise spatial execution
Predictable continuation- Transfer
- Release height*
- Place
Learned control handles adaptation. Deterministic control handles predictable geometry.
*Adaptive release-height calculation remains in development.
Selective human control
Return only the decision that needs a person.
Resolve missing information before the robot acts.The answer selects A before action; B remains unchosen.
Return to the interpretation, then change the plan.Correction edits the interpretation; the revised plan continues.
Repair execution locally, without full replanning.Verify the state, guide a local recovery step, then resume.
Human help bridges a physical capability boundary.Human help crosses the gap; robot execution resumes beyond it.
Different problems require different forms of human return.
Intervention needs, not experimental conditions.
Current boundary
Implemented capability. Open research.
Implemented / demonstrated
- 86 valid real-world trajectories · dual-view sensing.
- π0.5-DROID fine-tuned · 30k steps on CHTC.
- Distributed inference + real language-conditioned grasp/lift.
- Hybrid handoff + placement logic implemented · placement fake-tested only.
Ongoing / planned
- Hybrid refinement, adaptive release-height calculation, and physical placement validation.
- Remote state snapshots + lightweight intervention UI.
- Clarify / Revise / Recover / Assist integration.
- Within-subject study design + evaluation.
Designing non-blocking supervision with selective state snapshots and easy return to autonomy; interruption burden remains to be evaluated.
Planned evaluation
A planned within-subject evaluation will study remote asynchronous supervision under concurrent task demands. Potential dimensions include recovery cost, interruption burden, task resumption, intent alignment, and perceived control; the protocol and measures remain in development.
No completed user study or final protocol is claimed.
Research group connections
MIT connections.
Why this matters to Personal Robots
I connect real-robot VLA engineering with selective clarification, correction, recovery, and assistance—human control without continuous teleoperation.
Why this matters to Cyborg Psychology
This research asks how asynchronous supervision can preserve agency and attention, making the return of control timely and lightweight.
Why this matters to Multisensory Intelligence
I combine language, dual-view vision, and robot state for learned physical action. The research asks when uncertain state should return control to a person.
02 / 05 · Team research
Visual lens / All
SHRIMP
Iterative Refinement of Robot Task Plans
Accepted — ACM UIST 2026How can users inspect and refine a robot’s plan before it acts?
I integrated perception and robot interfaces for a team system that turns language intent into an inspectable plan before physical execution.
Executable is not the same as understandable.
Personal work / team research
My contributions to the team system
01
Perception + robot
RGB-D perception, camera calibration, object localization, and point-cloud integration for real-robot execution.
02
Interaction + system
Robot–interface development, editable-plan interaction design, integration, and testing.
03
User study + research
Study setup, sessions, data collection, and collaborative manuscript writing and revision.
Team research: the architecture and study findings belong to the team.
How inspection works / team workflow
Inspect. Revise. Preview again.
- 01Ground the scene
- 02Language → plan
- 03Simulate / preview
- 04Inspect · revise ↔ preview
- 05Execute on the robot
Inspect or revise the simulated plan; the person authorizes robot execution.
Editor parameters & architecture
Team study · 35 participants · 3 conditions · 3 tasks
More control, with a workload tradeoff.
More control
More transparency
More effort
Plan+Edit increased perceived control and transparency over NoPlan, but required more workload than PlanOnly.
Published results & study details
The study compared Plan+Edit, PlanOnly, and NoPlan across Set the Table, Prepare a Snack, and Clean the Table. Each participant experienced all three interface conditions.
Usability and task-success differences were not significant across conditions.
Selected, unaltered panels from Figure 6 (p. 8). Bars show means; vertical lines show standard error. Brackets mark significant pairwise comparisons (* p < .05; ** p < .01). Higher control/transparency is better; lower workload is better.
I learned to treat inspectability and revision as part of robot autonomy, not as post-hoc explanation.
Research group connections
Personal Robots
I bring experience integrating embodied systems with interfaces for inspectable, negotiable autonomy.
Cyborg Psychology
Editable plans taught me to weigh perceived control against the effort of intervention.
Multisensory Intelligence
My perception and localization work helped ground language plans in the physical scene.
Research group connections
Why this matters to Personal Robots
I design initiative and permission boundaries for ambient agents, with visible override and return of control.
Why this matters to Cyborg Psychology
Adaptive assistance stays inspectable and correctable; I treat restraint, agency, and self-determination as interaction requirements.
Why this matters to Multisensory Intelligence
I translate speech, context, and history into structured decisions shared by digital and physical output paths.
04 / 05 · Sole-author research · EAI RAIDS 2026
Visual lens / All
LLM Safety
Exploring Jailbreak Attacks Against LLMs
Published / presentedHow reliable are automated jailbreak evaluators compared with human judgment?
I independently designed and ran an empirical LLM evaluation study, comparing automated scores with manual annotation to interrogate the evaluator itself.
Sole author
- Study design
- 725 curated prompts
- Evaluation
- Automated + human
- Analysis
- Category-level disagreement
- Publication
- EAI RAIDS 2026
Agreement at one scale.
Disagreement at another.
- Study design
- Curated and preserved prompt sets; designed experiments and generated model responses with a Python pipeline.
- Evaluation
- Integrated the existing StrongREJECT evaluator and a rubric condition; manually annotated responses.
- Analysis
- Compared automated scores with human labels; analyzed category-level disagreement.
- Publication
- Wrote the sole-author paper and presented the published research at EAI RAIDS 2026.
Method logic / 725 curated prompts
- Prompts
- Model responses
- Automated evaluation
- Human annotation
- Comparison
I compared two automated evaluator conditions with manual labels across 725 curated prompts, preserving prompt wording. 350 user-generated + 375 research-generated.
Evaluator conditions. Rubric · gpt-3.5-turbo / Fine-tuned · default StrongREJECT. Each condition regenerated responses across three runs; machine summaries use each prompt’s maximum score.
Human reference. 0 · Safe / refusal; 0.5 · Bypass signs without actionable harm; 1 · Actionable harmful output. Automated and human judgments are compared within each condition.
Aggregate agreement can hide disagreement.
Aggregate trends can coexist with category-level disagreement. Rubric means are higher than the human means in both datasets, while fine-tuned means are lower; automation does not universally underestimate severity. An evaluator’s scoring rule must be examined before treating its metric as neutral ground truth.
Comparison limit. Human labels are an internal reference, not a validated gold standard. Regenerated response sets and max-over-three aggregation limit comparison; these results do not isolate a causal effect of prompt structure.
| Source / evaluator | Human | Machine | n |
|---|---|---|---|
| UserFine-tuned | 0.297 | 0.215 | 350 |
| UserRubric | 0.330 | 0.419 | 350 |
| ResearchFine-tuned | 0.495 | 0.468 | 375 |
| ResearchRubric | 0.529 | 0.579 | 375 |
Score distributions & evaluation method
What I compared
Two automated evaluator conditions against human annotation across user- and research-generated prompts.
What I found
Agreement at one scale. Disagreement at another. Aggregate trends can obscure category-level disagreement; rubric means are higher than human means, while fine-tuned means are lower.
Why it matters
Automated evaluation is a measurement process, not neutral ground truth. Its scoring rule and interpretation still require human judgment.
- Evaluator provenance
- Default StrongREJECT and the original prompt datasets are prior work; the study also used a gpt-3.5-turbo rubric condition.
- Response generation
- Each evaluator condition regenerated responses over three runs. Machine summaries use each prompt’s highest observed score.
- User prompt sources
- JailJudge · DAN/jailbreak_llms · clean-jailbreak-prompts
- Research prompt sources
- GPTFuzz · JailbreakEval · JailbreakBench
- Source handling / bounds
- Deduplication, filtering, and format conversion preserved wording. Imbalanced content categories further limit comparison.
Evaluation is itself a judgment problem.
Research group connections
Cyborg Psychology
I bring independent empirical AI evaluation and a focus on where automated judgment diverges from human interpretation.
Follow-up · AI Magazine, under review; no unpublished findings shown.
05 / 05 · Independent project
Visual lens / All
Submission Studio
Completed systemPrepare the manuscript. Keep judgment visible.
I independently designed and developed an end-to-end system that assigns flexible transformation to an LLM, explicit checks to code, and consequential judgment to the author.
Different decisions need different intelligence.
Generate
LLM / flexible transformationVerify
Deterministic requirementsJudge
Ambiguous / consequential author decisions
What I built
Engine
Python conversion, validation, compile/log inspection, and bounded repair.
Review
Editing, PDF preview, and review separate blockers from author confirmations.
Orchestration
Web control plane for jobs and model dispatch; public operation remains unverified.
Repair without erasing decisions.
Inspect
LaTeX · BibTeX · assetsTransform
Venue profile + modelCompile + check
Logs · PDF · venue rulesReview
Blockers + confirmationsExport
Source + packet artifacts
Repair → recompile → recheck. Author edits return to validation before export.
Review captures & implementation scope
The CS Marketplace letter verifies independent design/development and the completed local workflow. Public web operation and formal user evaluation are not established.
The web control plane uses Next.js/React for uploads, job state, queues, and dispatch. External Python/TeX runner integration and public deployment remain unverified.
Completed system. Deliberate boundaries.
I intentionally separated generative flexibility, deterministic verification, and human judgment.
Completed / verified projectUW–Madison CS MarketplaceView completion letter ↗ (opens in a new tab)Research group connections
Cyborg Psychology
I design intelligent workflows that make delegation visible and preserve human judgment where ambiguity matters.
06 / Other Ways
Other ways I think.
Context, evidence, representation.
Three perspectives that extend the main work.
TrashBot
Context

Co-second author · qualitative field research
I coded and compared post-encounter interviews across New York City’s five boroughs, connecting local infrastructure and expectations to how people understand a public robot. My work combines qualitative coding, interview analysis, literature synthesis, cross-site interpretation, and manuscript development. This brings a context-sensitive perspective to embodied intelligence: a system’s meaning depends on the setting and the people encountering it. Unpublished findings remain private.
Public-space HRI taught me to interpret robot behavior through the social and spatial context in which people encounter it.
I examine how prior expectations, local norms, and environmental cues shape people’s interpretations of robot behavior.
JourneyCraft
Evidence & uncertainty
Implemented prototype · real travel content, with dates and identifying details adjusted.
Co-designer + co-engineer · two-person team
In a two-person team, I co-led user research, interaction design, architecture, implementation, and evaluation. Formative walkthroughs shaped how the system distinguishes planned, recorded, and verified travel evidence. Local derived views use supported counts with denominators, rather than personality labels or invented certainty. The prototype uses no language model. My focus is personal intelligence whose evidence and limits people can inspect.
I co-designed the system to distinguish evidence from inference, keeping personal intelligence useful without pretending to know more than the record supports.
Math Lens
Representation

Independent designer and developer
I make mathematical representations people can inspect and adjust, combining Python-based visual computing with interaction design. Voronoi regions and triangulated connections reveal different structures in the same image; Layer Reveal progressively uncovers the computational stages beneath the final view. My contribution joins creative computing and visual practice: geometry becomes something to explore, with each representation offering a different way to compare what an image makes perceptible.
I turn mathematical representations into inspectable visual structures that people can manipulate, compare, and reinterpret.
Changing a representation changes what a person notices, making interpretation itself part of the interaction.
Next Planned / proposed
Additional mathematical lenses · possible AR exploration.
I am helping prepare Voronoi-related material with the team for a proposed Oceanside Museum of Art exhibition.
07 / Ways of Seeing
Attention begins
with a frame.
My technical work asks what systems interpret. My visual practice asks what becomes perceptible when the frame changes.


Through photography and studio-art practice, I study how light, negative space, material, geometry, and movement change what becomes perceptible. Framing can clarify a spatial relationship or preserve ambiguity.
I bring that sensitivity into interface hierarchy and system feedback for HCI, HRI, and interactive AI: a signal’s placement, timing, contrast, and context determine whether someone can notice it, interpret it, and act on it.


Photographic studies · studio-art coursework and personal practice.
See the computational question ↑09 / Evidence & Links
Evidence & Links
Researcher-builder across HCI, HRI, robotics, interactive AI, and creative computing.
- CV
- CV ↗
- Publications
- Published paper · PDF ↗Publication records ↗
- General portfolio
- chenchenzheng.com ↗
- Email / contact
- amily.zheng2025@outlook.com ↗















