Robotics, Embodied AI, Navigation in vivo

We are excited to share our latest Comment published in #npj Digital Medicine:
“How can reasoning capability empower the AI copilot robot in endoscopic surgery”

Current AI copilots in endoscopic surgery are still largely reactive and vision-driven. While they can detect anatomy, instruments, and scene changes, they often still struggle to truly understand surgical intent, infer hidden tissue dynamics, and respond robustly to uncertainty—all of which are essential for safe and precise intraoperative assistance.

In this article, we highlight how reasoning capability can become a key enabler for the next generation of AI copilot robots in endoscopic surgery:

1. 🧠 Reasoning beyond perception: Enabling VLA-based surgical robots to go beyond simple visual recognition and translate high-level surgeon intent into precise, context-aware low-level motion goals.
2. 🔄 Multimodal and uncertainty-aware intelligence: Fusing endoscopic vision with preoperative imaging, intraoperative sensing, and tracking signals, while dynamically re-weighting information sources under occlusion, bleeding, smoke, and other uncertain conditions.
3. 🤝 Coordinated multi-instrument collaboration: Supporting synchronized control of multiple tools for subtasks such as traction, dissection, and hemostasis, with reasoning-guided adaptation to tissue deformation and workflow variation.
4. 🔮 Anticipatory and safer decision-making: Using chain-of-thought-style reasoning to forecast tissue response, evaluate possible action outcomes, and generate more conservative and interpretable assistance under risk.
5. 🏥 Surgeon-in-the-loop clinical deployment: Framing the AI copilot robot at LoA 2–3, where the system assists with task generation, monitoring, and bounded low-level execution under explicit safety constraints and continuous surgeon oversight.

We believe the future of endoscopic robotic assistance lies not only in systems that can see, but in systems that can reason, adapt, and collaborate. With reasoning-enabled VLA models, AI copilot robots may evolve from reactive executors into true cognitive partners in the operating room.

📃 Read the full paper here: https://www.nature.com/articles/s41746-026-02827-8

👏 Kudos to the team: Mr. Guankun Wang, Dr. Long Bai, and Prof. Hongliang Ren.

Thrilled to share our latest Advanced Science work on enabling highly transferable, autonomous navigation for wireless capsule endoscopy (WCE)—using a lightweight Edge-Contour-Depth Fusion module and deep reinforcement learning (DRL).

WCE has revolutionized GI diagnostics, but its potential is often restricted by incomplete mucosal coverage and the poor ability of existing AI navigation methods to adapt across different patient anatomies. This motivated us to ditch the heavy, brittle, traditional “end-to-end” visual video streams that cause AI models to overfit to a single patient.

🧠✨ What we developed:
A unified, clinically viable framework that features:
🔹 Anatomical Landmark Guidance: Operates on stable, low-dimensional coordinates of conserved gastric structures (the fundus and pyloric antrum) rather than high-dimensional raw video.
🔹 Lightweight Perception Module: Combines classical Canny edge detection and Hu moments with a compact monocular depth network (DispNet) to run efficiently on low-power clinical hardware.
🔹 Robust Sim-to-Real Pipeline: Utilizes a patient-specific digital twin combined with a model-free Adaptive Dynamic Programming (ADP) controller to actively neutralize real-world physical disturbances and actuator latency.

🎯 Key Results:
✅ >97% mucosal coverage achieved within 50 seconds across 8 diverse, patient-derived stomach models in simulation.
✅ 87% mean coverage stability and a 53% reduction in procedure time during real-world ex-vivo experiments compared to expert manual control.
✅ Drastically reduced computational overhead, allowing deployment on low-cost processors (<2 TOPS).

💡 Why it matters:
This study establishes a scalable paradigm that conquers the “reality gap” and patient anatomical variability in medical robotics. By decoupling perception from control, it removes the need for expensive, massive patient datasets and high-end GPUs, paving the way for operator-independent, intelligent GI diagnostics.

🌱 What’s next?
We are expanding our training to encompass extreme pathological distortions (like hiatal hernias) and advancing toward fully wireless clinical deployment with dynamic, target-reaching capabilities for intraoperative pathologies.

🔗 Paper Link: https://advanced.onlinelibrary.wiley.com/doi/10.1002/advs.202600008

June 10, 2026 – Prof. Hongliang Ren of CUHK is invited to deliver an invited talk at the MAE Seminar hosted by the Department of Mechanical and Aerospace Engineering, The Hong Kong University of Science and Technology (HKUST). The talk is entitled “Tethered & Tetherless Reconfigurations at Tissue-Continuum-Origami Interfaces in Vivo Soft Flexible Robotics.”

In this seminar, Prof. Ren shares recent advances in dexterous robotic motion generation and perception for intelligent image-guided procedures. His talk highlights the development of tethered and tetherless reconfigurable robots inspired by origami principles, aiming to address challenges in motion generation, flexibility, and adaptability in minimally invasive surgeries.

Prof. Ren introduces robotic systems that leverage variable-stiffness mechanisms and embedded context awareness to achieve dexterous manipulation within confined anatomical spaces. By eliminating tether constraints and utilizing reconfigurable origami-inspired structures, these systems provide new possibilities for safer, more adaptive, and more intelligent surgical interventions.

This seminar reflects Prof. Ren’s continuous efforts in medical robotics, soft continuum robots, intelligent control, multisensory perception, and next-generation minimally invasive robotic procedures.

June 6, 2026 – Prof. Hongliang Ren of CUHK was invited to speak at the Intelligent Medicine and Brain‑Computer Interface Industry‑Education Integration Innovation Conference held at Furong Laboratory, Changsha.

The conference, themed “Intelligent Medicine and Brain‑Computer Interface: Industry‑Education Integration Leading New Quality Productivity,” was guided by the Hunan Provincial Department of Science and Technology and hosted by Central South University.

Prof. Ren’s Presentation

Prof. Ren spoke in the Embodied AI Industry‑Education Integration session, presenting on “Endoluminal Robotics & Embodied AI in vivo.” He discussed recent advances in continuum robotics, motion perception, and intelligent image‑guided minimally invasive procedures, emphasizing how telerobotic systems with variable stiffness can assist surgeons in dexterous manipulations.

Conference Highlights

The event featured keynote addresses by academicians Lin Lu and Qingming Luo, followed by parallel sessions on brain‑computer interfaces, medical big data, and embodied AI. Prof. Ren’s talk was well received by researchers and clinicians, sparking discussion on clinical translation of robotic technologies.

REN Lab is excited to join #ICRA2026 in Vienna! 🤖✨

This year, our team will present a range of recent work across medical robotics, embodied intelligence, bioinspired design, and robot-assisted surgery. We look forward to sharing our research progress, exchanging ideas with the international robotics community, and connecting with colleagues and collaborators throughout the conference.

Prof. Hongliang Ren’s talk:

WORKSHOP#1 – Medical Robot Workshop

 (https://sites.google.com/view/icra26-workshop-medical-robot)

🗓 5 June, Friday, 9:20–9:50, Hall C

Talk: Endoluminal Robotics & Embodied AI in vivo

WROKSHOP#2 – Origami Robot Workshop

(https://sites.google.com/view/origamirob)

🗓 5 June, Friday, 10:10-10:30, Hall C

Talk: Origami and Kirigami Mechanisms in Medical Robotics 

——————————————————————————————————————–

📌 Paper Presentation 1

NeuroVLA: Surgical Scenario-Aware Learning of Debulking Skills in Endoscopic Robotic Neurosurgery Via Vision-Language-Action Model

Authors: Zhiwei Fang, Chi Kit Ng, Huxin Gao, Tao Zhang, Zhiqing Tang, Tat-Ming Chan, Hongbin Liu, Renzhi Wang, Hongliang Ren

🗓 2 June, Tuesday, 15:00–16:30

📍 Hall C, Interactive Session (Thl2l.287)

📌 Paper Presentation 2

GeoLanG: Geometry-Aware Language-Guided Grasping with Unified RGB-D Multimodal Learning

Authors: Rui Tang, Guankun Wang, Long Bai, Huxin Gao, Jiewen Lai, Chi Kit Ng, Jiazheng Wang, Fan Zhang, Hongliang Ren

🗓 3 June, Wednesday, 9:00–10:30

📍 Hall C, Interactive Session (Wel1l.271)

📄 Paper: https://arxiv.org/abs/2602.04231

🖥️ GitHub: https://github.com/Tomry1114/GeoLanG/tree/main

📌 Paper Presentation 3

TMR-VLA: Vision-Language-Action Model for Magnetic Motion Control of Tri-Leg Silicone-Based Soft Robot

Authors: Ruijie Tang, Chi Kit Ng, Kaixuan Wu, Long Bai, Guankun Wang, Yiming Huang, Yupeng Wang, Hongliang Ren

🗓 3 June, Wednesday, 9:00–10:30

📍 Hall C, Interactive Session (Wel1l.311)

📄 Paper: https://arxiv.org/html/2603.00420v1

📌 Paper Presentation 4

SurgVidLM: Towards Multi-Grained Video Understanding with Large Language Model in Robot-Assisted Surgery

Authors: Guankun Wang, Junyi Wang, Wenjin Mo, Long Bai, Kun Yuan, Ming Hu, Jinlin Wu, Junjun He, Yiming Huang, Nicolas Padoy, Zhen Lei, Hongbin Liu, Nassir Navab, Hongliang Ren

🗓 3 June, Wednesday, 15:00–16:30

📍 Hall C, Interactive Session (Wel12l.138)

📄 Paper: https://arxiv.org/abs/2506.17873

🖥️ GitHub: https://github.com/gkw0010/SurgVidLM

📌 Paper Presentation 5

IEEE Robotics & Automation Magazine: Transendoscopic Telerobotic System: Heterogeneous Flexible Manipulators for Bimanual Endoscopic Submucosal Dissection

Authors: Huxin Gao, Xiaoxiao Yang, Tao Zhang, Xiao Xiao, Changsheng Li, Max Q.-H. Meng, Xiuli Zuo, Yanqing Li, Hongliang Ren

🗓 3 June, Wednesday, 15:00–16:30

📍 Hall C, Interactive Session (Wel12l.332)

📄 Paper: https://ieeexplore.ieee.org/abstract/document/11304144/

📌 Paper Presentation 6

EndoDDC: Learning Sparse to Dense Reconstruction for Endoscopic Robotic Navigation Via Diffusion Depth Completion

Authors: Yinheng Lin, Yiming Huang, Beilei Cui, Long Bai, Huxin Gao, Hongliang Ren, Jiewen Lai

🗓 4 June, Thursday, 9:00–10:30 & 5 June Full day workshop @Embracing Intelligent Robotic Assistants for Robot-assisted Surgery in the Era of Embodied Intelligence: Trends, Opportunities, and Challenges

📍 Hall C, Interactive Session (Thl1l.111)

📄 Paper: https://arxiv.org/abs/2602.21893

🎥 Code: https://github.com/Yinheng-Lin/EndoDDC

📌 Paper Presentation 7

Bioinspired Kirigami Capsule Robot for Minimally Invasive Gastrointestinal Biopsy

Authors: Ruizhou Zhao, Yichen Chu, Shuwei Zhao, Wenchao Yue, Hongliang Ren, Raymond Shing-Yan Tang

🗓 4 June, Thursday, 9:00–10:30 & 5 June Full day workshop @Embracing Intelligent Robotic Assistants for Robot-assisted Surgery in the Era of Embodied Intelligence: Trends, Opportunities, and Challenges

📍 Hall C, Interactive Session (Thl1l.204)

📄 Paper: https://arxiv.org/abs/2602.06207

Thrilled to share our newly accepted paper in IEEE Transactions on Medical Robotics and Bionics, where we introduce a soft, endoscope-deployable microfluidic suction robot that combines multimodal intraluminal locomotion with localized aspiration and sampling for targeted mucus clearance and liquid biopsy.

🧠✨ What we developed:

A soft intraluminal robotic platform that:

🔹 Integrates Locomotion + Sampling: A pneumatically controlled 2×2 pouch matrix for multimodal actuation, paired with an independent microfluidic suction module for active liquid extraction and sample recovery.

🔹 Enables Stable Pitch Control: A balloon-based pitch control mechanism improves controllability for intraluminal operation, with the best overall performance observed at an initial pressure range of 2–3 kPa.

🔹 Balances Compliance and Safety: Single-pouch characterization guided the selection of a 2 mm pouch radius, achieving up to 246.91% maximum deformation; burst tests show a system safety factor ≈ 5.27 under the reported operating conditions.

🔹 Targets Real Clinical Pain Points: Designed for constrained lumens (e.g., distal airway) where conventional airway clearance approaches struggle with reach and effectiveness.

🎯 Key Results:

✅ Multimodal mobility: Differential actuation achieves 26.9 mm/min forward speed and 4.86° yaw per drive cycle.

✅ Robust suction across viscosities: Efficiently extracts 20–80% glycerol solutions within 10 s (via parameter tuning).

✅ In vivo feasibility: Endoscope-assisted porcine validation confirmed sequential pouch-driven motion and successful recovery of biological samples containing mucus and tissue fragments after saline irrigation.

💡 Why it matters:

This work demonstrates a compliant, integrated “move + anchor + suction” approach for narrow lumens—supporting safer localized intervention and sampling, with a path toward distal airway translation.

🌱 What’s next?

We’re moving toward more automated closed-loop pneumatic control, improved steerability/navigation, and miniaturization for deeper airway access—while expanding validation in airway-specific models.

May 29, 2026, Óbudai University – Prof. Hongliang Ren of The Chinese University of Hong Kong (CUHK) and Prof. Peter Kazanzides of Johns Hopkins University were invited to present a joint seminar at the Antal Bejczy Center for Intelligent Robotics (IROB), University Research and Innovation Center (EKIK), Óbuda University, Budapest, Hungary.

The seminar was held on 29 May 2026 at the BARK/IROB Nagylabor, with both in‑person and online attendance via Google Meet. The event was hosted by Prof. Tamás Haidegger, Medical Robotics Lead at IROB.

This bilateral academic exchange was supported by the Ministry of Science and Technology of China (MOST) and Hungarian bilateral research projects, fostering international collaboration in medical robotics.

Seminar Talks

Prof. Hongliang Ren presented a talk titled “Endoluminal Robotics & Embodied AI in vivo,” highlighting recent advances in continuum robotics, motion perception, and intelligent image‑guided minimally invasive procedures. He discussed how procedure‑specific telerobotic systems with variable stiffness and context awareness can assist surgeons in performing dexterous manipulations.

Prof. Peter Kazanzides presented a talk titled “Augmented Reality for Robotic Surgery,” sharing his extensive experience from the development of the da Vinci Research Kit (dVRK) and clinical systems.

Speaker Bios

Prof. Hongliang Ren is a Professor at The Chinese University of Hong Kong. He received his Ph.D. from CUHK in 2008 and has held positions at Johns Hopkins University, Harvard Medical School, and National University of Singapore. He is a recipient of the National Science Fund for Distinguished Young Scholars (Scheme A), the CUHK Young Researcher Award, and over 30 other prestigious awards. He has published over 240 papers with more than 22,000 citations and an H‑index of 75, and has been consistently listed among the world’s top 2% most‑cited scientists.

Prof. Peter Kazanzides is a Research Professor of Computer Science at Johns Hopkins University. He received his Ph.D. in Electrical Engineering from Brown University in 1988. He co‑founded Integrated Surgical Systems, which commercialized the Robodoc System for hip and knee replacement surgeries performed on over 20,000 patients. He later joined JHU and contributed to the development of the da Vinci Research Kit (dVRK).

Audience and Discussion

The seminar attracted researchers and students from IROB and partner institutions. Both talks were followed by a joint Q&A session, where attendees discussed technical challenges in continuum robotics, augmented reality guidance, and clinical translation of robotic systems. The event concluded with informal discussions over refreshments, reinforcing the bilateral collaboration between Chinese and Hungarian research groups.

May 21–23, 2026 – Prof. Hongliang Ren of The Chinese University of Hong Kong (CUHK) was invited to deliver a keynote lecture at the 6th International Conference on Robotics and Control Engineering (RobCE 2026), held at The Hong Kong Polytechnic University.

The conference was co‑organized by The Hong Kong Polytechnic University and Southeast University, with support from several engineering societies. Prof. Ren served on the Program Committee of the conference.

Keynote Lecture: Endoluminal Robotics & Embodied AI in vivo

On the morning of 22 May 2026, Prof. Ren presented a talk titled “Endoluminal Robotics & Embodied AI in vivo.” The lecture focused on recent developments in dexterous robotic motion generation and motion perception for intelligent image‑guided minimally invasive procedures.

Prof. Ren highlighted how procedure‑specific telerobotic surgical systems can assist surgeons in performing dexterous manipulations using continuum motion generation mechanisms with variable stiffness and context awareness. The talk addressed the opportunities and challenges brought by minimally invasive surgeries, particularly in surgical motion generation, motion understanding, and intelligent robotic manipulation.

Short Bio of Prof. Hongliang Ren

Prof. Ren received his Ph.D. in Electronic Engineering (Biomedical Engineering) from CUHK in 2008. He has held academic positions at Johns Hopkins University, Boston Children’s Hospital, Harvard Medical School, and the National University of Singapore. He is a recipient of the National Science Fund for Distinguished Young Scholars (Scheme A), the CUHK Young Researcher Award, and multiple best paper awards. He has been consistently listed among the world’s top 2% most‑cited scientists by Stanford University. He has published over 240 papers with more than 22,770 citations and an H‑index of 75.

Audience Engagement

The keynote session was attended by researchers, faculty members, and students from both online and offline venues. Prof. Ren’s talk was followed by a brief Q&A session, where the audience engaged in discussion on continuum robotics, embodied AI, and clinical translation of surgical robots.

May 17, 2026 – Prof. Ren Hongliang’s research group participated in an exchange with the NVIDIA Healthcare and Life Sciences team at The Chinese University of Hong Kong (CUHK). The visit focused on NVIDIA’s technology roadmap in healthcare and potential collaborations in medical robotics, AI‑driven imaging, and simulation.

The NVIDIA delegation included:

• P Dogra – Developer Ecosystems
• A Shapira – Developer Relationships
• Y Ji – Developer Relationships 

The meeting covered NVIDIA’s healthcare strategy and enabling technologies, including:

• Hardware acceleration for medical cloud and edge computing (H200, RTX 4000 series GPUs)
• NVIDIA Cosmos (world foundation model for healthcare) for synthetic surgical data generation and robot policy training
• Isaac Sim platform for rigid‑body physics simulation, sensor simulation, and full surgical procedure rehearsal

The CUHK and MRC teams shared their ongoing work in continuum robotics, endoscopic navigation, and autonomous laparoscopic control systems.

About the NVIDIA Healthcare and Life Sciences Team
The NVIDIA Healthcare and Life Sciences team applies AI, accelerated computing, and domain-specific software platforms to help advance innovation across healthcare and life sciences. Through platforms and tools such as NVIDIA Clara, BioNeMo, Parabricks, MONAI, and healthcare-focused generative AI microservices, NVIDIA supports workflows spanning drug discovery, medical imaging, genomics, healthcare robotics, MedTech, and digital health. Working with healthcare providers, biopharma companies, researchers, and ecosystem partners, the team helps organizations build, deploy, and scale AI-powered solutions that accelerate scientific discovery, improve operational efficiency, and support the development of next-generation healthcare technologies.

NANJING, May 16, 2026 – Prof. Hongliang Ren of The Chinese University of Hong Kong attended the 38th Chinese Control and Decision Conference (CCDC 2026), which opened at the Nanjing Fengda International Hotel and featured a forum titled “Knowledge and Data-Driven Intelligent Diagnosis and Treatment“. The forum gathered leading experts in control theory, biomedical engineering, and artificial intelligence to explore how modern intelligent systems are reshaping the future of healthcare.

Chaired by Prof. Guanglin Li of the Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, the forum featured invited speakers (Profs. YT Zhang, Max QH Meng, L Meng, HL. Ren, HR. Li & Y Chen) sharing perspectives on how knowledge-driven models, data-driven methods, and intelligent control technologies can support the next generation of diagnosis, treatment, and clinical decision-making.

The forum’s central theme revolved around the integration of “knowledge-driven” and “data-driven” paradigms for intelligent diagnosis and treatment. Panelists deliberated on how cutting-edge technologies—including advanced AI theories, modern signal processing, and intelligent decision-making and feedback mechanisms—can be applied across the entire clinical workflow, ranging from model-based disease dynamic prediction and intelligent drug dosage regulation to personalized rehabilitation robots and closed-loop neuromodulation systems. This multi-disciplinary exchange aims to drive the transformation of diagnostic and therapeutic models from passive and static to active, dynamic, and closed-loop interventions.

Connecting Robotics Innovation with Intelligent Healthcare

Prof. Ren leads pioneering work in intelligent surgical robotics, soft continuum robots, and medical mechatronics, with a strong focus on translational biomedical engineering. His recent project, “Embodied Intelligence Systems for Fine Perception and Dexterous Manipulation in Flexible Endoscopy,” funded as a key national project, aims to develop perceptive, compliant, and intelligent surgical systems.

Following the talk, Prof. Ren received a Certificate of Appreciation from CCDC 2026 and was pictured with Forum Chair Prof. Guanglin Li.

Prof. Ren’s team’s developing miniature flexible robots capable of navigating narrow, tortuous lumen branches to perform multi-modal micro-biopsy and immune-sensing in the vicinity of small confined spaces.

The forum concluded with a forward-looking discussion on how the integration of AI, control theory, and robotics can address major clinical challenges, catalyze original technological breakthroughs, and provide strategic support for the development of intelligent healthcare system, bridging foundational robotics and mechatronics with clinical needs to create next-generation smart surgical systems.