Search Results for author:
Found 16 papers, 9 papers with code
LLMBind: A Unified Modality-Task Integration Framework
In the multi-modal domain, the dependence of various models on specific input formats leads to user confusion and hinders progress.
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
In this work, we propose a simple yet effective training strategy MoE-Tuning for LVLMs.
Ranked #57 on
Visual Question Answering
on MM-Vet
Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting
The core idea is to combine the powerful image generation capability of the 2D diffusion model and the texture alignment ability of the repainting strategy for generating high-quality multi-view images with consistency.
Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models
Video-based large language models (Video-LLMs) have been recently introduced, targeting both fundamental improvements in perception and comprehension, and a diverse range of user inquiries.
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
In this work, we unify visual representation into the language feature space to advance the foundational LLM towards a unified LVLM.
Ranked #3 on
Zero-Shot Video Question Answer
on TGIF-QA
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
Ranked #1 on
Zero-shot Audio Classification
on VGG-Sound
(using extra training data)
ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space Manipulation
Despite the successful image reconstruction achieved by diffusion-based methods, there are still challenges in effectively manipulating fine-gained facial attributes with textual instructions. To address these issues and facilitate convenient manipulation of real facial images, we propose a novel approach that conduct text-driven image editing in the semantic latent space of diffusion model.
Temporal Contrastive Learning for Spiking Neural Networks
However, in this work, we find the method above is not ideal for the SNNs training as it omits the temporal dynamics of SNNs and degrades the performance quickly with the decrease of inference time steps.
Album Storytelling with Iterative Story-aware Captioning and Large Language Models
One natural approach is to use caption models to describe each photo in the album, and then use LLMs to summarize and rewrite the generated captions into an engaging story.
MADAv2: Advanced Multi-Anchor Based Active Domain Adaptation Segmentation
Unsupervised domain adaption has been widely adopted in tasks with scarce annotated data.
Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation
A key to this challenging task is to fully utilize the information in the support images by exploiting fine-grained correlations between the query and support images.
Ranked #4 on
Few-Shot Semantic Segmentation
on COCO-20i (1-shot)
Deformer: Towards Displacement Field Learning for Unsupervised Medical Image Registration
In this study, we propose a novel Deformer module along with a multi-scale framework for the deformable image registration task.
A New Bidirectional Unsupervised Domain Adaptation Segmentation Framework
Domain shift happens in cross-domain scenarios commonly because of the wide gaps between different domains: when applying a deep learning model well-trained in one domain to another target domain, the model usually performs poorly.
Multi-Anchor Active Domain Adaptation for Semantic Segmentation
Unsupervised domain adaption has proven to be an effective approach for alleviating the intensive workload of manual annotation by aligning the synthetic source-domain data and the real-world target-domain samples.
Ensembled ResUnet for Anatomical Brain Barriers Segmentation
However, due to the visual and anatomical differences between different modalities, the accurate segmentation of brain structures becomes challenging.
A Macro-Micro Weakly-supervised Framework for AS-OCT Tissue Segmentation
Primary angle closure glaucoma (PACG) is the leading cause of irreversible blindness among Asian people.
