Search Results for author:
Found 40 papers, 13 papers with code
Ruleformer: Context-aware Rule Mining over Knowledge Graph
Based on this idea, we propose a transformer-based rule mining approach, Ruleformer.
Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression
Recent Vision Transformer Compression (VTC) works mainly follow a two-stage scheme, where the importance score of each model unit is first evaluated or preset in each submodule, followed by the sparsity score evaluation according to the target sparsity constraint.
Prompt-fused framework for Inductive Logical Query Answering
Answering logical queries on knowledge graphs (KG) poses a significant challenge for machine reasoning.
Enhanced Sparsification via Stimulative Training
To alleviate this issue, we first study and reveal the relative sparsity effect in emerging stimulative training and then propose a structured pruning framework, named STP, based on an enhanced sparsification paradigm which maintains the magnitude of dropped weights and enhances the expressivity of kept weights by self-distillation.
MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer
To this end, we propose a novel framework named Multimodal Alignment-Guided Dynamic Token Pruning (MADTP) for accelerating various VLTs.
CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modelling
Precipitation nowcasting based on radar data plays a crucial role in extreme weather prediction and has broad implications for disaster management.
ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentation
Recently, foundational models such as CLIP and SAM have shown promising performance for the task of Zero-Shot Anomaly Segmentation (ZSAS).
Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformers
We first validate eight manually-defined partial fine-tuning strategies across kinds of datasets and vision transformer architectures, and find that some partial fine-tuning strategies (e. g., ffn only or attention only) can achieve better performance with fewer tuned parameters than full fine-tuning, and selecting appropriate layers is critical to partial fine-tuning.
Merging Vision Transformers from Different Tasks and Domains
This work targets to merge various Vision Transformers (ViTs) trained on different tasks (i. e., datasets with different object categories) or domains (i. e., datasets with the same categories but different environments) into one unified model, yielding still good performance on each task or domain.
Wideband Sample Rate Converter Using Cascaded Parallel-serial Structure for Synthetic Instrumentation
In the meantime, the decimation factor of the CIC filter can be adjusted flexibly in a wide range, which is used to improve the system configuration flexibility.
Rethinking of Feature Interaction for Multi-task Learning on Dense Prediction
In this work, we propose to learn a comprehensive intermediate feature globally from both task-generic and task-specific features, we reveal an important fact that this intermediate feature, namely the bridge feature, is a good solution to the above issues.
Efficient Architecture Search via Bi-level Data Pruning
Improving the efficiency of Neural Architecture Search (NAS) is a challenging but significant task that has received much attention.
Towards an end-to-end artificial intelligence driven global weather forecasting system
Initial states are typically generated by traditional data assimilation components, which are computational expensive and time-consuming.
Rethinking the BERT-like Pretraining for DNA Sequences
To address this research gap, we first conducted a series of exploratory experiments and gained several insightful observations: 1) In the fine-tuning phase of downstream tasks, when using K-mer overlapping tokenization instead of K-mer non-overlapping tokenization, both overlapping and non-overlapping pretraining weights show consistent performance improvement. 2) During the pre-training process, using K-mer overlapping tokenization quickly produces clear K-mer embeddings and reduces the loss to a very low level, while using K-mer non-overlapping tokenization results in less distinct embeddings and continuously decreases the loss.
StructChart: Perception, Structuring, Reasoning for Visual Chart Understanding
Charts are common in literature across different scientific fields, conveying rich information easily accessible to readers.
Ranked #19 on
Chart Question Answering
on ChartQA
(using extra training data)
Boosting Residual Networks with Group Knowledge
Specifically, we implicitly divide all subnets into hierarchical groups by subnet-in-subnet sampling, aggregate the knowledge of different subnets in each group during training, and exploit upper-level group knowledge to supervise lower-level subnet groups.
Real-time frequency measurement based on parallel pipeline FFT for time-stretched acquisition system
Real-time frequency measurement for non-repetitive and statistically rare signals are challenging problems in the electronic measurement area, which places high demands on the bandwidth, sampling rate, data processing and transmission capabilities of the measurement system.
RFD-ECNet: Extreme Underwater Image Compression with Reference to Feature Dictionar
Subsequently, an extreme UWI compression network with reference to the feature dictionary (RFD-ECNet) is creatively proposed, which utilizes feature match and reference feature variant to significantly remove redundancy among UWIs.
Experts Weights Averaging: A New General Training Scheme for Vision Transformers
As Vision Transformers (ViTs) are gradually surpassing CNNs in various visual tasks, one may question: if a training scheme specifically for ViTs exists that can also achieve performance improvement without increasing inference cost?
When Hyperspectral Image Classification Meets Diffusion Models: An Unsupervised Feature Learning Framework
Learning effective spectral-spatial features is important for the hyperspectral image (HSI) classification task, but the majority of existing HSI classification methods still suffer from modeling complex spectral-spatial relations and characterizing low-level details and high-level semantics comprehensively.
Stimulative Training++: Go Beyond The Performance Limits of Residual Networks
In this work, we aim to re-investigate the training process of residual networks from a novel social psychology perspective of loafing, and further propose a new training scheme as well as three improved strategies for boosting residual networks beyond their performance limits.
A2S-NAS: Asymmetric Spectral-Spatial Neural Architecture Search For Hyperspectral Image Classification
To address the above issues, we propose a multi-stage search architecture in order to overcome asymmetric spectral-spatial dimensions and capture significant features.
Hyperspectral Image Classification
Neural Architecture Search
JNDMix: JND-Based Data Augmentation for No-reference Image Quality Assessment
Despite substantial progress in no-reference image quality assessment (NR-IQA), previous training models often suffer from over-fitting due to the limited scale of used datasets, resulting in model performance bottlenecks.
$β$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search
To address the robustness problem, we first benchmark different NAS methods under a wide range of proxy data, proxy channels, proxy layers and proxy epochs, since the robustness of NAS under different kinds of proxies has not been explored before.
RFD-ECNet: Extreme Underwater Image Compression with Reference to Feature Dictionary
Subsequently, an extreme UWI compression network with reference to the feature dictionary (RFD-ECNet) is creatively proposed, which utilizes feature match and reference feature variant to significantly remove redundancy among UWIs.
Feature Reconstruction Attacks and Countermeasures of DNN training in Vertical Federated Learning
To address this problem, we develop a novel feature protection scheme against the reconstruction attack that effectively misleads the search to some pre-specified random values.
Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing
In this work, we aim to re-investigate the training process of residual networks from a novel social psychology perspective of loafing, and further propose a new training strategy to strengthen the performance of residual networks.
Neural-Symbolic Entangled Framework for Complex Query Answering
In this work, we propose a Neural and Symbolic Entangled framework (ENeSy) for complex query answering, which enables the neural and symbolic reasoning to enhance each other to alleviate the cascading error and KG incompleteness.
Efficient Joint-Dimensional Search with Solution Space Regularization for Real-Time Semantic Segmentation
In this paper, we intend to search an optimal network structure that can run in real-time for this problem.
$β$-DARTS: Beta-Decay Regularization for Differentiable Architecture Search
Neural Architecture Search~(NAS) has attracted increasingly more attention in recent years because of its capability to design deep neural networks automatically.
Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search
The NAR explores the quality tiers of the search space globally and classifies each individual to the tier they belong to according to its global ranking.
b-DARTS: Beta-Decay Regularization for Differentiable Architecture Search
Neural Architecture Search (NAS) has attracted increasingly more attention in recent years because of its capability to design deep neural network automatically.
Average-Case Communication Complexity of Statistical Problems
Our motivation is to understand the statistical-computational trade-offs in streaming, sketching, and query-based models.
Compatible braidings with Hopf links, multi-loop, and Borromean rings in $(3+1)$-dimensional spacetime
In this paper, we provide a field-theoretical approach towards a complete list of mutually compatible braiding phases of topological orders in (3+1)D spacetime.
Strongly Correlated Electrons High Energy Physics - Theory Mathematical Physics Mathematical Physics
A random forest system combination approach for error detection in digital dictionaries
We investigate combining methods and show that using random forests is a promising approach.
Detecting Structural Irregularity in Electronic Dictionaries Using Language Modeling
Dictionaries are often developed using tools that save to Extensible Markup Language (XML)-based standards.
Active Sampling for Subjective Image Quality Assessment
Subjective tests based on the Mean Opinion Score (MOS) have been widely used in previous studies, but have many known problems such as an ambiguous scale definition and dissimilar interpretations of the scale among subjects.
Convolutional Neural Networks for No-Reference Image Quality Assessment
In this work we describe a Convolutional Neural Network (CNN) to accurately predict image quality without a reference image.
Beyond Human Opinion Scores: Blind Image Quality Assessment based on Synthetic Scores
Instead of training on human opinion scores, we propose to train BIQA models on synthetic scores derived from Full-Reference (FR) IQA measures.
Real-Time No-Reference Image Quality Assessment Based on Filter Learning
Second, the proposed method has the potential to be used in multiple image domains.
