Search Results for author:
Found 10 papers, 3 papers with code
SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining
In this work, we propose to parameterize the weights as a sum of low-rank and sparse matrices for pretraining, which we call SLTrain.
Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
Moreover, we identify a connection between the proposed IRL based approach, and certain self-play approach proposed recently, and showed that self-play is a special case of modeling a reward-learning agent.
SA-Attack: Speed-adaptive stealthy adversarial attack on trajectory prediction
This method searches the sensitive region of trajectory prediction models and generates the adversarial trajectories by using the vehicle-following method and incorporating information about forthcoming trajectories.
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
In the evolving landscape of natural language processing (NLP), fine-tuning pre-trained Large Language Models (LLMs) with first-order (FO) optimizers like SGD and Adam has become standard.
Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms
We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds.
A Riemannian ADMM
The iteration complexity of the proposed algorithm for obtaining an $\epsilon$-stationary point is analyzed under mild assumptions.
Solving combinational optimization problems with evolutionary single-pixel imaging
Single-pixel imaging (SPI) is a novel optical imaging technique by replacing the pixelated sensor array in a conventional camera with a single-pixel detector.
Federated Learning on Riemannian Manifolds
This paper studies FL over Riemannian manifolds, which finds important applications such as federated PCA and federated kPCA.
Playing Tic-Tac-Toe Games with Intelligent Single-pixel Imaging
Single-pixel imaging (SPI) is a novel optical imaging technique by replacing a two-dimensional pixelated sensor with a single-pixel detector and pattern illuminations.
Stochastic Zeroth-order Riemannian Derivative Estimation and Optimization
We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve Riemannian optimization problem with only noisy objective function evaluations.
