Position
681 papers with code • 0 benchmarks • 0 datasets
Benchmarks
These leaderboards are used to track progress in Position
Libraries
Use these libraries to find Position models and implementationsMost implemented papers
Non-local Neural Networks
Both convolutional and recurrent operations are building blocks that process one local neighborhood at a time.
RoFormer: Enhanced Transformer with Rotary Position Embedding
Then, we propose a novel method named Rotary Position Embedding(RoPE) to effectively leverage the positional information.
Self-Attention with Relative Position Representations
On the WMT 2014 English-to-German and English-to-French translation tasks, this approach yields improvements of 1. 3 BLEU and 0. 3 BLEU over absolute position representations, respectively.
Dual Attention Network for Scene Segmentation
Specifically, we append two types of attention modules on top of traditional dilated FCN, which model the semantic interdependencies in spatial and channel dimensions respectively.
3D human pose estimation in video with temporal convolutions and semi-supervised training
We start with predicted 2D keypoints for unlabeled video, then estimate 3D poses and finally back-project to the input 2D keypoints.
A Transformer-based Approach for Source Code Summarization
Generating a readable summary that describes the functionality of a program is known as source code summarization.
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
The results demonstrate that Vim is capable of overcoming the computation & memory constraints on performing Transformer-style understanding for high-resolution images and it has great potential to be the next-generation backbone for vision foundation models.
Deep Domain Confusion: Maximizing for Domain Invariance
Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias on a standard benchmark.
The Case for Learned Index Structures
Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not.
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Since the introduction of the transformer model by Vaswani et al. (2017), a fundamental question has yet to be answered: how does a model achieve extrapolation at inference time for sequences that are longer than it saw during training?