Publications

Fast Transformers with Clustered Attention
A. VyasA. KatharopoulosF. Fleuret
NeurIPS, 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
ICML, 2020
Processing Megapixel Images with Deep Attention-Sampling Models
A. KatharopoulosF. Fleuret
ICML, 2019
Not All Samples Are Created Equal: Deep Learning with Importance Sampling
A. KatharopoulosF. Fleuret
ICML, 2018
Learning local feature aggregation functions with backpropagation
EUSIPCO, 2017
Fast Supervised LDA for Discovering Micro-Events in Large-Scale Video Datasets
ACMM, 2016

Software

The following is a non-exhaustive list of open source software projects that I started or contribute to. They are at various stages of development and usefulness so tread carefully. Each project may be distributed under different open source license although I prefer highly permissive licenses such as the MIT.

Fast Transformers
GitHub stars PyPI Downloads
Fast Transformers is a library for efficient transformer implementations for PyTorch with a focus on efficient self-attention.
simple-3dviz
GitHub stars PyPI Downloads
simple-3dviz is a lightweight and easy-to-use library that provides an easy interface for visualizing 3D objects with hundreds of thousands of vertices efficiently without the overhead of writing hundreds of lines of code.
Attention Sampling
GitHub stars PyPI Downloads
Attention sampling allows neural networks to process extremely large images with a complexity decoupled from the size of the images.
Importance Sampling
GitHub stars PyPI Downloads
Keras Importance Sampling is a library that speeds up neural network training by selecting important examples from the dataset for each iteration.
LDA++
GitHub stars
LDA++ is a library that allows experimentation with the Latent Dirichlet Allocation topic modeling algorithm and provides relatively fast supervised and unsupervised implementations.
NlpTools (PHP)
GitHub stars Packagist Downloads
NlpTools is the most comprehensive PHP library for natural language processing. It provides a relatively large collection of algorithms for text classification, clustering, topic modeling, stemming, preprocessing and more.