XFormers
XFormers is a deep learning library to implement many complex attention operations.
It focuses on providing the Memory Efficient Attention as well as many other operations.
For example it also has BlockSparseAttention support or any other forms of attention where the attention bias matrix induces a complex attention pattern, like for example in the sliding window attention used in Mistral models
These are my way to install XFormers on Mac M1:
brew install libomp
brew install llvm
export PATH="/opt/homebrew/opt/libomp/bin:$PATH"
export PATH="/opt/homebrew/opt/llvm/bin:$PATH
export CC=/opt/homebrew/opt/llvm/bin/clang
export CXX=/opt/homebrew/opt/llvm/bin/clang++
Further reading
Read more in the tech topic.
Let's talk!
I'm Carlo Nicolini — I am interested on the reliability of AI reasoning systems (interpretability, inference-time methods, probabilistic language programming) and on quantitative portfolio optimization (I am a maintainer of skfolio). If you're working on something in these areas and think we might collaborate, chat, discuss, I'm happy to talk about it!
The best way to reach me is on via DM on LinkedIn.