FlashAttention-3 unleashes the power of H100 GPUs for LLMs

FlashAttention-3 is a new technique that uses the full capacity of Nvidia H100 GPUs to compute the attention values of LLMs.