Apr 3, 2024 · Compared to LoRA, PiSSA updates the principal components while freezing the "residual" parts, allowing faster convergence and enhanced performance.

Jul 17, 2024 · The PiSSA-initialized models are shared on Models for easy reuse. They retain the same input and output as the original models but are split into residual models and PiSSA adapters for …

We combine PiSSA with NF4 quantization to propose QPiSSA, which reduces quantization error by about 20% compared to QLoRA, while maintaining the fast convergence and high performance of …

We combine PiSSA with NF4 quantization to propose QPiSSA, which reduces quantization error by about 20% compared to QLoRA, while maintaining the fast convergence and high performance of …

Jul 17, 2024 · The PiSSA-initialized models are shared on Models for easy reuse. They retain the same input and output as the original models but are split into residual models and PiSSA adapters for …

Sep 25, 2024 · Comparative experiments of PiSSA and LoRA across 11 different models, ranging from 184M to 70B, encompassing 5 NLG and 8 NLU tasks, reveal that PiSSA consistently outperforms …

Apr 3, 2024 · PiSSA provides a novel direction for research in PEFT by identifying and fine-tuning the principal components within the model, analogous to slicing and re-baking the richest slice of a pizza.

May 24, 2025 · Unlike Data Parallel LoRA or PiSSA, which maintain identical adapters across all devices, HD-PiSSA assigns different principal components of the pre-trained weights to each GPU, …

Comparative experiments of PiSSA and LoRA across 12 different models, ranging from 184M to 70B, encompassing 5 NLG and 8 NLU tasks, reveal that PiSSA consistently outperforms LoRA under …

We stay serious about pizza Without taking ourselves too seriously.