I am pleased to announce that our paper "LoRACode: LoRA Adapters for Code Embeddings" has been accepted at the 3rd Workshop on Deep Learning for Code (DL4C), co-located with ICLR 2025 in Singapore!
The Problem
Code embeddings are essential for semantic code search, but current approaches often struggle to capture the precise syntactic and contextual nuances inherent in code. Open-source models such as CodeBERT and UniXcoder exhibit limitations in scalability and efficiency, while high-performing proprietary systems impose substantial computational costs.
Our Approach
We introduce a parameter-efficient fine-tuning method based on Low-Rank Adaptation (LoRA) to construct task-specific adapters for code retrieval. Our approach reduces the number of trainable parameters to less than two percent of the base model while achieving competitive performance.
Key Benefits
- Dramatically reduced training costs
- Efficient adaptation to new code retrieval tasks
- Maintains competitive retrieval accuracy
This is joint work with Saumya Chaturvedi and Aman Chadha.