Petals: Collaborative Inference and Fine-Tuning of Large Models (arxiv.org)

Researchers have introduced "Petals," a groundbreaking system that enables collaborative inference and fine-tuning of large language models (LLMs) like BLOOM-176B and OPT-175B. This innovative approach allows multiple parties to pool their resources, overcoming the technical hurdles and high costs associated with using these massive models, which typically require access to high-end hardware. Petals achieves impressive performance, managing interactive inference at about one step per second on consumer-grade GPUs, far surpassing existing methods like RAM offloading and API limitations. The significance of Petals lies in its ability to democratize access to large LLMs, making it feasible for researchers and developers who lack access to powerful computing resources. By not only facilitating faster inference but also exposing hidden states of models for custom training and fine-tuning, Petals opens new avenues for research and application in the AI/ML community. This enhances the flexibility and adaptability of LLMs, inviting a diverse range of innovations and improvements in natural language processing tasks, while fostering collaboration across different sectors of the research community.