Fine-tuning an LLM to write docs like it's 1995 (passo.uno)

A recent experiment involved fine-tuning a language model (LLM) to imitate the style of technical writing from the 1990s, using a rich corpus sourced from Bitsavers, which hosts old computer manuals. The author downloaded and processed over 37 million words from Microsoft documentation, creating a training dataset of 192,456 examples using an efficient cleaning process. By employing QLoRA (Quantized Low-Rank Adaptation), the author was able to modify existing LLMs without the need for extensive resources, demonstrating that fine-tuning can effectively tailor models to specific writing styles. This approach is significant for the AI/ML community, as it showcases the potential for fine-tuning models locally on affordable hardware rather than relying solely on powerful cloud-based systems. The tests revealed that fine-tuned models performed well in generating authentic documentation in the desired style, even effectively handling prompts about a fictional function. While the results established that fine-tuned models can convincingly mimic historical tech writing, the study also highlighted inherent limitations; these models are not replacements for dedicated human authors but can assist in creating documents that adhere to particular style guides, thereby enhancing productivity in technical writing tasks.