Bespoke Nimble: open data, open model, open recipe for an open Jev (twitter.com)

Bespoke Labs has unveiled Bespoke Nimble, an innovative open framework consisting of open data, an open model, and an open recipe aimed at enhancing AI capabilities. Central to this project is a novel data curation approach known as contrastive data curation, which involves generating negative data by subtly altering facts. This method helps refine the model's decision-making skills without relying on elaborate probabilities. The synthetic dataset spans ten categories and is carefully split for training and evaluation purposes, further empowering the model's evaluation process. Technically, Bespoke Nimble employs a LoRA finetuning strategy on the Qwen 3.5-9B model and utilizes parallel constrained decoding for model inference. Impressively, post-training results show a significant improvement in performance, with Nimble achieving a 90% accuracy on curated evaluations compared to 66% for Qwen, while the Jev model leads with 93%. However, a caveat remains: the lack of standard benchmarks raises questions about its efficacy across diverse tasks. Despite this, the release aims to inspire further research and innovation within the AI/ML community, promoting a collaborative approach to advancing artificial intelligence.