Nous Research / Capybara 34B

Yi-34B Foundation

Built on the Yi-34B base trained natively for 200K context length, enabling deep understanding of extensive input sequences.

Amplify-Instruct Synthesis

Utilizes innovative Amplify-Instruct technique combining top data synthesis methods for efficient, high-quality training data.

Multi-Turn Focus

Over 60% dataset comprises multi-turn conversations (>1,000 tokens average), enabling natural back-and-forth dialogue handling.

Advanced Summarization

Trained on hundreds of complex summary tasks across technical studies and philosophical topics for precise knowledge extraction.

Contamination Prevention

Rigorous dataset checks ensure training data integrity, preventing memorization of evaluation benchmarks.

Transformer Architecture

Employs standard transformer design optimized for long-context processing and reasoning tasks like reality analysis and rationality.

Bilingual Reasoning

Native support for multilingual tasks with strong performance in cross-lingual reasoning and comprehension.

Memory-Efficient Design

Achieves high performance with 10x smaller dataset than comparable models through targeted instruction tuning.

Model Overview

• Model Name: Capybara 34B
• Model Family: Nous-Capybara (based on Yi-34B)
• Provider: NousResearch (fine-tuned version deployed via Cyfuture AI)
• Type: Large-scale autoregressive large language model (LLM)
• Use Cases: Multi-turn conversational AI, long-form summarization, reasoning, complex NLP tasks

Model Architecture & Parameters

• Base Architecture: Yi-34B foundation model
• Parameter Count: ~34 billion parameters (nominal design scale)
• Context Length: Up to 200K tokens maximum context window
• Note: Some deployed or quantized variants may use shorter context lengths for memory efficiency

Training & Datasets

• Training Strategy: Fine-tuned from Yi-34B using proprietary Capybara dataset with Amplify-Instruct method
• Training Composition:
  • Multi-turn conversation examples (>60%)
  • Average ~1,000 tokens per dialogue
• Epochs: Trained for multiple epochs on curated datasets
• Pre-Training Cutoff: Knowledge up to ~late 2022

Capabilities & Performance

• Long Context Handling: Optimized for extremely long documents, summarization, and extended dialogue
• Multi-Turn Dialogue: Enhanced conversational memory and context retention
• Complex Reasoning & NLP: Strong performance in summarization, generation, and reasoning tasks

Quantization & Deployment Options

• Quantization Formats: GGUF variants including Q3_K, Q4_K, Q5_K, Q6_K
• Memory Requirements: ~14–36 GB RAM depending on quantization
• Hardware Support: GPU clusters such as NVIDIA A100, L40S, and H100 via Cyfuture AI
• API Integration: Available through Cyfuture AI inference API with on-demand deployment

Ideal Application Scenarios

• Enterprise chatbots
• Long-document summarization
• Knowledge management systems
• Context-rich dialogue agents
• Complex reasoning workflows

Licensing & Accessibility

• Model License: Open license for base distributions (e.g., MIT — deployment dependent)
• Deployment Model: Publicly listed on Cyfuture AI model registry with enterprise support

Typical Inference Settings (API)

• Max Tokens per Request: Configurable (e.g., up to 16,384 tokens)
• Sampling Parameters: Temperature, top_p, top_k, presence and frequency penalties

Yi-34B Foundation

Built on the powerful Yi-34B base model, delivering strong language understanding and reasoning capabilities across diverse tasks.

200K Context Length

First 34B model from Nous Research achieving unprecedented 200K token context, enabling deep, coherent long-form conversations.

Multi-Turn Dialogues

Over 60% of training data focuses on multi-turn conversations averaging 1,000+ tokens, excelling in natural back-and-forth interactions.

Amplify-Instruct Training

Fine-tuned using innovative Capybara/Amplify-Instruct dataset synthesis, combining advanced data generation techniques for superior instruction following.

Advanced Summarization

Specialized training on hundreds of complex summary tasks, capable of distilling advanced topics, studies, and technical content effectively.

Bilingual Excellence

Superior performance in both English and Chinese, with strong reasoning, reading comprehension, and cross-lingual capabilities.

Multimodal Extension

Obsidian variant adds vision processing, making Nous Research / Capybara 34B a foundation for the world’s smallest competitive multimodal LLM.

Knowledge Recall

Extensive training data enables accurate recall of information up to late 2022 without external connectivity.

Efficiency Optimized

Achieves high performance with 10x smaller dataset than comparable models, balancing capability with resource efficiency.