> ## Documentation Index
> Fetch the complete documentation index at: https://docs.okchat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Training Data Preparation Best Practices

> Learn how to prepare high-quality training data for your chatbot, including Q&A pairs, document uploads, and CSV formatting guidelines.

## Overview

This guide covers best practices for preparing training data to ensure optimal performance of your chatbot. We'll cover different data formats and provide specific guidelines for each type.

## Q\&A Pairs

### Individual Entry Guidelines

* Keep questions clear and specific
* Provide comprehensive answers
* Use consistent formatting
* Include relevant context
* Avoid duplicate questions

### Batch Upload (JSON)

```json theme={null}
[
  {
    "question": "What are your business hours?",
    "answer": "We are open Monday through Friday, 9 AM to 5 PM EST."
  }
]
```

#### JSON Format Requirements

* Use an array of objects
* Each object must have "question" and "answer" fields
* Use UTF-8 encoding
* Maximum file size: 10MB
* Validate JSON structure before upload

#### Best Practices

1. **Question Formatting**

   * Use natural language
   * Include variations of similar questions
   * Keep questions concise but descriptive

2. **Answer Quality**
   * Provide complete, accurate information
   * Use consistent tone and style
   * Include necessary context
   * Break down complex answers into digestible parts

## Document Uploads

### Supported File Types

* PDF (.pdf)
* Markdown (.md, .mdx)
* Microsoft Word (.docx, .doc)
* CSV (.csv)
* JSON (.json)

### General File Guidelines

1. **File Naming**

   * Use descriptive names
   * Follow kebab-case convention
   * Include version numbers if applicable
   * Examples:
     * ✅ `product-return-policy-2024.pdf`
     * ✅ `customer-faqs-v2.md`
     * ❌ `doc1.pdf`
     * ❌ `New Document (1).docx`

2. **File Organization**

   * Group related documents
   * Use consistent naming patterns
   * Keep file sizes manageable
   * Remove unnecessary metadata

3. **Content Structure**
   * Use clear headings and sections
   * Include a table of contents for longer documents
   * Maintain consistent formatting
   * Remove irrelevant content

## CSV Best Practices

### File Structure

1. **Headers**

   * Use clear, descriptive column names
   * Avoid spaces (use underscores or hyphens)
   * Include a description row if needed
   * Example:
     ```csv theme={null}
     title,description,category
     Product Returns,How to process product returns,Support
     ```

2. **Data Cleaning**

   * Remove empty rows
   * Fill or remove blank cells
   * Standardize data formats
   * Check for and remove duplicate entries

3. **Formatting Guidelines**
   * Use UTF-8 encoding
   * Consistent date formats (YYYY-MM-DD)
   * Proper escaping of special characters
   * Consistent number formatting

### CSV Content Best Practices

1. **Data Organization**

   * One concept per row
   * Consistent data types per column
   * Logical column ordering
   * Appropriate data granularity

2. **Quality Control**
   * Validate data accuracy
   * Check for formatting consistency
   * Remove trailing spaces
   * Verify character encoding

## General Tips

1. **Data Quality**

   * Regular content updates
   * Version control
   * Quality assurance reviews
   * Consistent terminology

2. **Performance Optimization**

   * Compress large files
   * Split very large datasets
   * Remove redundant information
   * Optimize images and media

3. **Maintenance**
   * Schedule regular reviews
   * Document changes
   * Archive outdated versions
   * Monitor chatbot performance

## Common Pitfalls to Avoid

1. **Content Issues**

   * Inconsistent formatting
   * Duplicate information
   * Outdated content
   * Ambiguous answers

2. **Technical Issues**

   * Wrong file encodings
   * Missing data validation
   * Improper file organization
   * Oversized files

3. **Process Issues**
   * Lack of version control
   * Poor documentation
   * Inconsistent naming conventions
   * Missing backup procedures

## Testing and Validation

1. **Before Upload**

   * Validate file formats
   * Check for errors
   * Review content quality
   * Test with sample queries

2. **After Upload**
   * Verify data integration
   * Test chatbot responses
   * Monitor performance
   * Gather user feedback

<Note>
  Regular maintenance and updates of your training data will ensure optimal
  chatbot performance. Set up a review schedule to keep your content fresh and
  accurate.
</Note>

<Warning>
  Always backup your training data before making major changes or updates to
  avoid data loss.
</Warning>

## Content Quality Guidelines

### Preventing Hallucinations

1. **Explicit References**

   * Replace pronouns with specific nouns
   * Examples:
     * ❌ "It provides fast shipping"
     * ✅ "Amazon Prime provides fast shipping"
     * ❌ "They can help with returns"
     * ✅ "Customer service representatives can help with returns"

2. **Specific Quantities and Metrics**

   * Use exact numbers when possible
   * Include units of measurement
   * Examples:
     * ❌ "Delivery takes a few days"
     * ✅ "Delivery takes 3-5 business days"
     * ❌ "The product is quite large"
     * ✅ "The product dimensions are 10" x 12" x 15""

3. **Clear Entity References**

   * Avoid ambiguous references
   * Name specific products, services, or departments
   * Examples:
     * ❌ "Contact support for assistance"
     * ✅ "Contact technical support at [support@company.com](mailto:support@company.com) for assistance"
     * ❌ "Use the tool to analyze data"
     * ✅ "Use the Google Analytics dashboard to analyze website traffic data"

4. **Temporal Clarity**
   * Specify exact dates or time periods
   * Use absolute references instead of relative ones
   * Examples:
     * ❌ "The feature was recently added"
     * ✅ "The feature was added in January 2024"
     * ❌ "Updates coming soon"
     * ✅ "Updates scheduled for Q2 2024"

### Writing Style Guidelines

1. **Factual Language**

   * Use concrete, verifiable statements
   * Avoid subjective descriptions
   * Examples:
     * ❌ "The best solution for your needs"
     * ✅ "A solution that offers features A, B, and C"
     * ❌ "Amazing performance improvements"
     * ✅ "30% faster response time compared to version 1.0"

2. **Consistent Terminology**

   * Maintain a glossary of approved terms
   * Use the same term for the same concept
   * Examples:
     * ❌ "login, sign-in, log in" (mixing terms)
     * ✅ "login" (consistent usage)
     * ❌ "customer, user, client" (mixing terms)
     * ✅ "user" (consistent usage)

3. **Structured Information**

   * Break down complex concepts
   * Use lists and tables for clarity
   * Include step-by-step instructions
   * Example:

     ```markdown theme={null}
     To reset your password:

     1. Navigate to login.company.com
     2. Click "Forgot Password"
     3. Enter your email address
     4. Follow the instructions in the reset email
     ```

4. **Context Completeness**
   * Include all necessary context in each answer
   * Avoid assuming prior knowledge
   * Examples:
     * ❌ "Enable the feature in settings"
     * ✅ "Enable the dark mode feature in User Settings > Display > Theme"
     * ❌ "Use the API key to authenticate"
     * ✅ "Use the API key from your dashboard at api.company.com/keys to authenticate requests"

### Data Validation Checklist

1. **Accuracy Verification**

   * Cross-reference with official documentation
   * Verify URLs and email addresses
   * Confirm product specifications
   * Check policy statements

2. **Completeness Check**

   * All required fields are filled
   * No missing steps in procedures
   * Contact information is complete
   * Reference links are provided

3. **Consistency Review**
   * Terminology alignment
   * Format standardization
   * Unit consistency
   * Date and time formats

<Note>
  When preparing training data, remember that explicit, specific information
  helps prevent AI hallucinations and improves response accuracy.
</Note>