HuggingFace Hub Client (huggingface_hub): Tests HF API compatibility
Custom HTTP Client (requests): Tests custom endpoints and validates API schema

Why Dual Testing?

HF Client tests: Ensure compatibility with existing HF ecosystem
HTTP Client tests: Validate custom endpoints and catch reverse-engineering errors

If HF client fails but HTTP client succeeds → Our reverse-engineering of HF API is wrong If both fail → Our implementation is broken If both succeed → ✅ Perfect compatibility

Prerequisites

1. Deploy KohakuHub Server

Tests require a running KohakuHub instance (via docker-compose):

# From project root
cp docker-compose.example.yml docker-compose.yml
# Edit docker-compose.yml with your configuration

# Build and start
npm install --prefix ./src/kohaku-hub-ui
npm run build --prefix ./src/kohaku-hub-ui
docker-compose up -d --build

# Verify server is running
curl http://localhost:28080/api/version

Important: Tests connect to http://localhost:28080 (nginx port) by default.

2. Install Test Dependencies

# Install test requirements
pip install pytest pytest-xdist requests huggingface_hub

# Or from project root
pip install -e ".[test]"

Configuration

Tests are configured via environment variables or defaults in config.py:

Variable	Default	Description
`TEST_ENDPOINT`	`http://localhost:28080`	KohakuHub API endpoint (use nginx port!)
`TEST_USERNAME`	`testuser`	Test user username
`TEST_EMAIL`	`test@example.com`	Test user email
`TEST_PASSWORD`	`testpass123`	Test user password
`TEST_ORG_NAME`	`testorg`	Test organization name
`TEST_REPO_PREFIX`	`test`	Prefix for test repositories
`TEST_TIMEOUT`	`30`	HTTP request timeout (seconds)
`TEST_CLEANUP`	`true`	Cleanup resources after tests

Example: Test Against Custom Endpoint

export TEST_ENDPOINT=http://my-server.com:28080
export TEST_USERNAME=myuser
export TEST_PASSWORD=mypass
pytest tests/

Running Tests

Run All Tests

# From project root
pytest tests/

# With verbose output
pytest tests/ -v

# With coverage
pytest tests/ --cov=kohakuhub --cov-report=html

Run Specific Test Files

# Authentication tests only
pytest tests/test_auth.py -v

# Repository CRUD tests
pytest tests/test_repo_crud.py -v

# File operations
pytest tests/test_file_ops.py -v

# LFS operations
pytest tests/test_lfs.py -v

Run Specific Tests

# Run single test
pytest tests/test_auth.py::TestAuthentication::test_version_check -v

# Run tests matching pattern
pytest tests/ -k "upload" -v

# Run tests with specific marker
pytest tests/ -m lfs -v

Test Markers

Tests are marked for easier filtering:

# Run only LFS tests
pytest tests/ -m lfs

# Skip slow tests
pytest tests/ -m "not slow"

# Run tests for specific repo type
pytest tests/test_repo_crud.py::test_create_different_repo_types[model] -v

Available markers:

lfs - Tests requiring LFS (large files >10MB)
slow - Slow running tests (files >50MB)
repo_type(type) - Tests for specific repository types

Parallel Execution

# Run tests in parallel (4 workers)
pytest tests/ -n 4

# Auto-detect CPU cores
pytest tests/ -n auto

Note: Some tests may not be thread-safe. Use with caution.

Test Structure

tests/
├── __init__.py              # Package init
├── conftest.py              # Pytest fixtures and configuration
├── config.py                # Test configuration
├── base.py                  # Base test classes and utilities
├── test_auth.py            # Authentication tests
├── test_repo_crud.py       # Repository CRUD operations
├── test_file_ops.py        # File upload/download/delete
├── test_lfs.py             # Large file storage tests
└── README.md               # This file

Test Categories

1. Authentication Tests (`test_auth.py`)

User registration
Login/logout flow
Session management
API token creation/deletion
Token-based authentication
whoami endpoint

2. Repository CRUD Tests (`test_repo_crud.py`)

Repository creation (model, dataset, space)
Repository deletion
Repository listing
Repository info retrieval
Repository move/rename
Private repository handling
Duplicate detection

3. File Operations Tests (`test_file_ops.py`)

Small file upload/download (<10MB)
Folder upload
File deletion
File listing (tree API)
File metadata (HEAD request)
Content integrity verification
Commit messages

4. LFS Tests (`test_lfs.py`)

Large file upload (>10MB)
LFS batch API
File size threshold (10MB boundary)
LFS deduplication
Mixed file sizes
Content integrity for large files
LFS metadata in tree API

Fixtures

Session-Scoped Fixtures

Created once per test session:

test_config: Test configuration
http_client: Unauthenticated HTTP client
api_token: API token for test user
authenticated_http_client: HTTP client with auth
hf_client: HuggingFace Hub client wrapper
test_org: Test organization

Function-Scoped Fixtures

Created per test function:

temp_repo: Temporary repository (auto-cleanup)

Example Usage

def test_something(hf_client, temp_repo):
    """Test with HF client and temporary repository."""
    repo_id, repo_type = temp_repo

    # Upload file
    hf_client.upload_file(
        path_or_fileobj="test.txt",
        path_in_repo="test.txt",
        repo_id=repo_id,
        repo_type=repo_type,
    )

    # Repository will be cleaned up automatically

Debugging

Enable Verbose Logging

# Pytest verbose mode
pytest tests/ -vv

# Show print statements
pytest tests/ -s

# Show local variables on failure
pytest tests/ -l

Run Failed Tests Only

# Run last failed tests
pytest tests/ --lf

# Run failed first, then others
pytest tests/ --ff

Stop on First Failure

pytest tests/ -x

Interactive Debugging

# Drop into debugger on failure
pytest tests/ --pdb

# Drop into debugger on first failure
pytest tests/ -x --pdb

Common Issues

1. Connection Refused

Problem: ConnectionRefusedError or Connection refused to localhost:28080

Solution: Ensure KohakuHub is running:

docker-compose ps
curl http://localhost:28080/api/version

2. Wrong Port (48888)

Problem: Tests connecting to backend port instead of nginx

Solution: Use TEST_ENDPOINT=http://localhost:28080 (nginx port)

3. Authentication Errors

Problem: 401 Unauthorized errors

Solution: Check test user credentials or recreate test user:

# Delete old user from database if needed
docker-compose exec hub-api python -c "from kohakuhub.db import User; User.delete().where(User.username == 'testuser').execute()"

4. File Permission Errors

Problem: Cannot create temporary files

Solution: Check disk space and permissions for temp directory

5. LFS Upload Failures

Problem: Large file uploads timing out

Solution:

Increase TEST_TIMEOUT environment variable
Check MinIO is running: docker-compose ps minio
Verify S3 credentials in docker-compose.yml

Cleanup

Tests automatically cleanup resources if TEST_CLEANUP=true (default):

Temporary repositories are deleted
Temporary files are removed
API tokens are revoked (optional)

Manual Cleanup

If tests fail and leave resources:

# List test repositories
curl http://localhost:28080/api/models?author=testuser

# Delete manually via API
curl -X DELETE http://localhost:28080/api/repos/delete \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"type": "model", "name": "test-repo-name"}'

# Or use kohub-cli
kohub-cli repo delete testuser/test-repo-name --type model

Continuous Integration

GitHub Actions Example

name: API Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'

    - name: Install dependencies
      run: |
        pip install pytest requests huggingface_hub

    - name: Start KohakuHub
      run: |
        cp docker-compose.example.yml docker-compose.yml
        npm install --prefix ./src/kohaku-hub-ui
        npm run build --prefix ./src/kohaku-hub-ui
        docker-compose up -d --build
        sleep 30  # Wait for services to start

    - name: Run tests
      run: pytest tests/ -v

    - name: Stop services
      run: docker-compose down

Contributing

When adding new tests:

Follow existing patterns in test files
Use both HF client and HTTP client where applicable
Add appropriate markers (@pytest.mark.lfs, etc.)
Ensure cleanup in teardown or use fixtures
Document test purpose in docstring
Update this README if adding new test categories

Support

Issues: https://github.com/KohakuBlueleaf/KohakuHub/issues
Discord: https://discord.gg/xWYrkyvJ2s
Documentation: ../docs/

README.md

KohakuHub API Tests

Test Strategy