mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-05-05 21:17:30 -05:00
8d3025afc5
Reorganize TinyTorch CLI from flat structure to hierarchical organization with subfolders for complex commands with subcommands. Changes: - Create subfolders: module/, system/, package/ - Move module commands: module_workflow.py → module/workflow.py - Move module_reset.py → module/reset.py - Move system commands: system.py → system/system.py - Move system subcommands: info.py, health.py, jupyter.py → system/ - Move package commands: package.py → package/package.py - Move package helpers: reset.py, nbdev.py → package/ - Archive deprecated files: clean.py, help.py, notebooks.py, status.py - Update all imports in moved files and main.py - Add __init__.py exports for each subfolder - Create comprehensive CLI test suite (52 tests) - test_cli_registry.py: Validate command registration - test_cli_execution.py: Smoke tests for all commands - test_cli_help_consistency.py: Help text validation - Update tests to match new structure Benefits: - Clear ownership: Easy to see which helpers belong to which commands - Better organization: Related files grouped together - Scales cleanly: Adding subcommands is straightforward - Zero user impact: All commands work exactly the same All 52 tests passing ✅ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
146 lines
4.4 KiB
Markdown
146 lines
4.4 KiB
Markdown
# CLI Test Suite
|
|
|
|
Comprehensive test suite for the TinyTorch CLI (`tito`). Ensures all commands are properly registered, documented, and executable.
|
|
|
|
## Test Files
|
|
|
|
### 1. `test_cli_registry.py` - Command Registration Tests
|
|
Validates the command registry in [tito/main.py](../../tito/main.py#L74-L99):
|
|
|
|
- **TestCLIRegistry**: Core registry validation
|
|
- All commands inherit from `BaseCommand`
|
|
- All commands have descriptions (>10 chars)
|
|
- All commands implement `execute()` and `add_arguments()`
|
|
- Parser creation succeeds
|
|
- No duplicate command names
|
|
- Help text accessible for all commands
|
|
|
|
- **TestCommandFiles**: File system consistency
|
|
- All registered commands have corresponding files
|
|
- No orphaned command files (files without registration)
|
|
- Detects commands that need cleanup
|
|
|
|
- **TestEpilogDocumentation**: Help text consistency
|
|
- Epilog mentions all registered command groups
|
|
- No outdated references
|
|
|
|
### 2. `test_cli_execution.py` - Command Execution Tests
|
|
Smoke tests for actual command execution:
|
|
|
|
- **TestCommandExecution**: Basic execution
|
|
- Bare `tito` command shows welcome screen
|
|
- `tito -h` shows help
|
|
- `tito --version` works
|
|
- All 18 commands can show help (`tito <cmd> -h`)
|
|
- All subcommands can show help
|
|
|
|
- **TestCommandGrouping**: Discoverability
|
|
- Student-facing commands visible in welcome screen
|
|
- Developer commands documented in help
|
|
|
|
- **TestErrorMessages**: Error handling
|
|
- Invalid commands show helpful errors
|
|
- Missing subcommands show help
|
|
|
|
### 3. `test_cli_help_consistency.py` - Help Text Quality
|
|
Ensures help text is consistent and complete:
|
|
|
|
- **TestHelpConsistency**: Formatting consistency
|
|
- All command helps mention 'tito'
|
|
- Bare `tito` vs `tito -h` serve different purposes
|
|
- No references to removed commands
|
|
|
|
- **TestWelcomeScreen**: Welcome screen quality
|
|
- Shows Quick Start section
|
|
- Organizes commands into groups
|
|
- Includes example commands
|
|
|
|
- **TestCommandDocumentation**: Documentation completeness
|
|
- All registered commands documented
|
|
- Checkpoint command specifically validated
|
|
|
|
## Running the Tests
|
|
|
|
Run all CLI tests:
|
|
```bash
|
|
pytest tests/cli/ -v
|
|
```
|
|
|
|
Run specific test file:
|
|
```bash
|
|
pytest tests/cli/test_cli_registry.py -v
|
|
```
|
|
|
|
Run with output:
|
|
```bash
|
|
pytest tests/cli/ -v -s
|
|
```
|
|
|
|
## What These Tests Catch
|
|
|
|
### ✅ Prevents
|
|
|
|
1. **Orphaned Commands**: Command files that aren't registered
|
|
2. **Missing Documentation**: Commands without descriptions
|
|
3. **Broken Help**: Commands that crash when showing help
|
|
4. **Inconsistent UX**: Different help formats across commands
|
|
5. **Stale References**: Help text mentioning removed commands
|
|
|
|
### ✅ Validates
|
|
|
|
1. **Registration**: All commands in `TinyTorchCLI.commands` dict
|
|
2. **Implementation**: All commands inherit from `BaseCommand`
|
|
3. **Documentation**: All commands have clear descriptions
|
|
4. **Execution**: All commands can run without crashing
|
|
5. **Discoverability**: Key commands visible to users
|
|
|
|
## Test Coverage
|
|
|
|
**52 tests** covering:
|
|
- 18 registered commands
|
|
- 8 subcommand groups
|
|
- Registry validation
|
|
- Execution smoke tests
|
|
- Help text consistency
|
|
- Welcome screen UX
|
|
|
|
## How Modern CLIs Handle Testing
|
|
|
|
These tests follow industry best practices from tools like:
|
|
|
|
1. **Click** (Python): Command registration validation
|
|
2. **Git**: Comprehensive help text testing
|
|
3. **Docker**: Smoke tests for each command
|
|
4. **kubectl**: Subcommand hierarchy validation
|
|
|
|
Key patterns used:
|
|
- **Registry-based validation**: Single source of truth in code
|
|
- **Smoke tests**: Don't test functionality, just "does it run?"
|
|
- **Help text parsing**: Ensure documentation stays current
|
|
- **Snapshot testing**: Compare outputs (could be added later)
|
|
|
|
## Maintenance
|
|
|
|
When adding a new command:
|
|
|
|
1. Add to `TinyTorchCLI.commands` dict in [tito/main.py](../../tito/main.py#L74-L99)
|
|
2. Create command file in `tito/commands/`
|
|
3. Add to epilog if it's a major command group
|
|
4. Tests will automatically validate it!
|
|
|
|
When removing a command:
|
|
|
|
1. Remove from `TinyTorchCLI.commands` dict
|
|
2. Delete command file OR add to `known_internal` in tests
|
|
3. Update epilog/welcome screen
|
|
|
|
## Future Enhancements
|
|
|
|
Consider adding:
|
|
|
|
- **Snapshot tests**: Save known-good help output, compare changes
|
|
- **Integration tests**: Test actual command workflows
|
|
- **Performance tests**: Ensure CLI startup is fast
|
|
- **Completion tests**: Validate shell completion scripts
|
|
- **Config tests**: Test config file parsing and validation
|