Skip to content

Conversation

@inureyes
Copy link
Member

@inureyes inureyes commented Aug 21, 2025

Summary

This PR implements comprehensive SFTP file transfer functionality for bssh, enabling secure file upload and download operations across multiple cluster nodes in parallel.

Changes

Core SFTP Implementation

  • ✅ Implemented SFTP upload functionality using async-ssh2-tokio
  • ✅ Implemented SFTP download functionality
  • ✅ Added parallel file transfer across multiple nodes
  • ✅ Renamed copy command to upload for better clarity
  • ✅ Added new download command for retrieving files

Glob Pattern Support (NEW)

  • ✅ Added wildcard/glob pattern support for batch operations
  • ✅ Local glob expansion for upload (e.g., *.txt, logs/*.log)
  • ✅ Remote glob expansion for download via SSH ls command
  • ✅ Automatic directory detection for multi-file uploads
  • ✅ Human-readable file size formatting in output

Progress Tracking

  • ✅ Real-time progress indicators with spinners
  • ✅ Status messages showing connection, transfer, and completion states
  • ✅ Per-node success/failure reporting

Documentation

  • ✅ Updated manpage with upload/download command documentation
  • ✅ Added glob pattern examples and usage instructions
  • ✅ Included SFTP requirements and configuration notes

Usage Examples

Upload files with glob patterns

# Upload all log files
bssh -c production upload "*.log" /var/logs/

# Upload specific pattern
bssh -c staging upload "config/*.yaml" /etc/app/

Download files with glob patterns

# Download all conf files
bssh -c production download "/etc/*.conf" ./backups/

# Download application logs
bssh -c webservers download "/var/log/app*.log" ./collected_logs/

Single file operations

# Upload single file
bssh -c production upload deploy.tar.gz /tmp/

# Download single file
bssh -c production download /etc/passwd ./downloads/

Technical Details

  • Uses async-ssh2-tokio's native SFTP support for secure transfers
  • Implements proper error handling with detailed error messages
  • Maintains unique file naming for downloads (prefixed with hostname)
  • Supports both individual and batch file operations
  • Preserves async/parallel execution model

Testing

  • Added test_glob.sh script for glob pattern testing
  • Manual testing completed on localhost
  • All existing tests pass

Checklist

  • Code follows project style guidelines
  • Documentation updated (manpage)
  • Tests pass
  • Glob pattern support implemented
  • Error handling implemented
  • Progress indicators working

This completes the SFTP implementation milestone from TODO.md.

@inureyes inureyes added the type:enhancement New feature or request label Aug 21, 2025
@inureyes inureyes self-assigned this Aug 21, 2025
inureyes and others added 7 commits August 22, 2025 00:11
- Add glob dependency for pattern matching
- Implement local glob expansion for upload command
- Support uploading multiple files with wildcards (*.txt, logs/*.log)
- Implement remote glob expansion for download via SSH ls command
- Support downloading multiple files with wildcards
- Add automatic directory detection for multi-file uploads
- Add human-readable file size formatting
- Update manpage with glob pattern examples
- Add test_glob.sh script for testing glob functionality

This allows users to:
- Upload multiple files: bssh upload "*.log" /remote/dir/
- Download with patterns: bssh download "/var/log/*.log" ./downloads/

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add unit tests for CLI command parsing (upload/download)
- Add unit tests for executor file transfer results
- Add glob pattern matching tests
- Add error handling tests for various failure scenarios
- Add integration tests for localhost SSH operations
- Test coverage for:
  * Upload/download result structures
  * Parallel executor with file transfers
  * Glob pattern resolution
  * Error cases (invalid hosts, ports, files)
  * Multiple file operations

All unit tests pass (47 tests total). Integration tests require
SSH server on localhost to run successfully.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add SSH key detection for integration tests
- Only use SSH key if it exists, otherwise fall back to agent
- Update CI workflow to skip integration tests
- Fix formatting in all test files
- Ensure tests work both locally and in GitHub Actions

Integration tests now gracefully handle environments without:
- SSH keys
- SSH agent identities
- localhost SSH access

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@inureyes inureyes merged commit 26213df into main Aug 21, 2025
2 checks passed
@inureyes inureyes added priority:high High priority issue status:done Completed feature labels Sep 9, 2025
@inureyes inureyes deleted the feature/sftp-impl branch October 30, 2025 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:high High priority issue status:done Completed type:enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants