Skip to content

mpenning/useful_things

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 

Repository files navigation

Useful Things

Programming Infrastructure

  • vulture - Find dead python code
  • pylama - Useful python code-audit tool
  • pyupgrade - Automatically upgrade python syntax to baseline against specific python versions
  • yamllint - Simple yaml linter
  • yamlfix - Automatic yaml formatting fixer
  • pudb - Full-screen terminal Python debugger
  • pdoc - Documentation system for moderate-sized python projects
  • ALE - Add IDE features to vim and neovim
  • Neovide - Enhanced neovim GUI with a focus on vim as an IDE
  • nvim-treesitter - Parser and auto-complettion for neovim
  • YouCompleteMe - Auto-completion for ALE
  • PySnooper - Debug python with decorators
  • snoop - Debug python with decorators; similar to PySnooper
  • bandit - Find security problems in python code
  • pss - Code-search tool written in python
  • rust-analyzer - Linter for rust projects

Profilers

  • pyinstrument - Call stack profiler for python libraries; detect why python is slow
  • py-spy - Sampling profiler for python code

GitHub Actions

  • act: Run GitHub actions locally on your laptop
  • zizmor: Static YAML analysis for GitHub Actions YAML

Cloud Infrastructure

Python Language

Python - General Usage

  • uv: A rust replacement for pip
  • ty: A Python type checker in rust
  • pre-commit - Add hooks to git commit actions
  • ultisnips
  • cookiecutter - A disk and file templating framework
  • pyautogui - Framework to exert control over GUI interfaces
  • pywinauto - Framework to exert control over GUI interfaces
  • diagrams - Diagrams as Python code
  • mermaid-py - Python interface to mermaid-js diagramming library
  • jupyter-notebook - Dynamically analyze python code as a savable notebook

Python - Compilers / Transpilers

  • codon - Compile Python into static binaries
  • PyO3 - Write Python modules in rust, or embed Python in a rust binary
  • Cython - Accelerate Python with C
  • py2many - A python transpiler (i.e. convert Python to Rust, C++, etc...)
  • pyinstaller - Compile to binary including required packages
  • Nuitka - Compile python code to a binary
  • Cython - CPython compiler

Python - Code Libraries

  • Arrow - Flexible Python time-handling
  • whenever - Time-handling replacement for Python datetime and some Arrow use-cases
  • traitlets - Python classes with strong typing and attribute observer operations.
  • attrs - Python classes without boilerplate
  • GitPython - Manage git repositories with python
  • loguru - Delgan's excellent Python logging framework
  • rich - Library for terminal colorization
  • scrapy - Web scraping framework
  • marshmallow - Implement Python object serialization an de-serialization with type validation; works well with nested dicts
  • trio - Python async and concurrency library
  • orjson - Fast and correct json parsing library
  • ptftpd - Python TFTPd and PXE tool suite
  • pyjwt - Python implementation of JWT (JSON Web Token)
  • python-saml - Python SAML toolkit
  • dnspython - Python interface to DNS queries
  • splunk-sdk-python - Splunk SDK for Python
  • hvac - Python interface to Hashicorp Vault
  • pyshark - Python interface to tshark / Wireshark in the Terminal
  • docker-py - Python API for the Docker ecosystem
  • peewee - Python ORM for sqllite, postgresql, mysql and cockroach DB
  • records - Make SQL queries and ingest results as python objects
  • python-prompt-toolkit - For interactive python terminal prompt apps
  • psutil - Cross platform process and system monitoring
  • watchdog - Monitor file-system events in python
  • invoke - Pythonic command execution and task management
  • pandas - Pythonic data manipulation and statistical analysis library
  • polars - Python data manipulation and statistical analysis written in rust
  • Seaborn - Simple statistical graphing package
  • plotly - Graphing and plotting library
  • prophet - Facebook's canned time-series forcasting library
  • pysmb - Python SMB client
  • pyyaml - Pythonic yaml handling
  • ruamel.yaml - Pythonic yaml handling
  • rocketry - Pythonic scheduling and cron-like recur using python decorators
  • pexpect - Pythonic Expect library
  • pyparsing - Popular python parsing library via PEG parser
  • TextFSM - Stateful templated python text parsing and value extraction
  • transitions - Python Finite State Machine
  • fuzzywuzzy - Fuzzy text matching
  • click - Flexible CLI arguments library as an improvement over argparse
  • rich - Command-line text formatting library
  • python-prompt-toolkit - Build interactive Python command-line programs
  • deepdiff - Diff python nested dicts
  • dictdiffer - Diff python dicts
  • dotenv - Manage .env files in Python
  • transitions - Object-oriented pythonic State Machine
  • dominate - Write HTML with python
  • matplotlib - Build graphs with Python
  • seaborn - Build statistical graphs with Python
  • PyGithub - GitHub v3 API python library
  • GitPython - Python API for git repositories
  • atlassian-python-api - REST API for Atlassian products (i.e. Jira, Confluence)

Python - Network Automation

  • fabric - Automate command execution on linux systems via SSH
  • netmiko - Automate command execution on routers / switches via SSH
  • exabgp - BGPd written in Python
  • nornir - Network inventory and execution abstractions
  • ntc-templates - Templates to parse command output via TextFSM
  • scrapli - Automate command execution on routers / switches via SSH
  • suzieq - Canned network metrics collection
  • meraki-cli - Meraki CLI tool for interacting with the dashboard
  • py-junos-eznc - Junos automation library
  • boto3 - Automate AWS environments

Machine Learning

Rust Language

The best single-source of rust examples is Rust By Example.

CLI Argument parsing

  • argh - Simplified Rust CLI arg parsing
  • clap - Full-featured Rust CLI arg parsing; can be complex and often deprecates previously-used syntax

Error Handling

  • snafu - Rust error handling
  • anyhow - Rust Application error handling
  • thiserror - Rust Application error handling

General libraries

Go Language

  • go-charts: Go charts library
  • [plot][plot]: A repository for plotting and visualizing data

Tools

General Use utilities

  • difftastic: A text file differ that understands many programming languages. Diffs are intelligent based on the language in the file.
  • taskwarrior: A task-list manager with a useful CLI
  • yamlfix: YAML formatter and fixer
  • just: just is like make, but without the Makefile syntax warts
  • sd: Find and replace simplified
  • ripgrep: Recursive grep through directories
  • RustScan: A fast port-scanner in Rust (can replace nmap for some tasks)

Terminal utilites

  • asciienema: A terminal session recorder built with Rust
  • obs-studio: Record YouTube videos from your laptop

Backend utilites

  • influxdb: A time-series database
  • rrdtool: A time-series database, which intelligently compresses data

Network Tools - Services

  • caddy: Go webserver / reverse proxy with auto-TLS certificate generation
  • frp: A Golang fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.
  • nsd: DNS server in C
  • coredns: Golang DNS server that supports UDP, DoT, DoH, DoQUIC and more.
  • unbound: Popular DNS resolver library
  • scylladb: Fast NoSQL DB
  • Netbox: Network source of truth; based on Django
  • NIPAP: Open-source IP address management (IPAM)
  • fail2ban: Monitor and black-hole repetitive attacks on your Linux server
  • maltrail: Detect malicious network traffic
  • RustScan: Port-scanner; like nmap but in Rust
  • secure_cartography: Use CDP / LLDP / SSH to compile accurate network diagrams

Network Emulators

  • containerlab: Build arbitrary topologies of your favorite Cisco / Arista / Juniper / Nokia Operating System as a Docker container
  • vrnetlab: Tool to convert router images to a Docker container

Network Tools - Diagnostics and Testing

  • trippy: Traceroute / mtr written in Rust
  • mtr: Traceroute / mtr written in Rust
  • sniffnet: Cross-platform network traffic capture with a nice GUI (Windows / MacOS / Linux)
  • gping: Graph ping response times (written in Rust) in the terminal window
  • pingtracer: Progressive ping logger written for MS Windows; discovers all hops in the path and pings them
  • nethogs: Detect which linux processes are consuming the most bandwidth
  • bandwhich: Detect which linux processes are consuming the most bandwidth (written in Rust)
  • sniffer: Detect which linux / Windows processes are consuming the most bandwidth (written in Go)
  • mitmproxy: An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
  • go-mitmproxy: A port of mitmproxy to Go
  • flent: A network performance test client, which requires [fping][fping] (client-side), netperf (server-side) or iperf2 (server-side)
  • crusader: Rust-based network testing and results graphing
  • goben: Measure TCP / UDP throughput between hosts
  • TRex: Cisco's flexible Network test tool
  • netperf: A network performance test server, written by Hewlett Packard. It's best to install the Debian package instead of trying to compile netperf

Embedded Operating Systems

  • Zephyr: A popular RTOS for Embedded systems
  • Embassy: A rust embedded operating system
  • rtic: RTOS for ARM Cortex-M microcontrollers
  • Tock: A secure RTOS for embedded system

About

A brief list of useful libraries and tools

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages