Skip to content

Modern local data query and visualization tool. Upload CSV/JSON files, run SQL queries, apply visual filters, and create interactive charts. Built with FastAPI, DuckDB, React, and Material-UI.

Notifications You must be signed in to change notification settings

BethanyWeisberg/querypad

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data Query Tool

A modern, locally-hosted data analysis platform that allows you to load, query, and visualize structured data files with ease. Built with React, FastAPI, and DuckDB for a Snowflake-like experience.

Features

  • Smart File Analysis: Automatically detects file type, delimiter, and data types
  • Flexible Data Loading: Support for CSV, TSV, TXT, and JSON files
  • Header Detection: Auto-detects headers with option to customize
  • Column Management: Rename columns or let the tool generate names
  • SQL Query Interface: Write and execute SQL queries with Monaco editor
  • Visual Filters: Point-and-click filtering without writing SQL
  • Data Visualization: Create bar charts, line charts, and pie charts
  • Export Functionality: Export query results to CSV or JSON
  • Modern UI: Clean, responsive Material-UI interface

Tech Stack

Backend

  • FastAPI: Modern Python web framework
  • DuckDB: High-performance analytical database
  • Pandas: Data manipulation and analysis
  • Chardet: Character encoding detection

Frontend

  • React: UI library
  • Material-UI: Component library
  • Recharts: Data visualization
  • Axios: HTTP client
  • Vite: Build tool

Project Structure

bethany-project/
├── backend/
│   ├── app/
│   │   ├── __init__.py
│   │   ├── main.py              # FastAPI application
│   │   ├── models.py            # Pydantic models
│   │   ├── file_analyzer.py     # File analysis logic
│   │   └── database.py          # DuckDB integration
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── components/
│   │   │   ├── FileUpload.jsx
│   │   │   ├── DataPreview.jsx
│   │   │   ├── QueryInterface.jsx
│   │   │   ├── FilterInterface.jsx
│   │   │   └── DataVisualization.jsx
│   │   ├── services/
│   │   │   └── api.js
│   │   ├── App.jsx
│   │   └── main.jsx
│   ├── index.html
│   ├── vite.config.js
│   └── package.json
└── README.md

Installation

Prerequisites

  • Python 3.8+
  • Node.js 16+
  • npm or yarn

Backend Setup

  1. Navigate to the backend directory:
cd backend
  1. Create a virtual environment:
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
  1. Install dependencies:
pip install -r requirements.txt
  1. Run the FastAPI server:
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

The API will be available at http://localhost:8000

Frontend Setup

  1. Navigate to the frontend directory:
cd frontend
  1. Install dependencies:
npm install
  1. Start the development server:
npm run dev

The application will be available at http://localhost:3000

Usage

1. Upload a File

  • Drag and drop a file or click to browse
  • Supported formats: CSV, TSV, TXT, JSON
  • The tool will automatically analyze the file

2. Review Analysis

The tool will display:

  • File type and delimiter
  • Number of rows and columns
  • Detected column names and data types
  • Preview of first 10 rows

3. Configure Settings

  • Toggle "File has header row" if needed
  • Rename columns if no header or ignoring header
  • Click "Load Data" to proceed

4. Query Data

SQL Query Interface:

  • Write SQL queries in the editor
  • Press Cmd/Ctrl + Enter to execute
  • View results in a table
  • Export results to CSV or JSON

Visual Filters:

  • Add multiple filter conditions
  • Choose column, operator, and value
  • Apply filters to see results
  • Export filtered data

Data Visualization:

  • Select chart type (bar, line, pie)
  • Choose X-axis column
  • Select aggregation (count, sum, avg, min, max)
  • Choose Y-axis for numeric aggregations
  • Generate interactive charts

API Endpoints

POST /api/analyze-file

Upload and analyze a file

POST /api/load-data

Load analyzed data into DuckDB

POST /api/query

Execute SQL query

POST /api/filter

Apply visual filters

GET /api/export/{session_id}

Export data in CSV or JSON format

DELETE /api/session/{session_id}

Delete session and cleanup resources

Examples

SQL Query Examples

-- Get all data
SELECT * FROM data_table LIMIT 100

-- Filter by condition
SELECT * FROM data_table WHERE column_1 > 50

-- Aggregation
SELECT column_1, COUNT(*) as count
FROM data_table
GROUP BY column_1
ORDER BY count DESC

-- Join (if multiple tables loaded)
SELECT a.*, b.*
FROM data_table a
JOIN other_table b ON a.id = b.id

Visual Filter Examples

  1. Equals: column = 'value'
  2. Contains: Find rows where column contains text
  3. Greater Than: Numeric comparisons
  4. Is Null: Find missing values

Development

Backend Development

The backend uses FastAPI with automatic API documentation:

  • Swagger UI: http://localhost:8000/docs
  • ReDoc: http://localhost:8000/redoc

Frontend Development

The frontend uses Vite for fast development:

  • Hot module replacement (HMR)
  • Fast refresh for React components

Building for Production

Backend

cd backend
pip install -r requirements.txt
uvicorn app.main:app --host 0.0.0.0 --port 8000

Frontend

cd frontend
npm run build
npm run preview

Troubleshooting

Backend Issues

Import errors:

# Make sure virtual environment is activated
source venv/bin/activate
pip install -r requirements.txt

Port already in use:

# Change the port
uvicorn app.main:app --reload --port 8001

Frontend Issues

Dependencies not installed:

rm -rf node_modules package-lock.json
npm install

CORS errors:

  • Make sure backend is running on port 8000
  • Check vite.config.js proxy settings

Future Enhancements

  • Multiple file support
  • Save queries and filters
  • More chart types (scatter, area, etc.)
  • Export to Excel
  • Database connection support
  • Query history
  • Data profiling and statistics
  • Column type conversion
  • Advanced joins across datasets

License

MIT License

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

About

Modern local data query and visualization tool. Upload CSV/JSON files, run SQL queries, apply visual filters, and create interactive charts. Built with FastAPI, DuckDB, React, and Material-UI.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •