AutoOpt is an end-to-end automated optimization framework that takes the image of a mathematical program (optimization problem) and solves it without human intervention.

Currently available modules:

• M1 · Image → LaTeX (MER) — Hybrid CNN‑Transformer model for converting image of a mathematical program into LaTeX script.
• M2 · LaTeX → Pyomo — Fine‑tuned code generation model for translating LaTeX script to PYOMO (modelling language) code.

M3 · Solver — Understands the PYOMO code and solves the mathematical program using a bilevel decomposition approach (To be released soon)

The AutoOpt framework unifies perception, translation, and solving of optimization models through three sequential modules:

• M1 (Image→LaTeX): Recognizes handwritten/printed optimization models using a hybrid ResNet + Swin Transformer encoder and mBART decoder.
• M2 (LaTeX→Pyomo): Converts the LaTeX into clean Pyomo scripts using a fine‑tuned LLM.
• M3 (Solver): Integrates a Bilevel Optimization‑Based Decomposition (BOBD) solver (coming soon).

Dataset: AutoOpt‑11k on Kaggle Paper: AutoOpt: A Dataset and a Unified Framework for Automating Optimization Problem Solving

git clone https://github.com/Shobhit1201/AutoOpt.git
cd AutoOpt/Module_M1
pip install -U pip wheel setuptools
pip install -r requirements.txt

Make sure your PyTorch version matches your CUDA environment. Refer to the PyTorch installation guide.

_{HW: Handwritten; PR: Printed; HW+PR: Handwritten+Printed.}

!gdown --fuzzy "https://drive.google.com/file/d/1WqfAv0b4FwR8-xqyDLXHTFxBMCas2Fej/view?usp=drive_link" -O checkpoints/m1_weights.pt
python tools/m1_infer.py --image path/to/image.png --checkpoint checkpoints/weights.pth

Prepare your dataset similar to AutoOpt‑11k and execute:

python tools/train_experiment.py --config_file config/base.yaml --phase train

Use the hosted Colab notebook to perform inference or fine‑tuning directly:

AutoOpt/
├── Module M1/
│   ├── nougat‑latex‑ocr/           # MER model code and training utilities
│   ├── Module_M1_Inference.ipynb   # Colab‑ready notebook for inference & finetuning
│
├── Other MER models/
│   ├── Original_Nougat/
│   ├── ChatgptComparison.ipynb
│   └── GeminiComparison.ipynb
│
├── Module_M2.ipynb                 # LaTeX → Pyomo translation notebook
└── README.md

The Colab notebook Module_M1_Inference.ipynb allows users to perform end‑to‑end inference or fine‑tuning without setup.

Access: Kaggle Dataset

• 11,554 images (handwritten + printed) labeled with verified LaTeX.
• 1,018 paired Pyomo scripts for direct LaTeX→code training. The dataset spans linear, nonlinear, convex, nonconvex, and stochastic optimization problems across domains. See AutoOpt_Paper.pdf for detailed composition and statistics.

@inproceedings{Sinha2025AutoOpt,
  title     = {AutoOpt: A Dataset and a Unified Framework for Automating Optimization Problem Solving},
  author    = {Ankur Sinha and Shobhit Arora and Dhaval Pujara},
  booktitle = {Proceedings of the 39th Annual Conference on Neural Information Processing Systems (NeurIPS 2025)},
  year      = {2025},
  note      = {arXiv preprint arXiv:2510.21436},
  url       = {https://arxiv.org/abs/2510.21436}
}

Released under the MIT License.

⭐ If you find this repository useful, please consider leaving a star on GitHub!