Add new submission for SWE-bench evaluation 20251009_MAYA_claude-sonn…

[yashjakhar-2929]

MAYA: Multi Agent Yottaframe by Adya

MAYA is a modular, multi-agent debugging system that can plug into any framework or development pipeline. It decomposes bug resolution into four coordinated roles: Classification (identify error type), Analyzer (trace root cause), Planner (design precise edit instructions), and Solver (generate clean, Git-ready patches). This structured workflow transforms debugging from a manual, opaque process into a transparent and reproducible pipeline.

Unlike monolithic debuggers, MAYA is framework-agnostic and self-healing. It produces minimal patches that preserve existing functionality while iteratively resolving errors. Every step emits auditable artifacts — from root cause summaries to unified diffs — making MAYA not just a tool for fixing bugs, but a universal debugging fabric that improves reliability, traceability, and developer velocity.

Performance

Submission summary for 20251009_MAYA_claude-sonnet-4-5-20250929 on SWE-bench lite split

Resolved 155 instances (51.67%)

Resolved by Repository

• astropy/astropy: 3/6 (50.0%)
• django/django: 44/114 (38.6%)
• matplotlib/matplotlib: 9/23 (39.13%)
• mwaskom/seaborn: 1/4 (25.0%)
• pallets/flask: 0/3 (0.0%)
• psf/requests: 0/6 (0.0%)
• pydata/xarray: 0/5 (0.0%)
• pylint-dev/pylint: 3/6 (50.0%)
• pytest-dev/pytest: 17/17 (100.0%)
• scikit-learn/scikit-learn: 20/23 (86.96%)
• sphinx-doc/sphinx: 11/16 (68.75%)
• sympy/sympy: 47/77 (61.04%)
  ==================================================
  Resolved by Time
• 2012: 0/1 (0.0%)
• 2014: 0/3 (0.0%)
• 2015: 0/1 (0.0%)
• 2016: 2/4 (50.0%)
• 2017: 11/16 (68.75%)
• 2018: 12/21 (57.14%)
• 2019: 37/59 (62.71%)
• 2020: 35/66 (53.03%)
• 2021: 18/42 (42.86%)
• 2022: 24/57 (42.11%)
• 2023: 16/30 (53.33%)

Details

Report
Site

Checklist

• Is a pass@1 submission (does not attempt the same task instance more than once)
• Does not use SWE-bench test knowledge (PASS_TO_PASS, FAIL_TO_PASS)
• Does not use the hints field in SWE-bench
• Does not have web-browsing OR has taken steps to prevent lookup of SWE-bench solutions via web-browsing

[yashjakhar-2929]

Hi @john-b-yang, I wanted to follow up on the submission I made last month as I haven’t received a response yet. Please let me know if you need any additional information from my side. Thanks!

[john-b-yang]

Hi guys, i took a look at the technical report - https://adya.ai/blogs/maya-multi-agentic-way-build-apps

There's no mention of SWE-bench or how you actually ran your system on SWE-bench? I see your system is closed source, which is fine, but this technical report feels a bit threadbare. Can you describe how your system actually works on SWE-bench? See other technical reports from other PRs if you need a reference.

[yashjakhar-2929]

Hi @john-b-yang , Thanks for the feedback! I've updated the report(https://adya.ai/blogs/maya-multi-agentic-way-build-apps) to include details on how the system was run on SWE-bench and added the missing explanations you pointed out. If there’s anything else you’d like clarified or if more information would be helpful, just let me know — happy to add it.