Hello I'm

Valmeekam Karthik

Scroll down
For a tour into my world

I am a

I was born in Hyderabad, India but my childhood was shaped by two vastly different places; the sub-tropical highlands of Addis Ababa, Ethiopia (from K.G. to 5th Grade) and the sun-scorching deserts of Ibra, Oman (5th to 10th Grade). Whereever I went, I carried my love for stories and thought of myself as a storyteller.

After my frivolous attempt at cracking the JEE, I joined Vellore Institute of Technology to explore and tell stories about Computers and Algorithms. During my Masters at Arizona State University, I discovered that rigorously formalized ideas are great stories to tell! That drove me to pursue research under the guidance of Prof. Subbarao Kambhampati.

Currently, my world revolves around Large Language Models (LLMs) and Planning. While many were caught up in the hype of LLMs’ supposed planning capabilities—or panicking about them taking over the world—I was busy helping my group cut through the noise and ground the conversation. My efforts earned me the prestigious IBM PhD Fellowship Award (2024) in recognition of my work on the reasoning and planning abilities of LLMs.

When I’m not immersed in research, I channel my creativity into screenwriting, dancing, photography, and playing the keyboard. I also enjoy listening to a wide variety of music, ranging from heavy metal to carnatic ragas.

News

Apr 12, 2025 "(How) Do reasoning models reason?" will appear in Annals of the New York Academy of Sciences, 2025!
Mar 13, 2025 "A Systematic Evaluation of the Planning and Scheduling Abilities of the Reasoning Model o1" will appear in TMLR, 2025!
Jan 21, 2025 "On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks" has been accepted to ICLR 2025!
Jan 01, 2025 Recipient of the 2024 IBM PhD Fellowship Award! Thanks to IBM Research for this recognition of my ongoing work on the planning/reasoning abilities of LLMs.
Sept 25, 2024 "Chain of thoughtlessness? An Analysis of CoT in Planning" has been accepted to NeurIPS 2024 as a poster!
Sept 23, 2024 New preprint about OpenAI's o1 () evaluation on PlanBench is now available on arXiv
May 1, 2024 "LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks" has been accepted to ICML 2024 as a Spotlight Position Paper!

Passing through the Academic Jungle

2021 - Present.

Arizona State University.

Tempe, Arizona, US.

PhD in Computer Science.
GPA* 4.0/4.0
Graduate Research Assistant at Yochan under Prof. Subbarao Kambhampati

2019-2021.

Arizona State University.

Tempe, Arizona, US.

M.S. in Computer Science.
GPA 4.0/4.0
Graduate Research Assistant at Yochan under Prof. Subbarao Kambhampati
Thesis: A Study of Explainable Decision Support for Longitudinal Sequential Decision Making

2015-2019.

Vellore Institute Of Technology.

Vellore, Tamilnadu, India.

B.Tech. in Computer Science and Engineering.
GPA 9.19/10.0
Research Aide under Prof. Anuradha J
Vice-Chair Management of Codechef-VIT

2013-2015.

Narayana Junior College.

Hyderabad, Telangana, India.

Intermediate (11th & 12th Std).
Percentage: 95.1%

2007-2013.

Indian School Ibra.

Ibra, Oman.

10th Std.
GPA 9.8/10.0
Computer Whizkid
Best Dancer

Learning - A continuous process

May - Aug 2024.

Amazon Science.

Palo Alto, California, US.

Applied Scientist Intern
Worked on LLMs and Planning.

Skills

Programming Languages Python, C++, JavaScript, HTML/CSS, SQL, Bash, SAP-ABAP
Frameworks HuggingFace Transformers, LoRA PEFT, Flask, Bootstrap, jQuery, D3.js, Pytorch, Tensorflow, Keras, OpenCV, ROS
Tools Git, Docker, Kubernetes, AWS, GCP, Jupyter, Colab, Adobe Photoshop, Adobe Premiere Pro, Adobe Lightroom, Blender, Gazebo, Unreal Engine

Execution - That's how dreams become reality

Publications.

*Formalizing Curiosity*

  1. (How) Do reasoning models reason?

    Subbarao Kambhampati, Kaya Stechly, Karthik Valmeekam,
    Annals of the New York Academy of Sciences 2025

  2. A Systematic Evaluation of the Planning and Scheduling Abilities of the Reasoning Model o1

    Karthik Valmeekam*, Kaya Stechly*, Atharva Gundawar, Subbarao Kambhampati
    TMLR 2025
    *equal contribution

  3. On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks

    Kaya Stechly*, Karthik Valmeekam*, Subbarao Kambhampati
    ICLR 2025
    *equal contribution

  4. Planning in Strawberry Fields: Evaluating and Improving the Planning and Scheduling Capabilities of LRM o1

    Karthik Valmeekam*, Kaya Stechly*, Atharva Gundawar, Subbarao Kambhampati
    A Version to appear in TMLR, 2025.
    *equal contribution

  5. LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBench

    Karthik Valmeekam*, Kaya Stechly*, Subbarao Kambhampati
    arXiv Preprint, Sept 2024.
    *equal contribution
    (Coverage in The Decoder, Top paper in Alpha Signal Newsletter)

  6. Chain of Thoughtlessness? An Analysis of CoT in Planning

    Kaya Stechly*, Karthik Valmeekam*, Subbarao Kambhampati
    NeurIPS 2024
    *equal contribution
    (Coverage in TechTalks)

  7. LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks

    Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Kaya Stechly, Mudit Verma, Siddhant Bhambri, Lucas Saldyt, Anil Murthy
    ICML 2024 (Position Paper)

  8. Robust Planning with LLM-Modulo Framework: Case Study in Travel Planning

    Atharva Gundawar, Mudit Verma, Lin Guan, Karthik Valmeekam, Siddhant Bhambri, Subbarao Kambhampati
    arXiv Preprint, May 2024.

  9. On the role of large language models in planning

    Subbarao Kambhampati, Karthik Valmeekam, Lin Guan
    Tutorial at AAAI 2024

  10. On the Planning Abilities of Large Language Models--A Critical Investigation

    Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, Subbarao Kambhampati
    NeurIPS 2023 (Spotlight) (Earlier at KLR@ICML 2023)

  11. Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning

    Lin Guan*, Karthik Valmeekam*, Sarath Sreedharan, Subbarao Kambhampati
    NeurIPS 2023 (Earlier at KLR@ICML 2023)
    *equal contribution

  12. PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, Subbarao Kambhampati
    NeurIPS 2023 (Benchmark Track)

  13. Can Large Language Models Really Improve by Self-critiquing Their Own Plans?

    Karthik Valmeekam, Matthew Marquez, Subbarao Kambhampati
    FMDM@NeurIPS 2023

  14. Relative Behavioral Attributes: Filling the Gap between Symbolic Goal Specification and Reward Learning from Human Preferences

    Lin Guan, Karthik Valmeekam, Subbarao Kambhampati
    ICLR 2023

  15. On the role of large language models in planning

    Subbarao Kambhampati, Karthik Valmeekam, Matthew Marquez, Lin Guan
    Tutorial at ICAPS 2023

  16. Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change)

    Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, Subbarao Kambhampati
    FMDM@NeurIPS 2022

  17. RADAR-X: An Interactive Interface Pairing Contrastive Explanations with Revised Plan Suggestions

    Karthik Valmeekam, Sarath Sreedharan, Sailik Sengupta, Subbarao Kambhampati
    ICAPS 2022; AAAI 2021 Demos

  18. Opinion Mining on Emojis using Deep Learning Techniques

    Karthik Valmeekam, Dheeraj Nair, J Anuradha
    Procedia Computer Science, 2018

Projects

*Get the engine started.*

  1. LLMs for Planning Loading stars...
  2. Reinforcement Learning for Sequence: An Imperfect Information Board Game
  3. High Speed Autonomous Drifting using Deep Reinforcement Learning
  4. Implementation and Evaluation of Nash Q Learning
  5. Parsing and Writing AI Planning models (represented in PDDL)
  6. Paws in Space - Data Visualization
  7. How to Open in Chess? - Data Visualization

*All the background images are captured by me
(with the obvious exception of the first one).