Hello I'm

Valmeekam Karthik

Scroll down
For a tour into my world

I am a

I was born in Hyderabad, India but I grew up in two quite opposite places (weather-wise); Addis Ababa, Ethiopia (from K.G. to 5th Std) and Ibra, Oman (5th to 10th Std). I have always loved stories and thought of myself as a storyteller. After my frivolous attempt at JEE, I joined Vellore Institute of Technology to learn and tell stories about Computer Science and Engineering.

During my Masters at Arizona State University, I realized that rigorously formalized ideas are great stories to tell!
Hence I opted to pursue PhD under Prof. Subbarao Kambhampati. Currently, Large Language Models (LLMs) and Planning are keeping me occupied!

In my free time, I write screenplays, dance, learn photography, play my keyboard and listen to a wide variety of music ranging from heavy metal to carnatic ragas.

News

Sept 25, 2024 "Chain of thoughtlessness? An Analysis of CoT in Planning" has been accepted to NeurIPS 2024 as a poster!
Sept 23, 2024 New preprint about OpenAI's o1 () evaluation on PlanBench is now available on arXiv
May 1, 2024 "LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks" has been accepted to ICML 2024 as a Spotlight Position Paper!

Passing through the Academic Jungle

2021 - Present.

Arizona State University.

Tempe, Arizona, US.

PhD in Computer Science.
GPA* 4.0/4.0
Graduate Research Assistant at Yochan under Prof. Subbarao Kambhampati

2019-2021.

Arizona State University.

Tempe, Arizona, US.

M.S. in Computer Science.
GPA 4.0/4.0
Graduate Research Assistant at Yochan under Prof. Subbarao Kambhampati
Thesis: A Study of Explainable Decision Support for Longitudinal Sequential Decision Making

2015-2019.

Vellore Institute Of Technology.

Vellore, Tamilnadu, India.

B.Tech. in Computer Science and Engineering.
GPA 9.19/10.0
Research Aide under Prof. Anuradha J
Vice-Chair Management of Codechef-VIT

2013-2015.

Narayana Junior College.

Hyderabad, Telangana, India.

Intermediate (11th & 12th Std).
Percentage: 95.1%

2007-2013.

Indian School Ibra.

Ibra, Oman.

10th Std.
GPA 9.8/10.0
Computer Whizkid
Best Dancer

Learning - A continuous process

May - Aug 2024.

Amazon Science.

Palo Alto, California, US.

Applied Scientist Intern
Worked on LLMs and Planning.

Skills

Programming Languages Python, C++, JavaScript, HTML/CSS, SQL, Bash, SAP-ABAP
Frameworks HuggingFace Transformers, LoRA PEFT, Flask, Bootstrap, jQuery, D3.js, Pytorch, Tensorflow, Keras, OpenCV, ROS
Tools Git, Docker, Kubernetes, AWS, GCP, Jupyter, Colab, Adobe Photoshop, Adobe Premiere Pro, Adobe Lightroom, Blender, Gazebo, Unreal Engine

Execution - That's how dreams become reality

Publications.

*Formalizing Curiosity*

  1. LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBench

    Karthik Valmeekam*, Kaya Stechly*, Subbarao Kambhampati
    arXiv Preprint, Sept 2024.
    *equal contribution
    (Coverage in The Decoder, Top paper in Alpha Signal Newsletter)

  2. Chain of Thoughtlessness? An Analysis of CoT in Planning

    Kaya Stechly*, Karthik Valmeekam*, Subbarao Kambhampati
    NeurIPS 2024
    *equal contribution
    (Coverage in TechTalks)

  3. LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks

    Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Kaya Stechly, Mudit Verma, Siddhant Bhambri, Lucas Saldyt, Anil Murthy
    ICML 2024 (Position Paper)

  4. Robust Planning with LLM-Modulo Framework: Case Study in Travel Planning

    Atharva Gundawar, Mudit Verma, Lin Guan, Karthik Valmeekam, Siddhant Bhambri, Subbarao Kambhampati
    arXiv Preprint, May 2024.

  5. On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks

    Kaya Stechly, Karthik Valmeekam, Subbarao Kambhampati
    arXiv Preprint, Feb 2024.

  6. On the role of large language models in planning

    Subbarao Kambhampati, Karthik Valmeekam, Lin Guan
    Tutorial at AAAI 2024

  7. On the Planning Abilities of Large Language Models--A Critical Investigation

    Karthik Valmeekam, Matthew Marquez, Sarath Sreedharan, Subbarao Kambhampati
    NeurIPS 2023 (Spotlight) (Earlier at KLR@ICML 2023)

  8. Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning

    Lin Guan*, Karthik Valmeekam*, Sarath Sreedharan, Subbarao Kambhampati
    NeurIPS 2023 (Earlier at KLR@ICML 2023)
    *equal contribution

  9. PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, Subbarao Kambhampati
    NeurIPS 2023 (Benchmark Track)

  10. Can Large Language Models Really Improve by Self-critiquing Their Own Plans?

    Karthik Valmeekam, Matthew Marquez, Subbarao Kambhampati
    FMDM@NeurIPS 2023

  11. Relative Behavioral Attributes: Filling the Gap between Symbolic Goal Specification and Reward Learning from Human Preferences

    Lin Guan, Karthik Valmeekam, Subbarao Kambhampati
    ICLR 2023

  12. On the role of large language models in planning

    Subbarao Kambhampati, Karthik Valmeekam, Matthew Marquez, Lin Guan
    Tutorial at ICAPS 2023

  13. Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change)

    Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, Subbarao Kambhampati
    FMDM@NeurIPS 2022

  14. RADAR-X: An Interactive Interface Pairing Contrastive Explanations with Revised Plan Suggestions

    Karthik Valmeekam, Sarath Sreedharan, Sailik Sengupta, Subbarao Kambhampati
    ICAPS 2022; AAAI 2021 Demos

  15. Opinion Mining on Emojis using Deep Learning Techniques

    Karthik Valmeekam, Dheeraj Nair, J Anuradha
    Procedia Computer Science, 2018

Projects

*Get the engine started.*

  1. LLMs for Planning
  2. Reinforcement Learning for Sequence: An Imperfect Information Board Game
  3. High Speed Autonomous Drifting using Deep Reinforcement Learning
  4. Implementation and Evaluation of Nash Q Learning
  5. Parsing and Writing AI Planning models (represented in PDDL)
  6. Paws in Space - Data Visualization
  7. How to Open in Chess? - Data Visualization

*All the background images are captured by me
(with the obvious exception of the first one).