Should we Adopt LLM Agents for Research?

I had a long discussion with a colleague on adopting LLM "agents" for ones research. It all started when he mentioned that the free student Copilot no longer offers Opus, and now he somehow has to get access to a Claude subscription.

He felt that not using LLMs could put one at a significant disadvantage compared to people who would. Not being able to iterate quickly on ideas or flesh out full 10k+ LOC projects in days would be a significant inhibitor to publishing, and in the current climate (publish or perish) this would mean one would eventually perish and not be able to complete ones PhD successfully.

Not being able to act quickly enough on ones ideas would mean someone else would. If you can't publish your ideas today, someone will do it next week, because it just takes so little time to publish today.

I was very perplexed by this opinion.

First, it's important to remember that the purpose of a PhD is not to publish 5 or 15 or 50 papers. The purpose of a PhD is to learn, after all it is possible to complete a PhD with 0 published papers¹.

Secondly, and this is even more important, adopting LLMs as a fundamental tool in your PhD is a very dangerous thing to do. Everything you decide to put on the critical path towards your PhD needs to be a very conscious decision. In my personal opinion, adopting a heavily subsidised service run by a for-profit company with a very fast-and-loose attitude that was built upon a general disregard for intellectual property rights sits squarely in the hell no camp in this regard. Just a few points I would be concerned about:

• It is well known now that these business models rely on first capturing the market, and then squeezing customers for what they have. I doubt that I would be able or willing to spend time looking for an alternative when my LLM provider raises my subscription by 10x during any stressful period
• It is also well documented that model providers can degrade model performance at will, so it's entirely possible that the workflows that worked fine until yesterday suddenly start failing

But most important of all, I believe that good research has very little to do with how many lines of code you can write in a day. It's about seeing the things in a field that others miss, connecting dots that seem unrelated, refining ideas, etc. These are skills that one should learn as part of their PhD. Paying for access to a companies machine to do these things for you appears antithetical to the whole concept of a PhD for me.

Not to mention the whole ethical and environmental side of the argument.

Caveat: Some people are working on projects in their PhD where they need to use LLM "agents" to have any chance of making any significant progress, either because their colleagues left them 10k+ LOC to debug, or because the amount of work required to finishing the project on time is simply too much for a single human.

I thought about this for quite some time, but I cannot come up with any single piece of advice that would help them in this position. This is a huge failure on part of the supervisor, but that is neither helpful not actionable at this point. I guess people should be careful when choosing a supervisor?