Skip to main content

Personal notes on LLMs

· 4 min read
Benjamin Tanone
Benjamin Tanone

I've been using Cursor and ChatGPT for a while now, and I've developed a decent amount of LLM-based features. Naturally, I wanted to extensively document my experience with using LLMs, but I've frankly given up at this stage. There's so much to talk about, and not enough time to talk about them all.

So I present to you: my short note on LLMs. Hopefully this would be better for you - o' dear reader - than me not publishing anything because, y'know, scope creep.

There's a few themes that I see here after using and developing on LLMs for a while:

  • LLMs will never replace humans. They're fake and there's an internal repulsion that people feel if they discover that they've been duped (to be written for another article).
  • LLMs will accelerate humans' productivity. It turns out there's a lot of tedium with progressing the human race. Y'know, like recording meeting notes and such.
  • Humans are valuable because they're masters of context. We're all professional context switchers, and we can decipher context from all sorts of sensory inputs. We can tell that our partners are angry when we see their eyebrows tick up - LLMs can't do that!
  • So... Make LLMs do tasks that have clear context and limited scope.

In no specific order, this is the pros and cons that I’ve seen so far of using LLMs:

  • Removes tedious tasks from your TODO list. Like, I get it, writing changelogs are important, but you can have an AI summarize the changelog for you and you can edit the final content for brevity and accuracy.
  • Great at solving one-shot problems, especially in greenfield context. For example: “Write me a NextJS app that allows me to record my TODOlist”. It won’t be 100% right, but it’ll get you 80% of the way.
  • Very stupid at gathering context. It is cut off from the real world. Good luck getting Mark from the marketing department to talk with it.
  • Very stupid at evaluating context. This has been proven by Gemini being injected with misinformation when it tries to read fake forum posts from scammers.
  • Text-based prompting is still very slow (for me).
  • Very stupid at preserving context. “Memories” somewhat get around this, but it’s nowhere near as advanced as the way humans can recall things.
  • Very stupid at learning. It’s great at adapting, but I feel like there is a difference between adaptation and actual learning. The former doesn’t build on anything; the latter incrementally improves the model by itself.
  • Absolutely awesome at summaries and distilling insights from pieces of text. Seriously: if I had to pick one good thing that came out from the LLM boom, it has to be this part. It sucks up 50% of my time, especially things like reading docs and finding the thing that I’m actually looking for. Asking ChatGPT to summarise things has helped me virtually eliminate the chore and allowed me to get into the content that is relevant to me immediately.
  • Planning is important in order to get any LLM to not hallucinate on complex tasks. I feel like Cursor has done this pretty well in terms of integrating it with coding workflows, but the general concept is independent of any LLM tools (I used chain-of-thoughts and planning when getting it to do copywriting).
  • Very stupid at solving non-text things. This isn’t really the fault of LLMs - and frankly I do feel like it’s a solvable in a decade or so once we spend enough time and money on it - but it does mean that LLMs can’t do complex, multimodal things that humans can do. For example, my LLM got stuck in a loop trying to execute my DB migration. Turns out I just forgot to turn on my Postgres container.
  • Will absolutely try to please you, which can be bad if you don’t manage it. There’s a few papers talking about this (lookup “RLHF sycophancy”). The 2 most dangerous results that it can lead you to:
    • “Yes you are absolutely right in that your boss sounds like a shithead” just because you vented that your boss yelled at you (because you caused a global outage, but you omitted that because you were venting).
    • “I will absolutely try to annihilate the human race for you” even when, clearly, it lacks the nuclear warheads to do so.