Things I learned about LLMs in 2024
I was reading Simon Willison - Things we learned about LLMs in 2024 and it made me realize this would be a good moment to reflect on my own learnings on LLMs in 2024.
-
LLMs were used in 3/4 of the four final/graduation projects at Fontys Venlo that I help review & evaluate in the first half of the year.
This required me to "catch-up" on technology and terminology that I was not yet familiar with in order to evaluate properly. It is also good to understand that software technology and tools are constantly evolving and this brings typical challanges and opportunities.
e.g. there were a fair amount of problems across students with respect to getting all the software and right dependencies deployed correctly.
-
Using GitHub CoPilot (as part of a "pilot" at work) was both exciting and frustrating sometimes.
At unexpecting moments it worked like IntelliSense on steroids, seemed to "know" that I was in the middle of refactoring code (and determined possible missing testcases due to that refactoring). It's almost like "rubber duck debugging" with a duck that actually talks back.
At other moments, I was struggling with thinking of how to best write the prompt that would give me the (better) correct results.
(I guess there is a reason for the term "prompt engineering")
And I can certainly understand that not everyone will question the answers that an LLM will give, which makes me wonder about the future of software engineering and the impact on personal learning.
-
From Simon Willison's article, the following learnings I found particular interesting
- increased context lengths: since longer context lengths increase the scope of problems that you can solve with an LLM
- running models locally is getting more and more powerfull and efficient: the models are getting better (to be run on limited hardware) and the Apple M series hardware is very capable of running models
I'm very curious of what will happen during 2025 w.r.t. LLMs and how I will be using them.