Skip to main content

Ai

Codex Evaluation

Recently I’ve wanted to understand more about using AI/LLM technologies to assist with code creation. I have in the past played with using chatbot style vibe coding. This is where one describes what is desired and the LLM produces an entire program. However I have found that subsequent iterations with further features hard to keep copy and pasting the entire source file, assuming a single source file is required, to be difficult. I have not really tried with in editor coding. However for the most part the “help me write this one function” to be smaller picture that I desired. So when I found out about Codex from ChatGPT I was really excited to understand how this works. For me it’s the integration with GitHub that make’s it really useful. In stead of having to write an ever larger perfect prompt that covers the entire aspect of the functionality, one builds up functionality one GitHub pull request at a time. I liked this concept because it’s also how I develop things, eg I start small with basic functionality and then iterate. Given that each unit of work is a pull request I can inspect each change to the code base’s multiple files to see if it’s making the desired changes.