The rise of LLMs has led to the development of sophisticated AI that can generate code that compiles and makes sense. This technology is being utilized by developers and companies to increase productivity, write better code, and detect issues earlier. In this article, we explore three major popular consumer LLM-based AI agents that assist in writing code and examine their strengths and weaknesses.

Repl.it.

Repl.it is a startup that pioneered web-based IDEs before many others and has since done an excellent job building a robust web-based IDE and associated tools. I have been a satisfied paying customer for quite some time.

The strength of this product lies in its well-integrated web-based IDE, which allows you to write, compile, run, and publish code. From my understanding, it's quite suitable for small projects and teams but may not be entirely enterprise-ready.

I have built several NextJS-based web applications using Repl.it.

Repl.it offers a coding assistant that can complete your code, write code, and debug issues through a chat-like interface. The company claims it is powered by an LLM trained on publicly available code.

In my experience, I found it to be remarkably similar to GitHub's Copilot, albeit slightly less helpful. However, if you are a Repl.it user, it is certainly worth the investment, as I used it frequently and relied on it for generating a substantial amount of code.

I wouldn't recommend Repl.it solely for its AI capabilities. However, if you have other reasons to use Repl.it, their AI assistant is a valuable feature that's worth the money, in my opinion.

Github Copilot

Github Copilot is essentially an OpenAI-based LLM. It features a chat-like interface where you can directly ask it to write code for you, or it can be integrated with your IDEs like Github Codespaces (a web-based VS Code) or VS Code running on your machine.

In my opinion, Copilot is quite impressive, especially for well-known and popular languages like PHP and Java.

However, it did not perform well with Kotlin. It consistently failed to sort imports and often duplicated them, making the code even worse. But when I was writing Spring Boot, I was pleasantly surprised by how well the bot recognized common design patterns and how proactive it was in generating code following all best practices.

Overall, Github Copilot comes across as quite advanced and can "wow" you from time to time. However, it also has many kinks that can be frustrating, such as messing up imports in my Kotlin code.

Google IDX

IDX is basically Google's answer to Github Copilot. It is still in experimental stage but a free product and Google is clearly a late entrant to this space. However, from what we know Google internally uses a similar tool to help its engineers write code and it has been the primary and most loved tool internally. Which also means it is somewhat enterprise tested and Google has all the pieces in order. All they need to do is build a coherent product around this.

From my initial experience with IDX, it is more focused on improving developer productivity rather than to wow you. Which means it is pretty good at everything it does but it wont try to surprise you as much as Github copilot.

For example, it was extremely good at sorting my Typescript imports. It was very good at suggesting unit test names and just brilliant when it came to suggesting method names and generating boilerplates for Angular controllers and views.

It especially excelled at Dart and Java, it felt like writing code with another human being who thinks just like you.

The IDX IDE itself is pretty subpar despite being just a VS Code web based version. While the underlying AI is pretty good, the IDE itself needs massive improvement or they should simply release Google IDX AI as an extension for VS Code and let individual developers simply use their own IDE and benefit from Google's AI.

Pricing

Repl.it AI offers a free plan and a paid plan. The free plan comes with basic AI but what you will need is their paid plan for real useful AI help. It costs around $120 a year. It will certainly pay for itself by saving you a lot of time.

Github Copilot is $100 per year for Individuals and also is pretty useful and will pay for itself in terms of saved time.

Google IDX is free right now but is in preview mode. Google might shut it down entirely like many other products and might offer it as VS code extension or as some kind of Google Cloud service. It is not clear what that path would look like but it is certainly worth trying but I prefer Github Copilot over it.

Comparison Chart

Conclusion

AI code generators and assistants are still in their early stages, but even in their current state, they are impressive and helpful, even if you have to pay for them. The product that succeeds in the future will likely be the one that provides a more seamless experience with the user's IDE of choice or creates a specialized IDE that is worth adopting solely for the superior AI. The key to success will likely be how well the AI integrates with the rest of the developer experience.