Project with alternative approach towards the OpenAI's tools paradigm

[khromalabs]

Hi,

Since a few months ago I've been developing a project which has been using since the very beginning an alternative approach to the 'tools' paradigm, (which now is being implemented in llama.cpp as well) instead of being a characteristic available trough the LLM server the client instructs the LLM about some posible actions, and then a command to request them when needed. Basically a bit of prompt and then a middle-ware which intercepts the command and processes the info trough the plugin or skill. I found that this approach works really well even with models in the 14B range (I didn't test with smaller models but I'll do eventually). Lately I managed to make it work in streaming mode as well, so it doesn't block the conversation at any moment. My project has already developed a few dozens of LLM action plugins which I called skills, one of the plugins I proud the most is a meta-search engine (searches in 4 engines at once) which works really well along the LLM information classification capabilities.

I admit I took this route doing independent research about the possibilities of this and at that time I wasn't aware of this tools architecture created by OpenAI, but I'm curious about what the community thinks about this approach.

My project is: https://github.com/khromalabs/Ainara