Mastodon

PhilBig hopes for Qwen3. IF the 30A3B model works well, <code>gptel-org-tools</code> will be very close to what I envision as a good foundation for the package. It's surprisingly accurate, especially with reasoning enabled. At the same time, I'm finding that <a href="https://fed.bajsicki.com/tags/gptel" rel="nofollow noopener" target="_blank">#gptel</a> struggles a lot with handling LLM output that contains reasoning, content and tool calls at once. I'm stumped. These new models are about as good as it's ever been for local inference, and they work great in both the llama-server and LM Studio UI's. Changing the way I prompt doesn't work. I tried taking an axe to gptel-openai.el, but I frankly don't understand the code nearly well enough to get a working version going. So... yeah. Kinda stuck. Not sure what next. Having seen Qwen3, I'm not particularly happy to go back to older models. <a href="https://fed.bajsicki.com/tags/emacs" rel="nofollow noopener" target="_blank">#emacs</a> <a href="https://fed.bajsicki.com/tags/gptelorgtools" rel="nofollow noopener" target="_blank">#gptelorgtools</a> <a href="https://fed.bajsicki.com/tags/llamacpp" rel="nofollow noopener" target="_blank">#llamacpp</a>

Phil<code>gptel-org-tools</code> update. 1. Cloned to <a href="https://codeberg.org/bajsicki/gptel-org-tools" rel="nofollow noopener" target="_blank">https://codeberg.org/bajsicki/gptel-org-tools</a>, and all future work will be happening on Codeberg. 2. Added <code>gptel-org-tools-result-limit</code> and a helper function for it. This sets a hard limit on the number of characters a tool can return. If it's over that, the LLM is prompted to be more specific in its query. Not applied to all tools, just the ones that are likely to blow up the context window. 3. Added docstrings for the functions called by the tools, so LLMs can look up their definitions. 4. Improved the precision of some tool descriptions so instructions are easier to follow. 5. Some minor improvements w/r/t function names and calls, logic, etc. Basic QA. Now, as a user: 1. I'm finding it increasingly frustrating that Gemma 3 refuses to follow instructions. So here's a PSA: Gemma 3 doesn't respect the system prompt. It treats it just the same as any other user input. 2. Mistral 24B is a mixed bag. I'm not sure if it's my settings or something else, but it fairly consistently ends up looping; it'll call the same tool over and over again with the exact same arguments. This happens with other models as well, but not nearly as frequently. 3. Qwen 2.5 14B: pretty dang good, I'd say. The Cogito fine-tune is also surprisingly usable. 4. Prompting: I have found that a good, detailed system prompt tends to /somewhat/ improve results, especially if it contains clear directions on where to look for things related to specific topics. I'm still in the middle of writing one that's accurate to my Emacs set-up, but when I do finish it, it'll be in the repository as an example. 5. One issue that I still struggle with is that the LLMs don't take any time to process the user request. Often they'll find some relevant information in one file, and then decide that's enough and just refuse to look any further. Often devolving into traversing directories /as if/ they're looking for something... and they get stuck doing that without end. It all boils down to the fact that LLMs aren't intelligent, so while I have a reasonable foundation for the data collection, the major focus is on creating guardrails, processes and inescapable sequences. These will (ideally) railroad LLMs into doing actual research and processing before they deliver a summary/ report based on the org-mode notes I have. Tags: <a href="https://fed.bajsicki.com/tags/Emacs" rel="nofollow noopener" target="_blank">#Emacs</a> <a href="https://fed.bajsicki.com/tags/gptel" rel="nofollow noopener" target="_blank">#gptel</a> <a href="https://fed.bajsicki.com/tags/codeberg" rel="nofollow noopener" target="_blank">#codeberg</a> <a href="https://fed.bajsicki.com/tags/forgejo" rel="nofollow noopener" target="_blank">#forgejo</a> <a href="https://fed.bajsicki.com/tags/orgmode" rel="nofollow noopener" target="_blank">#orgmode</a> <a href="https://fed.bajsicki.com/tags/orgql" rel="nofollow noopener" target="_blank">#orgql</a> <a href="https://fed.bajsicki.com/tags/llm" rel="nofollow noopener" target="_blank">#llm</a> <a href="https://fed.bajsicki.com/tags/informationmanagement" rel="nofollow noopener" target="_blank">#informationmanagement</a> <a href="https://fed.bajsicki.com/tags/gptelorgtools" rel="nofollow noopener" target="_blank">#gptelorgtools</a> PS. Links should work now, apparently profile visibility affects repo visibility on Codeberg. I would not have expected that. PPS. Deleted and reposted because of strong anti-bridge sentiment on my part. Screw Bluesky and bots that repost to it. Defederated: newsmast.*

Recent searches

Search options

Administered by:

Server stats:

#gptelorgtools