I want to understand the capabilities of running AI LLMs locally and do they offer value to my day job of Oracle security consulting, training and products and more generally can Oracle people; DBAs, Developers, Security people find value as well.
The key drivers for me are AI Sovereignty and protection of data. For years I have helped people protect data in Oracle databases and it seems AI has come along to drop us back 15 to 20 years; right to the beginning. People seem to be blind-sided by AI and think it is OK to open up the complete database schema to AI and transmit data in the form of prompts to AI providers.
I have spent years helping people lock down data using the controls within the core database and also helping implement additional cost and non-cost options such Database Vault, TDE, TSDP, audit trails, masking and basically everything possible including of course writing custom security code for Oracle databases.
One major gap in LLMs is the audit trails; yes there are some but they are not good enough. Any company serious about securing data in an Oracle should lock down that data, audit that data, control access to that data and have controls to provide logs of who accessed the data. In terms of LLMs we need to know who, when, why and what data was accessed (sent in a prompt) and what data was retrieved (response).
We should consider the design if we allow data from the database to be sent to an Large Language Model. We should limit or anonymise what data is sent and we should control the access paths to the data and ideally air gap it from production; in simple terms do not allow unfettered access to production with an LLM or agents or harnesses or...
In terms of whether an LLM is useful for day to day Oracle work; yes, it can be BUT it should not be a replacement for a DBA or developer. In other words don't think that you can reduce DBA roles and replace with an AI and use a harness, loop, agents and expect your database to work and be fully supported. In my experience so far of researching AI and using AI I have found that it may be great at vide coding or the new phrase One Shotting a web app or web based game or website BUT it seems to struggle with other areas. The main LLMs seem to be good at Javascript and HTML and CSS but less so with the details of Oracle databases or coding with PL/SQL or ...
There are clear gaps in using an LLM with Oracle; my guess would be because the main models have not been trained as well on the internals of Oracle and the documentation and there are also a lot less posts on various sources on the internet that focus on some Oracle tasks.
I think LLMs and particularly the open weight free ones are getting better and we can improve the use of LLMs with better more concise system prompts, tune the settings such as reasoning and temperature and context size. We can also teach open free weight models using tools such as Unsloth and we can of course use harnesses such as Pi or DeepSeek harenss to create agentic loops or graphs. There is a lot of power available with Local LLMs but it is clear they are getting better all of the time and it is clear that we need to put in a lot of effort in terms of inputs (RAG or training LoRA adaptors) to make them realy useful.
In simple terms the model needs to be set up and tuned and controlled and the right inputs need to be made available in terms of documentation fed into it via RAG or training and of course to use models successfully you need to be able to write prompts that work and be able to already know if the answer is correct or sounding correct. This is why I think the use of AI / LLMs is useful as an expert assistant and not a replacement.
MOST IMPORTANT; before you consider using either frontier models or set up your local LLM you need to consider auditing and data exposure as I said above BUT if you spend time on AI how does it provide ROI (Return On Investment) for you. There are mainly two ways; increase productivity to make more money by doing more in the same time or reduce costs. So, paying huge amounts for tokens that do not generate more turnover or revenue does not work. Use of AI should increase net profit otherwise what is the point.
I have been using a local AI set up with a Macbook Pro M5 MAX with 64gb of RAM for AI and normal use. This means I can run local models around 30 billion parameters easily and with good performance for one person (me) or larger models around 70B in some cases with reduced performance. I am using ollama to run the models and I was using OpenWebui to provide a chat interface but I wanted to move to AnythingLLM as a better model control interface and also to have some tools available OOTB (Out Of The Box). I am running the mac as a server and access OpenWebui and AnythingLLM from a Windows laptop via the browser.
Setting up AnythingLLM was easy. I had to download another model for the embedding nomic-embed-text. I also then had to pull anythingllm via docker on the mac and then use a docker command to run anythingllm on port 3001 within docker; I used port 3000 for OpenWebUI already. Once started I can then access the LLMs via AnythingLLM from a browser on my Windows laptop.
The real focus of AnythingLLM is the tools interface and the fact that you can upload documents and source code to it and these are indexed and stored in a vector database. The idea is that you can create different workspaces and different documents to use in the interface either by chat or by agents.
A first test i did was to ask Quen3.6:27b via AnythingLLM What is the latest version of the Oracle database for Windows?
The answer came back via AnythingLLM as:
Based on my latest training data, Oracle Database 23ai (version 23.0) is the newest major release, with 19c remaining the current Long-Term Support (LTS) version. Both are officially supported on Windows x64. Oracle frequently rolls out quarterly patches and minor updates, so for the exact latest build number, download links, and compatibility notes, please visit:
Official downloads: https://www.oracle.com/database/technologies/oracle-database-software-downloads.html
Patch/Release Notes: My Oracle Support (MOS) or OTN’s Release History page
If you share your Windows version and use case (development vs. production), I can help you choose the right edition, verify system requirements, or walk through installation/upgrading steps.
Hmmm, obviously wrong BUT it explained why it is wrong; at the time the model was trained Oracle 23ai was the latest version.
So, using AnythingLLM I can improve this result by enabling the web search in the current workspace and enable DuckDuckGo search which is built in and send the same prompt again.
This time I get this answer:
Based on the latest search results, the newest major version is Oracle AI Database 26ai (often referred to simply as Oracle Database 26).
This was officially released in early 2026 and serves as the new Long-Term Release (LTR), replacing Oracle Database 19c as the primary long-term support option for production environments. It is fully supported on Windows x64 and includes official installation guides for Windows (ntcli).
Key details:
Latest Major Version: Oracle AI Database 26ai (version 26.0+)
Release Type: Long-Term Release (LTS)
Windows Support: Yes, fully supported on Windows x64 platforms.
Predecessors: It succeeds Oracle Database 19c (previous LTS) and Oracle Database 23ai (which served as an interim release with a shorter support window).
You can download the latest installers for Windows directly from the Oracle AI Database Software Downloads page.
So clearly we can combine the models own weights (knowledge) with RAG (documentation) and search (DuckDuckGo) and even teach a model with data and unsloth.
I will be testing more with AnythingLLM in respect to my work with Oracle and security and also newer models as they become available and update you here.
#oracleace #oracleacepro #sym_42 #oracle #database #security #llm #ai #anythingllm #unsloth #rag #duckduckgo #qwen
