Pete Finnigan's Oracle Security Weblog

My day to day focus is helping customers secure data in their Oracle databases. As you will have seen from a small number of blogs here recently I have been writing a little about AI and in particular local LLMs as I have set up numerous local LLMs on my mac book pro M5 64gb laptop

This AI for me is just for fun and testing and also learning more about AI and how it can help in day to day work with Oracle database and in particular with development in PL/SQL and also even more particularly can it help in securing data in an Oracle database.

As I have said in previous posts, my view is that these local LLMs and even the commercial cloud based LLMs have greater value when used as an expert assistant rather than something to replace someone.

We cannot just come to the office on Monday and say all the DBAs and PL/SQL developers are fired and replace them with agents and LLMs. If an LLM was to take over a role then the person setting up the LLM and agents would need to know the role completely before this could be attempted. Also there is a big risk of edge cases that the LLM is not trained on and guessing or hallucinating is not must use over pressure of a deadline or solving an issue. If the agents/LLM hallucinate then that is dangerous for production. It makes sense that the person who sets up agents and LLMs must be an expert in the area being replaced to make it successful. My experience so far of commercial and free open weights models is that they do not know enough about Oracle. Yes, a lot of this could be solved or made better by pre-loading manuals / documents / code and much more but again the expert is needed for that phase to guide it.

There is a lot of talk around the internet about the wonder of AI and how it is fantastic and useful BUT there is also a growing number of doubts and negative talk. I watched a video yesterday about the issue of energy and water sources. If millions of people lose their jobs to AI then the video posited that there is not enough energy available to supply data centers full of GPUs or enough extra power to cool the GPUs or even enough water to cool them and more. So, if the energy supplies and water are not available how then is every job going to AI.

I would also think that if a lot of jobs need to move to AI where are all the experts in AI, agents etc to help model jobs and make them automated? it is like the time and motion guys of the 60s, 70s, 80s etc that measured each job in a factory to see how long it took and how it could be improved. If jobs truly need to move to AI then the jobs must be understood including edge cases and then set up properly with sufficient data inputs.

I have also read recently that a wide ranging study by MIT media lab showed that 90 - 95% of current corporate AI initiatives fail to deliver a measurable return on investment. In other words the AI did not improve the business and simply cost money; so either there was a net effect or even overall money was lost.

What about the AI companies themselves. Can we (corporate industry) rely on them being around in the future. It ss well covered in the news that they are all investing trillions overall to hyperscale or to build massive data centers but whilst their revenues are enormous for the normal person in the street (tens of billions) its nothing compared to the investment being made and the fact that its well known data centers could be obsolete as soon as they are built. i.e. faster better chips come along then do these massive builds get get ROI as well?

If we want to consider replacing people with AI and agents then we need to ask some basic questions:

How much does the current task cost - materials / people / wages?
What are the protected run time costs to run the AI - subscriptions / tokens?
How much will it cost to build the AI needed - materials / consultants?
What are the on-going maintenance costs to update the AI / prompts / agents etc when needed?
How much to lay off the staff - redundancy, other costs?

Then you can calculate a possible ROI i.e. AI runtime must be less than (costs + current runtime costs for staff)

This should be 10x or even more to make it viable to consider.

This does not even consider the technical aspects of the AI vs the person. Will the AI perform better in terms of technical accuracy than the person over all uses cases or only 20% of all use cases or ? Will a real person still be needed to assess what the AI has done and fix problems.

The value in AI is using it to supplement a persons role where it becomes an expert productivity tool BUT adds value because the tasks speed up but you still have the human input to check and meter the AI use and accuracy.

OK, back to Oracle security; is AI similar or does it have some of the same issues as Oracle security? yes, of course it does. In Oracle security we are not securing Oracle the database; we are securing the data that is held and processed in that database. We of course use Oracle security features and more to secure the data BUT the work is focused on securing data not the database.

In Oracle security I look at how the data flows into a system and out and all the touch points; I also consider the who and how accesses the system and why, what reason do they touch data? When we have this overview we can assess the technical aspects of the database itself, the design and architecture and permissions and access.

AI, in some senses is similar. You purchase a subscription and start to use AI to help but how does it help your company? obviously it can help generally because of its built in training but to help specifically you need to make it aware of your needs and data; to do this your staff start sending questions as prompts and uploading all of your source code, business intelligence, output of reports to get AI to find insights, schedules of staff to plan rotas, accounting information and much much more. Before you know it all of your company secrets are uploaded to the AI. What does the AI do with that? Just do a simple test in an online chat screen and you can see over a number of chats that it remembers your previous chats and even mentions them in the current chat!

This behaviour could affect GDPR, PCI, HIPPA and many other regulations that cover data security. This is like i have told people for over 20 years, do not post source code and secrets to forums and online sites. It is the same idea with AI. Assume that if you ask an AI to fix a bug in your source code then you will give that source code to the AI. You cannot for sure say what that means; is your source code available to others, is the context window only for you or will the AI companies teach the next iteration to include all your and everyone else prompts and history? we do not know for sure.

This is very similar to Oracle security; know where your data is and control complete access to it. This applies to AI

I have heard of apps using AI to allow users to query and question anything in a database that runs the application; Yes great selling point but very bad for security of data.

Do not get me wrong, AI is a great tool and can be used for good in any business BUT think about the security of your data in terms of AI, think about whether AI will cost more than now, will it increase profit, ROI?

Do not just throw AI at everything because you can. I have spoken to a lot of people who say AI is slowing them down but it is now a company requirement so they have to use it. AI is as great tool but you must understand it and know how to use it but just use it because you have to.

The person setting up good AI needs to be an expert in AI and also in the business process or task being replaced or enhanced by AI or be able to ask the right questions to understand it.

Do not forget hallucinations!, as humans we instinctively know if something sounds right or not, an AI just confidently guesses and can be wrong. All the AI companies say they cannot fix this, it is inherent.

#oracleace #sym_42 #ai #llm #oracle #database #security