Skip to main content

AI Hype, as the 1980's song sang "Don't Believe the Hype" when it comes to Security

  • May 2, 2025
  • 4 comments
  • 74 views

Geoff Burke
Forum|alt.badge.img+22

VeeamON was not an exception when it came to new AI announcements. While AI is an incredible tool and powerful advisor, there are concerns that it is being implemented too fast, too soon and being given too much control. Recently the Godfather of AI, University of Toronto and Nobel Laureate Geoffrey Hinton stated that there have already been cases of AI seeking to expand its control and access levels in order to better perform the tasks that is has been given and here explains why that probably happens:

 

There have also been incidents in China of AI agents in robots going bezerk and attacking humans

https://theconversation.com/a-robot-nearly-headbutted-a-festival-spectator-in-china-here-are-four-urgent-steps-to-make-the-tech-safer-250851

 

We often see humans introducing new technologies quickly for gain and profit leaving security and safety as an afterthought but could this transition to an AI world be the most risky tech adaptation of them all?

I have written an article here on the subject as well https://www.securityinfowatch.com/ai/article/55287478/why-human-oversight-is-essential-to-mitigate-emerging-risks.

 

I am interesting in hearing everyone’s take on this.

 

 

4 comments

lukas.k
Forum|alt.badge.img+13
  • Influencer
  • May 2, 2025

Great article ​@Geoff Burke!

My opinion on that: When it comes to data, most end-users and IT stuff as well deal with that differently. Since end-users often only see advantages and excitement in new tech they often throw data everywhere to see outputs - no matter what.

Especially in Europe we have specific rules on how to handle sensitive data but the reality looks different.

 

Let spin the wheel a bit further. Do these people see the risks? Imo - they don’t. The huge interest in new tech and AI in specific leads to overexcitement and leads to “checking out limits” - and often go beyond it.

 

I don’t now anyone who tries to get information about AI prior to simply putting in promts and for “try and error”. As covered in your article we simple cannot define the limits of AI right now - if there is one.

This leads to a huge risk in IT security because most people skip procedures and risk analysis that are performed and required for almost every other topic in modern IT - we kind of “reset” our brain in some way, which can be dangerous.

 

I see AI as a huge challenge but it can be both exciting and very successful but is also dangerous. Just like driving a sports car: Huge fun, huge risk!


Chris.Childerhose
Forum|alt.badge.img+21

AI is definitely coming along very fast and rapid to mainstream.  It is interesting to see how it has evolved since it first appeared and its capabilities.

I definitely appreciate it for many aspects of my job in IT as it has helped streamline workflow, work on automation, documentation of services, etc.  There is always that fear though of how much it will be able to do.

If used for the “right” reasons it will benefit many but the bad actors are the ones that are going to take it up a notch as it progresses to use it for other things.


Iams3le
Forum|alt.badge.img+11
  • Veeam Legend
  • May 2, 2025

… but could this transition to an AI world be the most risky tech adaptation of them all?

No, I don’t think so. This is only because the Artificial Intelligence Act (AI Act) has been missing. However, the EU AI Act has recently been implemented a legal framework for AI systems in the European Union. It focuses on risk management, transparency, data governance, and ethical AI practices.

But then, as AI technologies continue to evolve, the need for robust solutions to ensure compliance and foster responsible AI becomes increasingly imperative. Not sure about the US tho: https://www.whitecase.com/insight-our-thinking/ai-watch-global-regulatory-tracker-united-states


Iams3le
Forum|alt.badge.img+11
  • Veeam Legend
  • May 2, 2025

I don’t now anyone who tries to get information about AI prior to simply putting in promts and for “try and error”.

This leads to a huge risk in IT security because most people skip procedures and risk analysis that are performed and required for almost every other topic in modern IT

I see AI as a huge challenge but it can be both exciting and very successful but is also dangerous.


With the right controls in place, we can mitigate some of these concerns. Therefore, there is a need to leverage third-party security solutions, such as Microsoft Defender for Cloud to continuously discover, manage, and protect AI workloads. This approach helps in identifying and mitigating threats such as prompt injections, data leakage, data poisoning, and wallet attacks (DoW), among others.