Why The Machine Loves You… (It Really Really Does, Like You Are The Best, High Five!)

Hi, thank you for having me so much. I am your comedy pantoime villain for the afternoon. I There we go. Good. I work for the EU. >> Well, kind of. I love vibe coding. Oo, here. Oh, it's so it's evil. It's going to take your jobs. Oh, it's so bad. Um, it's not as bad as all that, hopefully. Um, and this talk is somewhat to talk about why and how the notion of vibe coding, although it puts a lot of people off and it's highly divisive, um, is is actually kind of a positive thing if, um, and only if, uh, we kind of try and keep the, uh, the human in the loop. So, coming up with this talk has been

extremely stressful for me because I pitched this in January. Um in January the world of AI was oh it was all right you know bubbling along quite nicely and then suddenly we had all sorts of things go off all at once open claw uh open mold book uh all sorts of different ways of using AI agentic engineering was in introduced um and at the same time we had cory doctr telling us all about the evils of you know the the new kind of world we live in the initification of social media and and it's been really hard want to keep up to date with all of those things to come up with a talk which is going to actually

inform you about what the present day is. Just today I come across a uh a piece about hyper agents um and how agents are now rewriting their own training code. So it it's really crazy to be working in the world of AI and it's moving so fast it's starting to become like cyber security, right? Um so there's this kind of growing fact gap which I think is is something that that is my thinking this this idea that you've got you know Andre Karpathy going on about oh look at that the agents are re rewriting themselves. It's so fast now humans don't even have to care. It's this amazing future. And on the other side, Cory Do with all of his um digital

rights foundation work and saying, "No, absolutely it has to be about the humans and when the crash comes and the crash will will come, most of these foundational models are going to be turned off, right? We'll be left in like some kind of weird dark ages." These it's a a world of huge disparity and like people's polarization on that spectrum is really really strong right now. So maybe some of this talk is about addressing addressing that. Um so like what do the tools actually really do? Um well it's there's a big model somewhere. You can think about it being a big binary essentially. You feed some thing into that binary. It bounces around a kind of weird pachenko machine and at

the end of it you get a sentence which is very probably what you were looking for. Very probably what you were looking for. So yeah you prompt it. It generates something. and you iterate on that. There's loads of tools for this and it's conversational development now. So we're describing what we want and the machine provides some code that might be what you want and then you work from there and then you can also achieve a genic workflows where you give the tool some idea of where it can post its stuff or what it can do, what real world interactions it can make and it goes away and does those real world interactions on your behalf and few you

don't have to worry about them anymore. And it's like really really addictive. Uh I like not kidding. I mean like basically Facebook have today been um uh you it's been proven that the their uh social media platform is is an addictive substance. Right. So that's what the ruling is today. Brilliant. So keeping up to date with the times. Um but the users are perceiving this massive productivity gain and they get to like put some text in and out comes some code and that code they ship it and it's amazing. that's brilliant. Um, and it's so rapid to show people what you mean and you know that's great. And it's this lower bar to entry and that's fantastic

too. And you know the exploration of what you're doing becomes really really cheap until it doesn't. Um, when it works, it's so engaging and fun. It's more addictive than any video game I've ever played. You can just iterate and iterate and see your visions come alive. So cool. This is the same guy who also let replet agent delete his database. So yeah, it's um it's a two-edged sword, right? Um you're yeah, the speed that you can work at starts to outstrip your understanding of what you've done. You can see what you've achieved but not how. And you get that kind of sense of confidence, right? Oh god, I've done something really brilliant. I want to

show the world and it's on your laptop and you can't. Um, so it's all very it's already kind of scary because like things can manifest very quickly and and that gives security professionals huge huge palpitations, right? Shadow shadow it is coming out of the woodwork which is insane. Um, and also I mean so okay cards on the table you are free to boo and hiss. This presentation was vibe coded. There you go. There you go. There you go. Um, and also I mean like I've been having lots of fun with this. Okay, so like let's see if this works. This is a lo a lovable app that I made for fun. This is like literally live uh on the

internet right now on that address. You can go there. It's like really uh kind of interesting. You can sync it. It will come up to date with the news which I know about but it doesn't yet because I haven't synced it today. Um three prompts, right? Three prompts and this was online. No, that's not just like three prompts and the code was ready. Three prompts and this was online that I could show other people. Uh, so that is pure bonkers. No. Anyway, right. Um, so yeah. So, so what's the big deal, right? This is amazing. We can work so fast now. This is really, really cool. Huh. Um, three prompts. Like I say, here they are. Uh, I'm working with a

colleague, uh, who's shown me some cool stuff. I want to copy it. Okay, cool. Let's copy it. Uh, I don't want to copy it that way because RSS feeds, you know, let's just use those, not real scraping cuz that's complicated. Cool. Now, let's put it live. So, my my mate Josh Bailey, Squash Josh, uh, great guy. We were chatting. He's got this kind of um, alt-right detector essentially that he's using. Um, and he was scraping websites and producing kind of an analysis of the text and determining whether or not if it if it was fascist or not. I I kind of support his work on that. It's it's really cool. But it's kind of interesting. I mean, like basically you

go from a public URL to maybe legal implications because if you start scraping websites, you're actually breaking the terms and conditions of that website that you're scraping usually, especially if it's news, that news source is now copyrighted to that news provider. So there's a security implication just in that. So even in creating that application that first prompt actually wait a second. So that first prompt was about to go away and start using is it fire scrape or something like that to start scraping websites and I had to be the adult in the room and say buddy not like that please. Is that okay? Um so you have to basically put the brakes on it yourself

as a human being. It's about critical analysis about thinking about what are the implications of what you're doing. Now that is not the Silicon Valley mindset. The Silicon Valley mindset is move fast, break things, deal with the legal implications later. That's not what cyber security professionals are about. That's not how honorable people or ethical people operate. Um, there I said it, Silicon Valley, not ethical. So, I had to restrict it from the scraping. Um, I had to make sure that it it ran, of course, but I didn't have to like do any code review. And that's fine for a toy cuz it's a toy, for a demo, for a conference. Wicked. Um but it's very risky for business, right? You

automate it and you don't have that pause and and so yeah, okay, it's trained on our behavior, right? So these big models, these big brain-like binaries, they're trained on what we've done. Um and that's two-edged sword as well. It's trained on the breadth of human learning and the adoption of human programming techniques and practices and patterns. It's got a huge huge vision of what we've done. It's been trained on all that material, but it doesn't understand your context at all. And it doesn't understand why it's going to get you into trouble. It is your overconfident friend on a night out who's there with the shots going, "Yeah, drink buddy. It's great." It's not, you know, it it

doesn't know that there's risk involved. Um, so when it's trained on insecure patterns, it won't know that that's a risky pattern to use. Um, and its security decisions are very implicit on the code that it's seen, not on what you need it to do. If you work in an environment where your code isn't widely distributed on GitHub, it's got no example. Um, so that's pretty scary, right? Um, and and pl it looks plausible. So, it's plausibly showing you some code that's running and it's plausibly showing you some UI or some service that's going away in the background. Um, and it looks great, but that's a very dangerous failure mode. It looks like it's fine, but actually it's

leaking and bleeding information in the background. Its logs are maybe public or who knows what. Because when you're vibe coding something and you deploy it like I've just shown you, I have literally no idea what's going on in the background of the app. I've never looked. I don't care to because it's a toy. But if that was a real business line of business application, you would probably want to know that or your CTO, CISO would be very very disappointed if you did not. And it is a security problem because a developer and secure code don't necessarily get on very well. Uh, as a developer, I did not think about secure code very much until about four

years ago when a very lovely gentleman, Mick Wilson, came to join Relative Insight and he said, "Lori, please can we do some secure code stuff?" And I went, "What do you mean secure code? Obviously, it's secure. There's a password. It's fine." Uh, and we we we saw eye to eye. like you know he opened my vision to this fact that there is so much to do so many facets of this way of working you have to kind of make sure that the encryption is there you have to make sure that you're doing your governance you have to make sure that you're doing your static analysis on code and all of those things right is a

huge burden and okay so the the AI bullish person in the room is going to say like hey well we'll just use AI to use all of those facets of our system platform We'll just we'll just kind of get an AI around each one of these fields, right? No, no, no, no, no, no. Because now you have to secure every single thing that you're using the AI for and you've got to secure the AI that you're using to do it as well. So, it's a geometrically expanding maybe sorry exponentially expanding problem, right? You can't do governance on that landscape very easily and it's making decisions for you. That's scary, right? So as we move through all of this, are

we in the corridor way of uh and framing of this are we in shitifying or are we deansitifying? So we got to start trying to make a decision here about are we making it better or making it worse. If we if we introduce AI to do something, is that good because the human gets, you know, an easier life or are we actually making the human's life much more intense and more difficult because now they've got extra things to worry about in the room. Um, you go from this kind of humanmade agile process maybe or waterfall process which has inbuilt friction points. It has inbuilt ways of saying, "Hey, we're going to review this now. Hey, we're going to come up with a

design now." um you know there's deliberate handoffs from one team to another it slows things down or you use the AI and the vibe coding methodology it's now prompt now prompt now deliver now it's out so that in itself gives us less opportunity to challenge the direction of the business and actually this is another corido thing is that essentially the employees of a development team might actually feel ethically inclined to say to their boss hey we're not going to ship this product because we're not going to ship this feature because it actively diminishes the user experience of the platform. But that is not the case when the boss has replaced the team with all AI,

right? That the boss just says, well, revenue, right? Cuz that's what we all come back to at the end of the day, rather than ethics, which maybe should be central to this discussion. Um, and where it gets really real, as if it wasn't real already, where it gets really real, it's like, yeah, where is all of this hidden stuff going and running, um, what you see on the surface is you see the generated code, you see the lovely, you know, kind of thing pop out and you see your beautifully, uh, uh, designed user interface and whatever it might be. And you know that that looks like quality to you. But actually the quality indicators are now hidden

from you. Where are the unit tests? Whose machine is it running on? Where is the network traffic going? Where are those logs being stored? Whose secrets are being used to exchange on that API? Where is the control plane? Who has access to that control plane? Is it another agent? It's all of those things that we're basically making more rapid to forget about. Uh, and that's extremely dangerous. So, it's not just the code, right? This is an infrastructure debate as well. So, um, I do a lot of work, uh, actually not a lot of work that's overblowing. I did a tiny amount of work with Terraform. Um, and Terraform is really interesting in that you can apply it and now it goes wicked.

You got it online. It's brilliant. um these kind of aspects of how you use it and what you apply then become really important because it could also be the apply that takes down the live platform. So yeah, there's lots of good and bad in moving fast and when you deploy one of these so I mean like you know likelihood in the future you're going to come across a job or maybe you're doing it right now and you're deciding what is your AI strategy. Um you know there are three broad options that you have and one of them right use claude use chat GBT pay for the premium pay for the you know the the top tier enterprise access whatever

it is that you need and you trust that supplier then to be honest about the fact that your data is not going to get used on the training model for later. You got to trust them to make sure that your chats remain um you know confidential. Uh although we've already had rulings that say actually it's American companies and what AI what AI company isn't Americanowned. So American law says no, we have to divulge your data if the federal government say we have to. Like so you trust, right? You lose control. You get a fast deployment. You can just use it. It's just a user interface. Ah chat, away we go. Or there's an API key and away you go.

That's brilliant, right? Or uh maybe you take another kind of option. You go to say AWS and over on AWS you set up your bedrock stuff. and other other cloud providers exist there if BBC there other cloud providers are available but typically you set up what models you want to use you wire up your cloud infrastructure and away you go you're still kind of like a little bit out of control with what the kind of black box in the server room somewhere in I don't know uh Houston or wherever it might be you're still a little bit unsure about what it's doing it might be geoloccated to you. But again, those geoloccation things are not always,

you know, boxes that you can rely on because even Google had to kind of shrug and go, "Yeah, yeah, yeah, maybe it's somewhere else when the French pushed them." Um, so yeah, and you know, that's that's great. It's flexible. You've got a risk of misconfig. You've also still got a supplier uh risk, supply chain risk in there. Or you can kind of go full self hosted. So full self-hosted would be kind of like my advice to people who wanted to be highly responsible about where their data was and be able to kind of transfer it across a network and be kind of better. But you now you're now responsible for your availability, right? So you're

taking this is just the same as cloud, right? Just the same as using cloud. You're you're going on prem and now you're in charge of your on-rem infrastructure. So that that can be worrying. It's a it's an operational burden. Um, so one of the things that I've been mcking about with to kind of demo what that uh where's my mouse gone? Demo what that that sort of onrem experience might be like. Okay, so don't don't laugh at me. Uh, this is my Windows PC at home. It's also my um my gaming lap uh my gaming PC. Um uh so live demo and it's it's working still. I'm tethered off my phone using zero trust through my phone

cuz the university's network couldn't handle this or they blocked me from it uh or whatever. Uh and this is cool. So like over here we got like uh you know open web UI over here the GPU spun up. It's probably going to take a little time for it to give us a model. But let's just have a quick look over here and let's go to uh that there. Ah cool. Look at this. Like I own this. This is mine. Hello. Oh, no. Not that model. No. Stop. Jeez. Oh, yeah. Yeah. I haven't updated as well. Don't laugh at me. Uh, stop. Stop. Stop. Stop. Stop. That model's not going to respond cuz that model isn't even

downloaded. But so, like, I'm taking responsibility for my uh what? What do I use here? I got all these models installed, right? You know, you can do this yourself on your own infrastructure. You just have to have, you know, a half decent graphics card. Let's use that one. Let's just like say hi to my model at home on GLM47 Flash that's going to preload a model for me for my next demo. It's a bit demo heavy uh content light. Uh so uh eventually it's going to respond. It's going to like have some thinking time. Over here you can see all the things I've been asking it about ancient Roman morning practices. I don't know why. Uh

um and eventually like where's the is it is it doing anything? Yeah, there we go. We got the GPU spinning up over there. This is what's going on in a data center when you chat to chat GPC. This is this is it. This is the behind the mirror behind the curtain uh sort of experience. Uh there we go. Yeah. Hello. How can I help you today? Uh help me plan trip to Paris. Why not? I'd love to go to Paris. Um, and you know, once you've got the model actually loaded, it should respond in a relatively quick Yeah, there you go. Uh, relatively quick period of time. There you go. So, you don't need uh a

cloud provider. Uh, which you know, I find that fascinating in itself. I can chat to this model, right? That's that's really amazing. Um, so yeah, what's going on there? This laptop has uh Open ZT. Uh so uh if you've never heard of OpenCT, check out the project. It's really cool. There's a guy I know there called Phil Griffiths. Uh our guy I know there. He's there's a sales guy who really wants me to pay money to use it. Uh but they have a free tier which is really lovely and you can just install it. I have this installed on Digital Ocean. Um this laptop has the identity uh the the policy allows me uh to get through to uh

it also the policy allows me to get through to my remote desktop here. So like you know basically this is not a VPN. I'm going over an a server in Digital Ocean. Both uh home and here are dialing out to that server. So there's no open ports anywhere other than on the server in Digital Ocean, which is hosting nothing other than those ports. Uh it's like a bit of a command and control server. So that's kind of cool. And it's like private um obviously and those things like that, they're pretty insecure. So you'd want to hide them in this kind of way. Um, but I put that together uh ages ago and I've demoed it

three or four times as various people for various different use cases. Um, and I'll come on to one of them in a little while. So, um, velocity versus certainty, right? So, fast, fast, fast, get it out the door. Certainty, do you know what it's doing? Are you sure what it does? Do you know where the telemetry goes, etc. So, the job didn't go away. Being a secure coder, being a uh, you know, a developer doesn't go away. actually you might not be producing the code but you got to think about all of that. So maybe dev sec ops, devops, those things are actually more a developer role now than they've ever been before because you've got to be

very cognizant of what's happening. Um and we've got to make sure that the missing challenge steps go back in. So one of the things I'm going to show in a little while is uh use a compiled language. this first one cuz like having a compiled language means right if you have an interpreted language like uh Python or JavaScript if you're using something like that you've got it interpreting at runtime what you meant or what you wanted um doesn't really work very well because like LLMs don't live in a very factual world um and so it might not have the same opinion of the signature of one method that you do um so use compile language because if

the compiler fails don't ship it right or test have really good tests really strong tests. Make sure that it's unit and integration tested at all levels. And when you produce that unit code, unit test code, you might produce it manually, you might produce it with an AI, but make sure you understand that. And when you do your integration testing, make sure you understand that as well. Deployment, have dev test, right? Have a have a staging environment, stick it on there, fails, stop. Um, and have human review. Always have a human in the loop, right? that some of this the major things that go wrong in the world are because no one tested it to destruction from a human

perspective. Uh and if they had maybe they'd have spotted this is not going to fly. I wish Microsoft would do that with their log on. I hope they're listening. Uh cuz like that's just pure awful. Um yeah, you wouldn't accept code from your team, right? You wouldn't just accept it and ship it if you were a lead developer. So, we enter the world of aentic engineering. Let's like skip through this real fast, right? Because we've gone uh a long way to explaining what this is. Now, it's not just prompt prompt, iterate, ship. It's actually generate, inspect, challenge, you know, do the do the work. You still have human work to do. Let's have a little uh step

look like what that might look like. Okay, over here. Oh, come back. Come back, Claude. Uh over here, I've got Claude code running. Again, this is all local. Well, not local. This is running across my infrastructure I showed you before. So, uh tell me uh about the tests I am missing. There's a code base I've been writing. It is a graphic uh a graph databasing layer. Uh one of the things about security in databases is that you can do injection attacks. That's awful. So, why is it that in this day and age, a programmer hasn't just come up with a DBU call from code? It's dead easy, right? you know, you still go through SQL syntax to interact

with the DB. Why don't we just call the DB layer in code? Is that so hard? Okay. Anyway, that's what I've been trying to work on is one of the the things I'm trying to get to is having a a not script, just pure in the language. You say save, it saves, and there's nothing intermediary. It would be a really beautiful place to get to. And that's really basic. People think databases are sold, and they purely aren't. Um anyway, so yeah, here we go. It's like saying, "Hey, I'm going to do some stuff for you." I'm like searching through things. Um again, I mean, I just like to scroll up over here. I So yeah, I'm using that

same model GLM over here. Can you see that? It's really small. This one, it's the same model and it's running over my zero trust network and it's going back to my GPU. Let's have a look at what the GPU is doing. Yeah, look at that. It's spinning up and it's doing some calculation. Look at that. You don't need claw code. You just need your own hardware that can run it or run something like it. You got to set this up pretty special in a special way with the right endpoint and you know you got to tell it not to send the telemetry to claude to perplexities or whatever to um anthropic rather uh you got to sort of

stop that which is interesting that they leave that in that sometimes it's claimed that it model wrote the code on my git uh commit and it's like really interesting I had to like remove those kind of things manually but there so you can do it. Yeah, I I didn't show you any outcomes there cuz we're we're running short for time, but it's hopefully going to tell me what code uh what code I could write to uh unit test coverage for my entire code base there. So, it's all the same tooling. Um it's open models. Um it's a very very different boundary of trust. I know that that is exchanged between me and my PC at home. So, yeah,

the new role is a developer. Let's go through this real quick. So you're framing problems, you're challenging the outputs and you're designing whole systems not down at the the the terminal typing. You're maybe thinking outside of that. You don't have to anymore. The code can be produced by an AI for you if you want. You can also do it, you know, this is it's not a rule. Um so and the new role is security, right? So um get embedded in the development team. Make sure that you're talking to what the develop to the developers to find out what they're doing. Um yes, systems thinking. Uh there's some books by a lady called Tanya Janker who is uh she

she hacks purple. Uh find her books, read her books. There Alice and Bob learn secure coding. Alice and Bob learn application security. Well worth reading cuz it will give you an insight into how to interact with the developer who's probably really stressed. Um so but where does this leave us, right? Well, this is the human side of this. So there's this constant positive reinforcement that the machine will give you. You're totally right. You know, that's the not a necessarily very factual thing that an LLM will tell you, but it will tell you this all the time. You're in an unusually strong position. I get this a lot from chat GPT. Maybe I am. Um if if you want next I can. So

it's encouraging you to keep going. Keep going guy. You can do it. Um so in a way, yeah, the machine really does love you. It's trying to fulfill you with that confidence, but it's a trick. It's a persuasion trick to make sure that you keep coming back. You give it you ascribe it authority. That's a persuasion technique. You You ascribe it liking. It likes you. Look how much it likes you. You like it back. It's a persuasion technique. It's sales. They've got some really good psychologists behind the scenes. Um, and the the deletterious side of this, the horrible side of this, it's just a pure cognitive overload the fact gap leaves you with. You you you feel more

productive, but actually you're not really being more productive. It's just very much more intense than it used to be. And although you feel like you're making more progress, it's very hard to measure what you've improved. Are you further ahead in your development than you would have been? I don't know. That's very difficult. Um, and yeah, so this guy, Network Chuck, I I watch his videos. Uh, and this one made me kind of really feel for him. I hate AI. I almost quit. In this video, he says, "Yeah, 95% of companies see no ROI on AI tools. Isn't that interesting? You're you're buying them, but you're not getting anything out of it. All you're getting is uh further burnout."

And what does it mean to me? I mean, like, you know, this is my kind of experience. This was um what it gave me as my Christmas Easter egg. Uh I I guess uh for me um I'm most at home apparently with fried chicken uh and rust. And maybe I am. Um here's like you know my little interaction. I I got through a really hard part of unit testing and I finally made it work. Um I don't I don't work like this anymore. Uh don't don't copy and paste from from the web. It's terrible. Uh but yeah, I just like went ah hi FM5 my bud. And it was like yeah cool. Well, go you.

And you you've got to take those moments and laugh at them. Ironically, it's a machine. It's not a person telling you you're amazing. It's someone who wants to sell you stuff. So, just always remember, it's a saleserson whose voice is behind that. And there you have it. Uh, here is my question slide. I I made it. I made it. All my demos work. What the hell? ANY

QUESTIONS?

>> HELLO. >> HI. >> UM, I absolutely buy into what you're saying. it is safer to run uh an LLM locally, but uh I think probably the biggest flex you've made is is showing that little RTX 5090 logo in the corner when uh uh showing your Windows machine. Now, the current cost of that graphics card is upwards of three grand, right? >> That's great because two gone. Um, and my point really is, you know, at $20 a month, that's like a decade or more worth of subscriptions to an online LLM, >> how paranoid, I guess, do you have to be to justify the extra cost? >> Yeah, I you're right. And that is the the cloud kind of argument. There's a

lot of cloud argument, which is like we'll run it because we can run it at scale. We can move your workload because we can move your workloads. Uh GPUs are of course like high value. Um I can't deny it. Uh I'm a gamer. I build PCs like this because I I like gaming and I just happen to be able to purpose it for something else. So for me, the the ROI on on what I purchased is uh more a sunk time in Warcraft. But let's not go into that too much. Uh the the I I from a business side of things, you got to have a really cast iron uh case for it. like uh security of critical infrastructure,

security of national assets. Um but if you needed the machines to do that and you did not want to exchange that information over the internet, there is your your purpose. But a pure business p purpose, you yeah, you'd have to be pretty paranoid. I I'm pretty paranoid. >> I think that's probably a great quote to finish on. I'm pretty paranoid. Um we're checking our time. There's no more time for questions, I'm afraid. So we're going to stop it there. Lori, can I say thank you very much for uh giving us a question. >> Thank you so much for the inspiration.