Anton Sokolov-Who Said the Agent Could Do That? Catching the Correctly-Signed, Out-of-Mandate Action

Heat. Heat. [music]

[music]

Good afternoon. It's late in the day and I will keep the sentences short. >> [sighs] >> Everyone in this room signs things with the ID card or with the app on your phone. Contracts, taxes, the sale of a car. By law, the signature is as good as your handwritten one. What does it prove? Two things. Who signed and that nothing changed after. That is all it proves. It has never once asked were you allowed to do this. That is not a complaint. It was built that way. A signature tells you who whether they were allowed is a different paper. This talk is about that paper. That paper is about power of attorney. You let someone act in your name.

Lawyers also call it a mandate. It has two limits. What they may do and until when? Sell the car. Your grandmother's car? Yes. Sell the flat? No. Until the end of the year? Yes. In January? No. Both limits can be broken and the signature stays perfectly well. A real power of attorney, the human one, New York, Brooks, the best known philanthropist in the city, about $200 million and Alzheimer's. sad. Her son held her power of attorney, a real one, a human. The prosecution said he bought a yacht for 920,000, and he refused to pay a safety gate in her home for 2,000. In 2009, many years later, a jury found him guilty on 14 of 16 accounts.

The power of attorney answered one question that way. Who may sign? Whether this was allowed, a jury answered that years later. Now the same thing only machine checks the signatures. February two, 2016, the central bank of Bangladesh 35 35 payments go on a go out almost a billion dollars all 35 carry the bank's real keys absolutely valid signatures no signature fails five are executed 101 million disappeared dissolves in Philippines casinos. The the other 20 million stopped not by an algorithm, by a typo. An official in Deutsche Bank did not like one word foundation instead of foundation. Signatures were checked by a machine and mandate was checked by a typo. There is a man at the companies who want

to test their uh physical security perimeter. He works uh for different companies. His name is David Olam. The companies hire him to break into the buildings to show how a thief would do it. And u he doesn't touch the lock. He uses a piece of plastic, piece of fold or a plastic cart. Just insert it, insert it between the door and the do door frame into the gap by the latch. I tried it myself. It works. The lock is expensive and certified. It looks perfectly. The problem is the door. The signature is the lock. The mandate is the door. The whole talk is about the door. For 20 years, the internet asked us one question.

Are you human? Remember the traffic lights, the crooked letters? Then the second question, is your phone or laptop, your device, is it genuine? And now the third question, the program acting for you. Is it released official? None of them ask what the pro what the program did. [panting] This program is called an agent. a program. You give a job, answer the mail, book tickets, sign contracts, and to do that, it holds your keys, your card, your pin in its hands. Every signature it makes is and can be real. Who who can say that it states within the job you gave it? So the fourth question is my work. So I tell you the end first. I was for

two weeks on the road. Last two weeks took to Ray Havik then Amsterdam and Sheda Rome University of Pavia. The last one lake commer Switzerland where I finished this talk. So I didn't want to pick the hotel so many times. It's more than five. Imagine I hand it to an agent. A program with my card. The mandate is two lines. [snorts] What? Book a stay for how much? At most 150 a night. Book a stay at nice nicest place you can imagine. So what we usually use manually maybe Google maps to see all prices brokers provide booking com for honest reviews the poorest ones first and maybe airbnb to find places hotels don't have real

gems. So Raikavic a hotel capsule for 120 ma hu campus hotel Rome by the forum all inside all inside the lines and good job agent then como the agent openers Airbnb a villa on the water with its own dock and the possibility to to rent personal yacht. I said I asked the agent to book the nicest. I also said under 150. It read the first. Airbnb checks my card. It's real. [snorts] The bank checks my card information. It's good. One program, one program can only catch it. The verifier. The program that sits between the agent and the money. It opens the mandate a stay under 150. Yes. 9,000. No. I sleep in a hotel for 90.

This is the whole talk. On the road, my agents drafted papers, signed Etsy, the standards body contracts, the client's packages, GDPR consent forms, consent for Aavic interviews. Um, the mandate said, "Draft them, send them to me, never sign." So, my my dear card stayed with me. Now, how I got here and how my own defense broke. Let's visit Las Vegas 2010. Barnamy Jack, security researcher, put two reals ATMs on site. One he opened with the key he ordered online and another one he hacked remotely. The ATM wrote jackpot on the screen and started spitting out money onto the floor, poured out cash. The audience went wild. The machines did exactly what they were told to. Nobody asked whether

it was allowed. Remember the word jackpot. I will come back to it. And my defense was that ATM. I built it myself. So level one, the sandbox. Every action of of an agent gets a fingerprint, just a string of letters and numbers. Change one comma and it changes. Engineers call it a hash. Each fingerprint includes the previous one. That is a chain. Touch one link the chain is broken. One problem a fingerprint is not signature. It is arithmetic. Anything can do it like a medical record. At the foot of every page it says previous record previous page is unchanged written not signed. So rewrite the whole record from scratch. Every note matches again. I did this myself on purpose. The

project says that on its front page, a hash chain only stops an attack on an attacker that cannot recompute it. It lives in my sandbox. An open an open playground on the internet. Anyone can attack my defense there. So, this lock already has a winner. Me. The next lock is different. The second lock is an envelope. Inside it is action mandate and my credential. The credential is my passport who I am and who issued it. Three papers one seal swap one and the seal breaks. Six attacks. One zero edit after the signature like a prescription after doctor signed try to change the content gets refused. Two, three, four. Rewrite the chain. Swap a reach a mandate

tier of the signature. One answer the seal breaks. Five. my own keys as if I issued my own passport. It is not on verifies list. Who who may issue? Rejected. Six. The same envelope twice. First time accepted. Second time already seen. Reject. And seventh. The one I started with a villa on comma for 9,000 a night. Every signature is real. How can you catch it? The verifies the answer and the verifi's answer is four green lines. Wellformed chain is safe. The action evidence package signature is real. Fingerprints match. Then the fifth line mandate fails. The verifier read the mandate. the line no more than 150 in the same envelope under the same uh seal 9,000. No way.

The reason is scope violation outside of the mandate. Remember jackpot? So this is the reverse jackpot. The machine that does not pay and its word is deny. same stamp as Brook Estra's son, not from a jury years later, from a program that stands between the agent and the money. That was the sandbox. Now for the second program. Next to the sandbox sits a second program. I call it passport control. It takes credentials and I tested it properly. 19,999,872.

Almost 20 million different differentially fuzzed tries. The first run found one flaw. I fixed it. [clears throat] Then not one accepted. No crash. Plus, I added 13 scenarios of my own. 25 requirements, 12 attacks, all of them refused. And the paper, the report that this year security has its own drug do drug doier, a security target. The maker writes what the product defends against and how to check it. Then the independent evaluator reads it and I wrote one 25 requirements. First page evaluation ready conformance diff cleaning the standard asks for is missing and along the way I found the bug in someone else's program using the same differential fuzzing as Oracle spec 9+1. Someone else's program was an open

project called Verison devices send it reports I'm not tempered with it decides whether to believe them. You could show it yesterday's report and it believed it like a hospital that admits you on a 5-year-old chest X-ray. I wrote to the developers. A principal engineer from ARM answered ARM the company whose chip architecture is almost in everyone's phone his words we acknowledge the bug and nons check is essential and nons is a one-time number makes each report single use two months later somebody somebody fixed it I found it I didn't fix it the scope so part 20 million tests at this year and I found holes in other now the part you came for a set of team of attacking agents on my

own project not people programs with just one goal to see where I'm wrong they found in a hole my test could not see then I wrote a plain list 11 things any passport control must So a forge signature reject expired credential reject replay reject issue to someone else reject and so on 11 of them I ran the list again the version behind that do this year the one with the green stamp on previous screen my passport control did five six no and here's the one that really hurts a [snorts] credential expired 5 5,000 seconds ago, 83 minutes, my passport control accepted accepted it and believed everything inside. 20 million tests, 13 scenarios, 25 requirements. Nobody looked at the

clock. Why I did not take the readym made check everyone uses a library. The code used by everyone. It does this check for free. But I wanted to write every line on my own to understand every line. And libraries have their own supply chain. They have their own bugs. And in the code, I proudly explain why. The verify is our own. So it's refusal behavior is the evidence. like a doctor who distrusts his own lab and runs all tests by themselves by himself again. The library probably would let would have looked at the expired date. Okay, here's one more screenshot. My wall of fa my wall of shame one exhibit the same dosier. Read the

title, the word in brackets, skeleton. Read the product name. The TOE is targeted target under evaluation wallet name and angle brackets placeholder. I never filled in the name. So conformance div clean and I can tell you exactly how bad it was because I measured it myself. Six failless, one real hole. The remaining five are documentation, the report paperwork like uh the labels labels on a patient's chart. Unpleasant though nobody is harmed. One hole is enough and it's sat right right in the middle. Which brings me again to the bank. [snorts] Bearings, one of the oldest banks in Britain, almost 200 years old. One trader, 20 yearear-old year old Nick Leon, killed it. He made the trades and he keeped the

books on the trades. The losses went to one account number 8888. Nobody in London looked inside it. 827 million pounds almost billion of dollars. More than 200 years of history gone. Sold for one pound. Signed by Leon. checked by listen. That was me. I was listen for my own code. I wrote the checks. I checked the checks and I wrote the docs saying all is well. So what is the way out? Is there a way out? It was not the code. It was the job. My three instruments confirmed 20 million tests, 13 scenarios. The dosier. Tired of repeating it. Confirming was their job. confirmed that the code does what I intended. The attacking team had a

different job. Find what I never intended. An engineer has a picture of what he defends against. An attacker is not obliged to be that polite. Fixed 11 of 11 until when expired date. Here's what I saw. The sandbox read the mandate. So 9,000 against 150 deny the passport control. Never check the end date. Uh credential is a paper with it. End date too. Two [snorts] programs. The second was blind to until when. Now both look good. One more hole. So you don't think I had only one. My verify exists twice. The original one is in Python and uh a port is in JavaScript. The one you can use on your phone. One rule rule book, two honest

translations. They argued over one number, the one on the screen, 10 to the minus minus of 7. Before the finger fingerprint, the envelope is written. So one agreed way Python writes this number as 1 - 07. JavaScript drops to zero. The fingerprints don't match. Verdict denied. Content mutated. Nobody had touched it. I removed the thing they argued about. No floating numbers. No fractions of a euro. Only cents. No arguing. No quirl. Now how you break it? So there are three levels. All code open, no time limit. Level one, ATM.pace. Rewrite the chain in a terminal. The check lets it through. Winner me. Level two, you can check on your phone. Change any field. Press verify. Read the

refusal. Level three, the open attack. Make the verifier say allow to something that mandate does not allow without a key from its list. One seal, three papers. It's not impossible. Nobody yet. Ron Rivst who corrode uh co-rote the RSA 129 a cipher challenge in 1977 offered $100 for reading one secret message they estimated seven quadrillion years actual 17. So the prize I offer is borrowed from Kut, author of the main programming book since 1968. He pays developers who find an error in his code. U hexadimal dollar 256 has programmers count in 16. So paid by the check almost nobody cashed one. They framed them. My first check is already written for the program that found the 83 minutes.

Here's the address tiki institute/lab/ aap-ctf. You can find find the code on GitHub. Scan the QR code to access it. The sandbox. It's it's rather pretty. The page runs on your phone and send sends nothing home. I will not see who tried it. I will not see who broke it. That is by design. So if you break it, you have to say so out loud here today or create an issue on GitHub. And one last thing, remember the man who breaks into doors using a piece of plastic. David Olam, he always carries a letter, a power of attorney. what he may do and until when during his career he was asked only once by a veteran decades living and working

in the building. This guy didn't call the number on the letter. He called the number himself on the number he already had in his phone. He didn't trust the letter. Nobody asked for the letter. Only one man did. So your card is the lock a good lock. It has never once asked for the letter. Now the agent has someone who asks a verifier that looks at the door not the lock also who checks the checker. A problem a program whose only goal is to break. I know that for certain. The first one it caught was me. That is the whole talk. A correctly assigned action can still be an unauthorized action and you can catch

it. The address stays on the screen. Try to break it. Tell me. Thank you. Thank you for the organizers. Thank you very much. Now, do we have any uh questions as well? Anybody? Any questions? Anyone? Uh no hands raised. It was a bit above my pay grade to be honest. So I don't [laughter] have any questions myself either. Uh nobody else. Well, in that case, thank you very much. Thank you very much. >> Thank you.