An Introduction To Information Theory For Information Security by James Bore

hopefully after some earlier technical issues this is working we'll find out uh introduction to information Theory it is a sample those of you who know something about information Theory might get the joke there and this is information Theory it is a systematic treatment of the concept parameters and rules governing the transmission of messages through a communication system [Laughter] didn't get enough information from that because despite being a mathematical Theory this sort of stuff really plays into it so let's try that again it's about communication and communication is the means by which one entity any entity mechanical computational biological affects another and particularly affects the decision-making and the actions of another this is all for the purposes of the mathematical theory of information so if you have philosophical disagreements take it up with Lord Shannon not me information itself is resolution of uncertainty and that's a really important one we'll go into uncertainty and entropy quite a bit but the key thing is that information resolves uncertainty and then entropy is a measure of the reduction of uncertainty how much does that information resolve that uncertainty is it by one bit is it by two bits noise anything that blocks or hinders disarm message transmission that might be from the source it might be in the communication Channel if someone starts screaming or we get protesters coming in that would be nice and symbol and this is an important one this goes into semiotics I'm not going to go into semiopics the key thing is a symbol is any component of a message it could be a letter could be a word could be a binary digit could be an image could be a hand gesture if we were talking about British sign language anything that makes up a message can be treated as a symbol and the message is a collection of symbols nice and tautological there to carry information definitions right moving on there are three problems of communication Theory and information Theory we have the technical problem the technical problem is about how we transmit a message so we send it picks up some noise through whatever transmission method we're using might be electromagnetic radiation might be doors opening and closing picks up noise and the message becomes cloudy and uncertain and then the message that is received May well be completely wrong because that noise has been brought in there's the semantic problem so we have this wonderful statement here the chicken is ready to eat could be KFC could be that you're feeding your pets there's a semantic problem there and your understanding of the meaning of that message will depend on your personal context and your personal choice of how to interpret it you don't want to kill and eat your pet chicken you're not going to read it that way and we have the effectiveness problem I could not work out a way to illustrate the effectiveness problem let me know the effectiveness problem is about how effectively we affect the entity of on the other side and that is effect and affect so how effectively are we helping them decide on a cross-section models actually the Shannon Weaver model is what I'm going to be talking about and really good news for you it doesn't address the semantic problem because that's philosophy and Linguistics and it does not address the effectiveness problem because that's hard problem communication systems if anyone disagrees with the Shannon Weaver model and there are things you can critique about it plenty of them it doesn't deal with two-way communication it doesn't deal with the source or the destination Beyond treating them as black boxes we don't have to answer that right now this is a lightning introduction so we have a communication system we have Alice because of course we have Alice and Alice is our information Source she has a message that she wants to send to Paul so she sends that message to a transmitter yes you may notice this goes a bit fractal because you could argue that the transmitter is also a destination if we go down that route we will be here all day so sends a message to a transmitter where it is encoded in some way you have a thought you send it to your mouth you encode it verbally it is then sent to the receiver via that verbal communication decoded and Central on to Bob who is the destination where does noise come in on this um so yes it also comes in at the information Source Alice can be a source of noise because what we're concerned about is clear communication of the message The Source can corrupt the message can be unclear comes in at the transmission to the transmitter comes in in the encoding process during the transmission in the decoding process transmitting to the destination and finally Bob himself can be a source of Noise by misinterpreting the message noise is everywhere yeah maths this is where I really need my notes so this is the function that defines Shannon anything Shannon entropy is that measure of the reduction of uncertainty of information [Music] few key things the variables we're dealing with are what are called discrete variables that means they are countable so they are not continuous the moment you go to continuous this breaks down because it doesn't do Infinity as well so all variables are discrete it might be one or zero it might be one to a million it might be blue red green but they are countable in some way H there is the symbol used for Shannon entropy function the x is a particular discrete variable that we're applying it to and what we do is we take that we take the probability of each outcome of X so if it's red green blue let's say there's a one-third probability of each there's one or zero flipping a coin 50 50. it might not be even probabilities maybe we've got a weighted coin and it's 90 10 but you take those probabilities you sum them up you multiply them by log B Because magic V there is a base so if you use two as the base you get the Shannon entity measured in bits if you use e as the base you get the natural Channel entropy units if you use 10 as the base you get bits because digital okay some of them all up multiply by log B and you get challenge for the ABS it so you get a number what that number tells you is how many bits you need to carry or encode that message at an absolute minimum so what is the information theoretic minimum amount of data you can transmit to carry that message so with that coin flip if it's a fair coin or even if it's not it's always going to be one you only need a single bit to say whether it's heads or tails as long as you've agreed what one and zero means if it's rolling a dice it's going to probably be one well actually it will be I'm not going to do the maths in my head it will be more than one because you need more than one bit to carry it so even though this has probabilities in it we are dealing with results which can be greater than one uh English language as an example has a Shannon entity somewhere between three and six or if you're Shakespeare about eight and that was done calculated by Sherman based on three thousand words because he took a sample because there's too many otherwise but you can work out the Shannon entropy of languages and it tells you how much information can be conveyed by a short message another way to look at it is the Shannon entity is the number of symbols in a message that you need to have a 50 50 chance of getting the next symbol so how much information do you need to have a fair chance of guessing what's coming next this is still part of Information clearing here we have Channel capacity to Italian so Channel capacity bandwidth how wide is the channel this was built for telephonic systems so it's about frequencies but we can also apply bandwidth to other things conversation has a very limited bandwidth ultimately it's got a bandwidth of about one because we can't focus on all the ones that laughs you do some maths you take the signal to noise ratio because that's important because this is all about communicating in the present noise and you get the maximum theoretic Channel capacity output how much information can we pass down that channel this is used in telephony this is used in just about anything that involves communication and Technology it's also the information theoretic excellent that you can pass down it we have not achieved that with any channel as yet we know that you can't exceed it it's kind of like the speed limit of the universe is the speed of light this is the absolute maximum mathematical amount that you can pass down the channel so what do they tell us well entropy of data is its compression limit you cannot compress information further than its entropy without losing it it becomes lossy and the channel capacity maximum theoretic transmission if you manage to achieve the maximum compression and make use of the maximum capacity you're going to be transmitting really fast that's good extent coding I am blazing through this because this was prepared as an hour long so the coating is you replace a symbol in a message with a coded symbol and cryptography we're replacing it with a secretly coded symbol still a coated symbol is still coding you can replace any symbol with any coated symbols as long as the recipient and the transmitter have the same code book Morse code is an example of this you can replace any letter in the alphabet you can also depend on the code book replace whole sentences with single symbols symbol might be a word it might be an image might be a letter it's we've been through that one and it can involve algorithms to increase the bandwidth of a channel so while I said we've not hit the information theoretic bandwidth or we're not using it 5G actually sees the theoretic bandwidth of the spectrum it's across because it's using spread Spectrum encoding it's using multiple bits and it's also using both frequency and wavelength for encoding whereas the original equation only allowed for a single one of those to adjust it was about radio or it was about to definitely now we reduce things to data and we can do much more clever stuff audience participation time you have the answer to this why should you encrypt before you compress information means yep that's part of it it's also that when you encrypt you increase the appearance of Randomness the best encryption will make a message appear completely random which means it has an extremely high entropy when you confess your aim is to maximize entropy so if you encrypt first you can't maximize the entropy as effectively because you've already increased it so you want to impress first and then you can encrypt it's also faster to encrypt compress stuff because there's less of it uh that's it any questions you're talking about loss lossless smooth compressions yeah thinking about things like jpeg which is glossy but still get the effectiveness yes so you don't always need all of the information so jpeg is a lossy encoding method or lossy compression method ultimately anything where you're taking analog data and turning it digital will be lots of unless you've got an infinite length you cannot compress all of that information so JPEG and MPEG are perfect examples they are designed to maintain Effectiveness selfie Effectiveness problem while still allowing data to be lost the algorithms and codecs used to do it are really clear that they do things like check when colors are reused facial recognition when you get audio on the phone you won't get a lot of frequencies all of that sort of stuff another one anyone here know about thermodynamic entropy the idea that the Universe has an energy Arrow going in that in One Direction yeah so fun fact the entropy in information Theory works pretty much the same way and there is a thought experiment and I won't go into too much detail but essentially with one bit of information you can do a certain amount of thermodynamic work the effort or the energy required to get that single bit of information is exactly the same as that thermodynamic world so if you had a perfect system then locally information transmission can reverse entropy locally but on a wider scale is increasing it because you're having to do the work to get the [Music] anyone else so in the um Channel we've moved for human to human encryption why you've got noise occurring at every point along the transmission is that more a case then for humans that were kind of it's not really noise it's more like not having the right decryption key at the end because the noise is in fact psychological cultural and physical possibilities there so you could to successfully decrypt you need to have all of those keys in place yeah so culture is a code book language is a code book we all have similar but not identical ones so yes you will have that um there is actually one form of ungrateful encryption called the one-time pad you may have heard of it ultimately if you really wanted to dig into it you could argue that a particular culture their code book is a one-time pad if they're not understood by anyone else yes you can learn it and you can break it but it's going to be larger than any single message they're transmitting so you cannot understand a wider Culture The Wider code but Consulting messages so is that where the safety a wolf hypothesis comes in about language-shaped people now we're going down into the system um the answer is yes and that's where we very rapidly move away from the maths and into psychology and philosophy which I love it for enough bit yeah I'm so sorry um but no it is that if you get into semiotics there's this really annoying problem that it symbols all the way down so when you're communicating I'm using words they are symbols they come from thoughts well as far as we know they're actually a sort of symbol encoded in the brain and Below them more symbols and more symbols and more symbols and we've got no idea whether there's anything at the bottom Turtles all the way down for anyone who knows that reference yeah right one other thing uh if you're interested in information Theory possibly from a slightly more technical event I have a range for this book to be free until Monday so you can grab it off Amazon happily or graphic in the QR code or you can look it up uh I would say it's a special deal for this peace item it is because it's scheduled for this for anyone can grab it so feel free to share link around I didn't write it was it your yeah it was actually my dad who wrote it he's why I know too much about information Theory I'm familiar with it and there's various other ones around as well I can't remember where they got to with the crypto sculpture oh it's going to ask you what your video is on their latest oh that might happen to call Southwest okay um but there is actually a fantastic book you can get they've just relaunched it it's nothing to do with the cryptos one but it's about 100 pages long six murders six motives in it pages are not in order so the puzzle is to try and put the message together so it'll reassemble essentially to reassemble friends you were doing the same thing as a happy switch Network okay we've got one and a half minutes left if there's any other questions otherwise I'll just partly sleep off to the side so is it fair to say that noises are inherent part of any system and you're always going to be condensating for those it's never really noise will always be part of any system you will never have a noiseless system and noise is also equivalent to thermodynamic entropy so Lloyds will only ever increase in the message without active work being done to decrease it [Laughter] foreign [Music] okay thank you [Applause]