Skip to main content Skip to navigation

Department of Statistics

Material from previous years (2016/17) - for archive only!

Week 1:

Tuesday/Friday: What are decisions under uncertainty? Some examples: investment, job offer, sport betting, non-emergency medicial treatment (e.g. chronic pain surgery, immunisation), simple game, St Peterburg paradox. In these examples, we have see key concepts that will be picked up later during Part I of this module. What is relevant in taking decisions? Mathematical models based on outcomes, (subjective) probabilities and values of these, utilities and risk attitudes are used to deteremine optimal (in some sense) decisions. It has some, but not full validity for homo sapiens, the species actually populating this planet, and issue we will pick up again in Part III.

Friday: What is probability? Normative probability (Kolmogorov's axioms) based on set algebras to model events as subsets of outcomes and probability measures as functions of these. In other words, probability measures are functions on set algebras. Concept of atoms as fundamental events. House price decision model as an example for a very simple finite algebra on an a priori more complex (continuous) outcome space. Examples to illustrate the use axiomatic probability. (i) Two-headed coin: From a box with n-1 normal coins and one two-headed coin you draw a coin at random and toss it three times. It shows three heads. Which box do you think it came from, and how sure are you? Generalisation to k instead of 3 tosses. The example is discussed in Section 2.5.1 of the ST111 Lecture Notes (see r.h.s.). This is decision problem of the type seen in statistical tests, where the task is in "re-engineering" which was the random process that generated an outcome using likelihood ratios. (ii) Nick & Penny: Flipping coins with different probabilities for heads, p1 and p2. Model for infinitely many coin tosses as 0-1 sequences obtained as an (infinite) product (sigma-)algebra. The number of heads in n coin flips distributed following a Binomial distribution with parameters (n, p1) for Nick and (n, p2) for Penny (seen in first year probability). Determine the distribution of random variables N = the first time that both Nick and Penny obtain heads, and of random variable M = first time at least one of them get heads.

Resources for week 1:
Study advice for week 1:

If you do not feel entirely confident with the axioms of probability and the calculations in the examples, read also the simpler examples in the lecture notes and/or review your first year (discrete) probability notes. And then tackle exercise sheet 1.

Questions by students in week 1:
  • What is a nickel? A 10 pence coin. A silly joke in that example with Nick and Penny...
  • Why do we need that infinite product of algebras in the Nick and Penny example? We could set an upper limit to the number of coin tosses, and then we would not need that. However, if we want to look at random variables such as M and N above, or even simpler ones such as the waiting time T for the first head, they do not have an a priori upper bound. While it's extremly unlikely that it takes Nick 1 million tosses to obtain the first head, the probability for this event (T=1 million), is larger than zero. Of course, you could obtain arbitrarily good approximations by setting increasingly large upper bounds, and that's a common technique used on proofs of asymptotic results.
  • Why do we need algebras at all? Can't we just define probability measures on the collection of all subsets of the outcomes space? Firstly, examples without finite horizon, such as the coin tosses above, the outcomes space is the set of all 0-1 sequences. This is of the same cardinality as a continuum, e.g. the interval [0,1] or the real numbers. The same is true, for example, about a number generator, that is, drawing a random number between 0 and 1. A deep results in measure theory says that it is impossible to construct a probability measure on all subsets of the real numbers. This is far(!) beyond the scope of ST222, but if you want to read about this surprising theory, then read about the Banach-Tarski paradox and Vitali sets. It all starts with cutting chocolate. Secondly, there are practical reasons. Even if you were in a situation where you could define probability measures on all subsets on the outcome space in theory, you may not know in a given example, what they would be. You may determine the probabilities based on historical data or on believes people have, and that may not be available for all possible events. In the stock market, for example, you have precise information about the past and the present, but you only have rough or no information about the future. Or a person may have knowledge about some events, but not about others. To model situations with such incomplete information, you need to restrict the collection of subsets of the outcome space. But you can't just take any collection, you do have some basic rules about this (to ensure probability measures can actually be well defined on them), and they are the properties that make up the defition of algebras (and sigma-algebras).

Week 2

Tuesday: Continuation of the Nick & Penny example (ii) shows that the random time until both of them get heads is a waiting time and has a geometric distribution with parameter p1*p2. The random time until at least one gets heads is also a waiting time and has a geometric distribution with parameter (1-(1-p1)(1-p2)). (iii) Class size example shows how two different perspectives can lead to different answers to a question if it is not phrased precisely enough. One could estimate the size of a class picked at random from all classes. Or one could estimate the size of a class a randomly picked student sits in. Depending on the class size distribution, this can lead to very different answers, a phenomenon called size bias, which is also behind the bus paradox. What does probability mean? Classical probability refers to the calculus of probabilities. Axiomatic probability is the rigorous mathematical set up for probability as a function from an algebra of events to [0,1]. There is also the frequency interpretation of probability, which can not serves as a stand alone definition, because it requires the law of large numbers to ensure existence of the limit of average numbers of relative frequency of an event in the first n trials as n goes to infinity. However, it serves as an interpretation.

Friday noon: Then notion of subjective probability puts an emphasis on subjectiveness and individuality. How can we quantify a person’s degree of belief an event will happen? The key idea is elicitation through bets. The subjective probability for an event A is defined as the amount of money the person would be prepared to pay to enter the bet b(M,A) which pays M if A occurs and nothing otherwise. This is assumed to the same as the minimum the person would ask someone else to pay when offering that bet - technically intuitive, but actually a psychological assumption. After normalisation, and assuming there is not a nonlinear effect of M, this leads to a concept of probability that can be shown to obey the axioms of probability. We introduce the notions of collections of bets and of equivalent bets in the sense of giving equal rewards and study examples. Is the resulting probability mathematically sensible? We proof that under basic rationality assumptions, that people are willing to trade equivalent bets, subjective probability is additive, and hence fullfill the probability axioms.

Friday 4pm: What happens if a personal has irrational beliefs about probabilities? We look at four examples. (i) In coin toss, J believes P(heads)=P(tails)=0.6. Students sell her the collection of bets b(1,{H}) & b({T}) for £1.2 which guaranties them a sure win of £0.2. (ii) Similar, but no J believes both probabilities are 0.4. Students get her to sell them for $0.4 each, which again gives them a £0.2 sure profit. (iii) Let Omega be an outcome space. J believes P(Omega)=a<1. She is hence willing to sell b(£1,Omega) it for £a to the students, who end up making a sure profit of £(1-a)>0. (iv) Similar, but now instead of a it is a’>1. Students get her to buy from them b(£1,Omega) for £b and make a sure profic of £(b-1). A Dutch book is collection of bets for which one party can not loose. As examples we look at a lottery ticket seller and at a bookies strategies in horse betting using an explicit simple example with 4 horses.

Resources for week 2:
Study advice for week 2:
  • If you forgot most of your probability theory, review it (e.g. use your own material from the first year, texbooks, ST111 lecture notes (2012)).
  • Read additional examples in the ST114 lecture notes not covered in class, such as an example for Dutch book using a race (with creative horse names such as "Go Lightening").
  • Find ways to contruct Dutch books in real life. For example, elicit probability believes of your house mates, friends or family and see if you can benefit from these.

Questions by students in week 2:

  • Why does the bookie make a profit? Because he set up the bets this way. He gets all the betting prices summing up to £210. But whichever horse wins, he only has to pay out £200 to the winner. In the languate of probability, he constructed a world where people pay for bets that add up to 1.05 total probability rather than 1, and he benefits from the extra 0.05.
  • Does this always work? What if unequal numbers of people bet on the hourses? You are spot on, I run out of time and didn't complete the discussion. Will pick that up again next week. Bookies need to adjust their odds depending on how many people buy bets on which horse. Also, they need to adjust them if a horse drops out before the race.

Weeks 10-3

They will soon be erased to make room for this year's summaries. However, they will stay here until the end of week 2 so you can find more information about ST222 while finalising your module choices.

Summary of content ST222 in 2016/17: Short summary sheet

Week 10 (scroll down for earlier weeks)

Tuesday: Framing of contingencies, introduction to prospect theory. We consider three different preference choices between lotteries. Two of these are mathematically equivalent, but one is framed as a two stage process. This leads to subjects making inconsistent choices. Instead, they make the same choices as they do in a set up that only consists of the second stage, a phenomenon called isolation effect. Behind the preferences itself is the certainty effect. The mathematical basis for prospect theory consists of two ingredients: an S-shaped probability weighting function w capturing the underrating of probabilities close to but not equal to 1, the overestimation of probabilities close to 0, and the unrealistic low increase for average probabilities. While there is a continuum of suitable functions, we introduce two common ones (by KT and by Prelec). The value function v replaces the utility function in EUT; special features include the reference point and the asymmetry between gains and losses. These functions have been validated by lab experiments. Multiplying the values of outcomes with their weighted probabilities leads to the value of a prospect, which corresponds to the expected value of a decision in EUT. 

Friday noon: Common consequence, common ratio, modelling. After reviewing the prospect theory model, we introduce abstract versions of paradoxes initially formulated by Allais. In many studies, a majority of subjects does not follow EUT in any of these situations. In contrast, PT (prospect theory) can explain their choices. We give an explicit example for w and v that explains the choices in the common ratio situation. A similar construction can be done for the common consequences situation (exercise sheet 5). In these examples, we did not even need to use a value function v other than the identity, but it was crucial to invoke a nonlinear probability weighting function w. These provide two classes of examples, where prospect theory can explain behaviour driven by the certainty effect. In contrast, the value function is crucial to explain the framing effect. How can biases in decision making be reduced? We need to address this in the whole process of decision making. In the editing phase information is collected and prepared. Biases can arise for example from changing or misunderstanding information and from selection of sources or insufficient monitoring of their quality. In the evaluation phase probabilities and values of outcomes are being combined. It is mostly in this step here that mathematical training can improve the process. Emotions have been shown to interfere with both editing and evaluation phases, also known as affect heuristics. For example, sunny weather has been shown to affect the stock market, stress has an impact on preferences, and feelings of insecurity lead to a decreased tolerance of cognitive dissonance hence enhance confirmation bias. Mental balance of the decision maker usually improves the decision process.

Friday 4pm: Exercises, modelling.

Resources for Week 10 

Study advice for Week 10

  • Which mathematical models do you know? Are they appropriate? Have they been empirically validated? How?
  • Solve problems from exercise sheet 5 (will be posted by end of this week)

Questions/comments by students in week 10:

  • Summery of results of the questions (corresponding to "Problem 5, 6, 7" in the lecture slides) distributed in Lecture 1.
  • What if they could entre the lotteries in the questions (distributed in lecture, see above) many times? Would people make choices following EUT more closely? Answer: I suppose yes, but it would have to be tried.
  • Why is the probability weighting function continuous? Answer: It's a model for something we can only indirectly observe. Building models means we want to have a simple but sufficiently complicated representation of the reality (here people's minds during decicion making under uncertainty), that can describe and predict such observations. A discontinuous probability weighting function would be a different model, but is appealing in that one could treat p=1 entirely differently from p<1. However, sticking to continious - actually, even better, differentiable functions - has the advantage that we can apply techniques from calculus.
  • Are there any field studies where prospect theory can explained observed behaviour better than expected utility theory? Answer: Here is a collection of study summaries.
Week 9

Tuesday: Paradoxes, heuristics and biases. Normative and descriptive theory of decision making exist in parallel coming from different angles. In addition, the prescriptive approach sitting at their intersection aim to train humans to make use of probability in the normative sense. The example of judging sample variation shows that this can work: Warwick ST222 students with training in probability gave much better answers to a question about the dependency of sample variation on sample size initially used by Kahneman and Tversky (K&T). However, a majority of Warwick ST222 students showed the same choice patterns in the Allais paradox as subjects in the literature, that is, not following EUT. In an experiment by Ellsberg with balls in urns subjects also show what looks like an inconsistent preference pattern that can be explained by ambiguity avoidance. In addition to randomly drawn outcomes, this set up features additional ambiguity by concealing the exact composition of colours in a bet.

Friday noon: Base rate neglect, reason based choice and more incoherent choices. In many situations, people forget to take into account the initial probabilities (prevalences) of outcomes or swap around the role of events in conditional probabilities. This can lead to very incorrect probability judgements. Further examples for inconherent choices include an availability bias in numbers of path in different graphic structures, resonses to lost bills/tickets and to missing a flight. Another question is the motivation behind decisions. It has been found that people make faster decisions in conditions where one options clearly dominates the other. In conflict situations, people hesitate and will even try to obtain more options. Uncertainty about another outcomes, even if irrelevant, can also delay decision making.

Friday 4pm: Conjunction fallacy, representativeness, confirmation bias. The Linda problem and its variations demonstrate how people give a higher priority to matching different parts of a story than to following the axioms of probability theory. More specifically, they chose rankings that suggest that P(B) is smaller than P(B and F), even though it should be the other way aournd.This has consistently been demonstrated under different types of experimental conditions and for subjects of different levels of justification, including ST222 students. It can be explain by representativeness theory. Another important and traditional phenomenon is confirmation bias, a tendency to select, weigh and deform information on order to, unconciously, make it conform to pre-existing beliefs. This is driven by human desire to minimise cognitive dissonance, though there is a lot of variation between humans when it comes to how much internal contradictions they can tolerate.

Resources for Week 9:

Study advice for week 9:

  • Use some of the questions considered during the lectures to test how your friends and family answer and try to find out from them what motivated their choices.
  • Find examples where you delay a decision (D) because you are waiting for another decision (D'). Determine whether D' was actually relevant for D. Can you find examples where D' was irrelevant for D, but you still waited?

Questions/comments by students in week 9:
Please submit if you have any...

Week 8

Tuesday: Overview of Part I, II and III, normative versus descriptive theory, models of human beings. We look at overall structure of the module connecting including Part I, II and III. Part I is based on the assumption that human beings are rational, in the sense of maximising their expected rewards. The validity of these assumptions will be challenged in Part III of this module. One of the learning objectives of this module is mathematical/statistical modelling. A model is not intended to be perfect, but to be a simple representation of a part of the real world; good enough to make explanations and predictions for specified aims. To put the modelling task in the context of our objective to understand human decision making, we first ask a more general question: What are models of human beings? To illustrate on a very concrete example what this could be, we look at the mathematically inspired homme moyen that forms the basis for Le Corbusier's architectural theory. The rationality assumption we used in Part I is also based on a model of human being, nicknamed homo economicus, which is very common in traditional economic theories (that means, before the rise of behavioural economics).

Friday noon: Imitating coin tossing. We conduct an experiment to get some experience with the process of generating sequences obtained by repeatedly tossing a fair coin. In one part of the classroom, students generate by themselves sequences that look like sequences of coin tosses (singles condition). In the rest of the classroom, students do that same but work in groups of three (trios condition). Mathematically speaking, we aiming to generate 0-1 sequences from independent identically distributed (i.i.d.) random variables with values 0 and 1, with equal probabilities for the two outcomes. Given a 0-1 sequence, how can you tell whether it was created by this model? You can not tell for sure, but you look at characteristics of the sequence and compare this with what be typical for a sequence created by the model. You can quantify this using probabilistic calculations and use the obtain probabilities to make judgements about the processes underlying the generation of your data (models). In sequences with just two outcomes, the main characteristics is how often they alternate or, in other words, how often they do not alternate. For example, we can look at runs. It is relatively simple to calculate the expected number of runs of a given length using indicator method. A more complex question is the distribution of the longest run. We derive a recursive formula that enables us to explicitly calculate the cumulative distribution function (CDF).

Friday 4pm: Fallacies (Gambler's fallacy, hot hand, clustering illusion, perception of random patterns and sequences, anchoring bias, framing effect). The gambler's fallacy describes the belief that after a long run of outcomes of one kine, the other kind of outcomes because more likely. In the case of a sequence of independently created events such as coin tossing or roulette, this is wrong. People typically believe this, because of their intuitive understanding of what the law of large numbers says, that relative frequencies converge to the theoretically expected probabilities. However, it is wrong to apply this thinking on small samples. For independently sampled random variables, there is no memory in the sequence that would change the probabilities of outcomes to balance things out on a finite horizon. Of course, there are also games where independence is not true, such as Black Jack, which needs to be modelled taking into account that cards are not replaced. Practical applications of gambler's fallacy in the 'real world' have been found in situations where people make a lot of similar type of decisions (judges, load officers, referees etc). The hot hand as a form of opposite of the gambler's fallacy has been investigated in sports. Poisson point patterns tend to look not random to viewers who did not yet study this process, because they have more clusters than people think would occur at random. The perception of random sequences can differ depending on whether or not students could see the whole sequence. If being asked to create black & white sequences, 90% of people start with black. This is an example for anchoring bias. Traditional examples of anchoring bias involve estimations of geographical facts (e.g. length of Mississippi river) or numerical calculations. Surprisingly high differences in answers are observed depending on people were exposed to smaller or larger numbers, or even entirely unrelated tasks involving length, before being asked the question. Finally, the framing effect is introduced with the classical example by Tversky & Kahneman about a disease prevention programme.

Resources for week 8:

Study advice for week 8:

Questions/comments by students in week 8:

  • Are we going to learn about prospect theory? Answer: Yes, in particularly we will introduce the mathematical model of probability deformation.
  • How do we really know whether a given sequence is really from a fair coin? After many heads in a row, maybe one should conclude that it is actually two-headed coin? Answer: You can never know anything for sure. What statistics can do for you, however, is to tell you how likely your observations are under a certain model. This is the basic principle of Starting from observations (data), extract characteristics or summaries and then calculate how likely these were generated by a model. Compare different models and choose the most likely to explain the data (Maximum Likelihood Principle). We actually did some toy examples for this in Week 1 involving two-headed coins. There are also methods detecting dependency. However, it can be difficult to distinguish, for example, between time inhomogeneous probabilities and dependency just from observations (model unidentifiability) and contextual information may have to be invoked to decide on a most credible model.
Week 7

Tuesday: Separability. Our set-up for games is so general. To find out more about properties and strategies, we need to look at special classes of games. The concept of separability describes games where there is no interaction between the players' decisions. Formally, this is captured by a decomposition of the reward function (which depends on both players' moves) into a sum of two reward functions that each only depend on the move of one of the players. Note that these functions are not unique. The prisoner's dilemma is an example for a separable game, as can be seen by an explicit construction of such function. We derive necessary conditions for a generalised version of the prisoner's dilemma to be separable. As an immediate consequence of the additivity of the expected value, we obtain a simplified formula for the expected rewards of a separable game.

Friday noon: Domination, purely competitive games. The concept of domination captures that some moves may be universally better or equal for a player than any of the other moves in the sense that regardless of what other player does they lead to a better or equal reward. Dominant moves should be played, because their expected reward is at least as high as it would be for any other move. This is a direct implication of the monotonicity of the expectation. A move is domimated if there is another move that dominates it. Moves that are dominated by others can be eliminated from the game in an iterative process. The use of these techniques can potentially lead to a simplified game matrix or even to a discriminating strategy. Examples show the techniques in action. Note that the assumption underlying all this is rationality of both players in the strict (and debated) definition that rationality can be reduced to the aim of maximising expected reward in each round of a game. Another important class are zero-sum games. They are a normal form (rewards adding up to 0) of what is captured in the concept of purely competitive games, where any amount that one of the players wins is at the other player's loss. A maximin strategy means to choose a move that brings a maximal reward over the worse cases determined as a function of what the other player does. In RSP, for example, that leads to the unfortunate outcome that each of the players expects to loose. Obviously, they can't both loose! What has not been taken into account is the interaction between them.

Friday 4pm: Solving zero-sum games, mixed strategies, fundamental theorem. To resolve the paradox occurring with maximin strategies we extend the available options to probability distributions of moves (mixed strategies) instead of only deterministic moves (pure strategies). The expected rewards can be expressed as expectations of the rewards over these distributions. Maximin for Player 1 corresponds to minimax for Player 2. The strategies can be calculated for each of the players and an obvious questions is whether the corresponding expected rewards are the same. This feels intuitive for zero-sum games and is indeed the case, as stated by the fundamental theorem of zero-sum two player games. However, the proof is not simple. One inequality yields in more generality and can be shown easily in a lemma. The other direction is indirect and based on a smart contruction of a version of the game which then leads to a contradiction invoking the separating hyperplane theorem for convex sets. Note that the proof of this theorem is not constructive. It does not actually provide the mixed strategies where the maximin/minimax is attained, but only demonstrates their existence. Also, note that the maximin attained is unique and the same for both players, but the strategies do not have to be unique.

Resources for week 7:

Study advice for week 7:

  • Write down some random game matrices and check if they are separable.
  • Construct game matrices with dominant moves. Find one that has dominant moves for Player 1, but not for Player 2. Find one that initially does not have dominant moves for Player 2, but does so after dominant moves for Player 2 were removed.
  • Look at shapes in your environment and decide which of them is convex. Start at breakfast: a bun? a croissant? a bagel? a pretzel? (Do this everywhere and long enough with every shape you notice, and you life will have changed forever! You can not revert back to not seeing convexity wherever it is.)
  • Can you see why the Separating Hyperplane Theorem (see slides of the proof of the fundamental theorem) needs the convexity assumption?

Questions by students in week 7:

  • How do I find these functions in the definition of a separable game? Answer: You can stare at the numbers and guess and this works sometimes... Or you can rewrite the condition as a system of linear equations, which you can than tackle with standard methods you learned in linear algebra. The system may not have a solution, which implies the game is not separable. If it has a solution, no matter whether unique or not, then the game is separable.
  • In an economics module we learned a definition for strictly competitive similar to the definition of purely competitive in this module, but involving inequalities. What is the connection? Answer: It looks like the class you defined in economics is more general. It is a less quantitative condition. If player 1 gains, then player 2 looses, but unlike in our definition, the loss does not have to be by the same amount as the gain. Taking away this equality covers more examples and is more applicable, but it makes it much harder to develop precise theory as for example today with the fundamental theorem of zero-sum games. Of course, you could use the theory for purely competitive games to derive approximate answers or upper/lower bounds for questions about a strictly competitive game.
Week 6

Tuesday: Archimedian axiom, independence, von Neumann-Morgenstein representation, Allais paradox. Introduction of two more properties of binary relations that are needed for a more explicit representation of utilities. The Archimedian property describes a preference order where, given any strongest x, medium y and weakest z, mixing in a little bit of the weakest z to the strongest x does not reverse the preference of the strongest over the medium y. In other words, the weakest can not be incommensurably weaker than the medium. A corresponding property holds on the other end, when mixing in a bit of the strongest x to the weakest z. There is a conceptual connection to the Archimedian property on the real numbers, which says that for any epsilon > 0 and any real x there is an integer n such than n*espsilon > x. Examples where the Archimedian property is not true can, for example, be generated by including outcomes that are hard to compare on the same scale such as monetary outcomes and death. The independence property captures the phenomenon that adding an additional option on both sides of a preference relation does not change the preference relation. Examples where the Archemedian property is not true occur, for example, when there is some kind of interaction between the additional option and one of the other ones. For example, providing a can opener is a useful addition to a can of soup but not for a load of dry bread. The von Neymann-Morgenstein representation theorem says that preferences relations that also these two properties can be represented as expected values of a utility function directly defined on the outcomes, with respect to the probability distributions that define the actions involved. In his 1957 seminal paper, the French economist Maurice Allais introduced en example involving lotteries challenging the dogma that humans make their choices based on expected utility theory. He conjectured, that people’s preferences in two pairs of lotteries would not be following expected utility theory. His conjecture has been confirmed in many empirical studies with human subjects in the sense that a large majority of people made their choices according to Allais’ predictions. Demonstrating that this contradicts expected utility theory amounts to showing that there exists no utility function that could represent them by constructing a contradiction between two inequalities derived from the conditions following from people’s preferences. The set-up for this was done in lecture, but you need to complete the calculations as homework, and you will also discuss that this can be connected to the independence property not being generally valid (see exercise sheet 4).

Friday noon: Classtest. Results will appear online within 4 weeks. You will receive an email notice.

Friday 4pm: Introduction to games. What is a game and how do we describe this mathematically? Games we are thinking of are rock-paper-scissors, board games, card games, prisoners dilemma. From a decision theory point of view, a game is just two (or more) people are taking turns making decisions. We consider games of two in this modules. For each combination of decisions, there are reward functions for each player which leads to a description of a game as a payoff matrix. Through playing both RSP and prisoners dilemma for a few rounds, we discovered and experienced some of the concepts that are relevant to describe and analyse games: sources of uncertainty, hypothesising about the opponents moves, iteration of this strategy and consequences, competetive versus cooperative games, the role of trust, the consideration of long term and short term goals and different implications, the role and the value of information and the gain of information as a potential intermediate gain in repeated games. If Player A has a probability distribution for the move Player B will make, then player A can use this to calculate his expected reward and vica versa. Natural questions are when expected rewards are independent of such distributions and what is the role and the implications of assuming rationality.

Resources for week 6:

Study advice for week 6:

  • Continue to solve the problems from Exercise Sheet 3.
  • Play games with your friends, family, roommates.
  • Explore playing RPS against a computer that was trained on humans.
  • Think about what distinguishes games from other games. What are sensible classes for payoff matrices?

Questions by students in week 6:

  • Can the payoffs be negative? Answer: Yes. And note they are not always uniquely defined. They are just a description of a game. For example, in RPS, we can use -1, 0, 1 to code for loss, draw, win, but we might have used other numbers.
  • What about utility? Answer: Yes, very important point. You could convert the given payoff matrix by replacing each players rewards by their utilities. Since the players may have very different utilities based on their personalities or circumstances, this may fundamentally change the game. A fair game may become unfair, for example, or what was an advantage for one player by become an advantage for the other.
Week 5

Tuesday: Certainty money equivalent and utility. Look at certainty money equivalent (CME) as a function of p. Example of a naive’s person’s CME for bets based on rolling dice, which shows a convex shape. Last time we defined utility as the inverse of CME. We can alternatively start with a utility function, and construct CME m as a function of probability through the equation U(m)=E[U(b(p)], where b(p) is the bet that pays £t with probability p and £s otherwise, for fixed parameters s, t. Both CME and utility are subjective context dependent functions. We consider the example of buying fire insurance for a house. The contrast between the utility for the owner of the house the the seller of the insurance explains why there is room for a deal. The owner is risk averse, because relative to the owner’s wealth the house means a lot. The insurer is risk neutral, because this deal is just one of a huge number of similar deals, hence it is justified to approach this with a simple expectation of wealth without further transformations. Risk aversion corresponds to a concave utility function and risk neutral corresponds to a linear one. A different example is lottery. Similar calculations (see resources below) show that playing the lottery can be explained with risk seeking attitude (convex utility).

Friday noon: Binary relations and their properties. We define binary relations on action spaces (e.g. spaces of bets) and introduce some of the fundamental properties. Asymmetry (A) states that given a relation, the opposite relation is wrong. Completeness (C) essentially means that for any two actions x and y, people need to make up their minds whether x is preferable to y, or y to x, or whether they consider them being equally preferable. While this sounds banale, it is not always justified in practical examples. For example, not all people can make sense of the alternative “be stupid and satisfied” or “be smart and dissatisfied”. Also, choices must be available at the same time in the same location (e.g. “foie gras” and “hot dog”). Transitivity (T) means that a relation between x and y combined with a relation between y and z implies that same relation for x and z. Rational people should have transitive preferences, because otherwise their preference would have cycles and one could engage them in a series of exchange trades that would cause them to loose money with certainty. However, transitivity may not be always true in real world situation and needs to be checked when preference are used in models. For example, relations like friendship, or multi-attribute based preferences (e.g. ijkphone example), may not be transitive. Also, incremental changes can clash with transitivity of the relation equally preferable. For example, coffee with n grains of sugar is equally preferable to coffee with n+1 grains of sugar for all n, but iterating transitivity would imply that coffee with 1 grain of sugar is equally preferable to coffee with 1 million grains of sugar, which is probably unrealistic for most people. Negative transitivity (NT) is a condition similar to transitivity and can be linked to transitivity when (C) and (A) are also assumed.

Friday 4pm: Preference relation, numerical representation, representation theorem, lexicographical order. A binary relation with (A), (C) and (NT) is called preference relation. A numerical representation of a preference relation is a function mapping the actions to the real numbers that preserves the order induced by the preference relation. We proof a theorem saying that preference relations on action spaces that have a countable order dense subset have a numerical representation. The numerical representation is not unique, which can be seen by going through the proof (several choices where made), or by showing that from a given numerical representation we can easily construct more (e.g. by multiplying them with a constant). The lexicographical order on the plane is defined by first ordering by the first coordinate, and if tied the second coordinate. We show that this order does not have a numerical representation.

Resources for week 5:

Study advice for week 5:

  • Think about what is your utility function in some real world situations. Compare with those created by your fellow students. Consider situations with gains and situations with losses.
  • Solve the problems on Sheet 3

Questions by students in week 5:

  • In the proof that there is no numerical representation for the lexicographical order you construct a map by saying for every r there is a q between u(r,2) and u(r,1). Is the map well defined? Is that unique? Answer: It is well defined, but it is not unique. As the choices are still countable, you do not even need the axiom of choice to justify you can pick one of the choices.
  • Can you give an explicit construction of this q? Answer: Yes, we can. See these notes (not covered during the lecture and not examinable).
  • Why does the theorem not apply to the lexicographical order? Which condition(s) of the theorem are not valid in this case? Explain why. Answer: Will appear here by Wednesday, try yourself first.
Week 4

Tuesday: EMV decision rule, reward perspective, decision trees, eye disease treatment decision. As decision rule is in an algorithm to determine a decision under uncertainty based on a set D of decision options, as set Chi of outcomes with (subjective) probabilities p_i and a loss function L. One very common decision rule is the expected monetary value strategy (EMV) which defines the optimal decision d* as one that minimises the expected loss E[L(d,X)]. Sometimes, it is more natural to phrase the consequences using a reward function R rather than a loss function. No new model is used for this, but we simply define L=-R and obtain that d* maximises the reward. Decisions under uncertainty can be visualised with decision trees. With more complex and multistage decisions, these can get very big (see next lecture using slides). Example for a medical decision based on uncertain evidence: A test for an eye disease with prevalence p is conducted involving the distinction of n images. People without the disease perform better than those with the disease, but either group performs perfectly. The physician needs to decide whether to perform an inexpensive treatement now (d_1), or do nothing. The latter means potentially having to perform a more costly treatment later, if the patient did have/was developing the disease. Some evidence comes from how many images r the patient recognised correctly in the test. This can be phrased as a decision problem and we can determine which is the optimal decision as a function r, given fixed parameters n and p. We set up everything, but didn't entirely finish the calculation - try this yourself and see below for the solution.

Friday noon preview: Complex multilevel decision tree example (Oil drilling). Value of information. We look at an decision about drilling for oil in either field A or field B. The decision is made more complex by the additional options to conduct test drills in A or B which give some evidence but not certainty about the presence of oil in these locations. All relevant probability estimates are given, but it needs Bayes rule to derive all probabilities need to apply the EMV strategy to derive the optimal decision. An interesting concept in this context is the value of information. In fact, there are two alternative definitions. One allows for imperfect information that is typically practically available. The other one is for perfect information that may not be available, but it may still be of interest to calculate its values; for example to serve as an upper bound for the value of any even hypothetical sources of information.

Friday 4pm: Pros and cons of EMV strategies and alternatives. Example Farmer. The EMV strategy is a transparent and systematics way of arriving at an optimal decision for quantitative outcomes such as money (gains or losses), time, or anything where quantitative measures are available (education outcome, health status etc). However, it requires that subjective probabilities are available. An alternative are, for example, maximin and maximax strategies that represent an optimist's and a pessimist's approach to decisions making and are very intuitive. A disadvantage is that they are driven by extremes. Another disadvantage of the EMV approach is that it does not take into account the subjective value of money, which has been proven wrong in empirical studies with people in many situations - including in a survey conducted with ST222 students last year! This can be overcome by applying a utility function to the loss function. We introduce the certainty money equivalent and define utility as its inverse. This provides a concept of utility based on elicitation through bets.

Resources for week 4:

Study advice for week 4:

  • Draw some decision trees of your own
  • Complete the missing calculations in the eye disease example
  • Solve the problems on Sheet 2

Questions by students in week 4:

  • Nobody really got a chance to ask a question yet, but a good question to ask would be: "Is d* unique?". The answer is no, it is not unique. There could be two (or more) optimal decisions in the sense that they all lead to the same expected monetary value.
  • After the Friday noon lecture there were some comments on the suitability of EMV. They were spot on, as we are about to consider alternatives to EMV in the Friday 4pm class. A particular issue is the appropriateness of using expectations for decisions that are only taken once. It will depend on the situation. If this is about the one time decision that can lead to ruin or death, expected values can be of limited use for an individual or a small company. Thinks of Keynes quote "In the long run, we are all dead!" questioning the overuse of asumptotic approaches in economics. On the other hand, if this is just one of many decisions taken by a big company, there is a justification to apply expectations.
Week 3

Tuesday: Dutch book example, elicitation of probabilities, rational individuals are coherent. In horse betting example for Dutch books from last week it is important to note that bookie may have to close bets or keep adjusting odds (e.g. horse dropped, unequal sales). Elicitation of subjective probabilities through comparison with bets in familiar situations such as simple probabilistic experiments (e.g. spinner, balls in urn). An individual is called coherent if her subjective probability assignments obey the axioms of probability. A person is rational if she would not make any deal that may be disadvantageous to her, in particularly would not bring herself in a situation that a Dutch book could be constructed against her. This can be used to show that a rational agent’s subjective probabilities must be coherent. This is discussed in detail in the lecture notes Theorems 3.1 and 3.2. We show one of the two cases of the proof for Theorem 3.1, the others are very similar. The key idea is to convert the subjective probabilities into bets and then construct Dutch books from suitable bets.

Friday noon: Review and examples conditional probability, random variables, expected value. Review of definition of conditional probability, non-symmetry, relationship with independence, law of total probability, Bayes theorem, Bayes rule. Called off bet b(A|B) is the best that only happens if B occurs. Its value is the same as the value of a simple bet b(C) on an event C with probability P(A|B). Example: Screening test for a condition (e.g. disease, drug use). Information about the quality of a tests is provided as the probability that the test shows a positive results given the subject has the condition and the probability that the test shows a negative ressults given the subject does not have the condition. From the point of view of a subject who was tested positive is relevant to find out what is the probability that this is actually correct. Bayes rule is used to calculate this. With imperfect tests and low prevalence of the condition (e.g. popultation wide rare disease testing without symptoms, drug test in people without specific evidence or history of use), the probability that given a positive test the subject really has the condition can be suprisingly low (in our example is was lower than 10%).

Friday 4pm: Expectation of random variables, prediction, loss function, decision making. Expectation of a discrete random variable as sum. If (countably) infinite the sum may or may not converge. (For continuous random variables they become integrals.) What is the best predictor for the value of a random variable? It depends on how you measure “best”. We introduce three alternative loss functions measuring deviations from the prediction and define the best predictor as the value that minimises the expected value of the loss function L applied to the random variable X. For show that for a simplistic L distinguishing only right and wrong, the best predictor b is the mode of X (follows from definition of the mode). For L measuring the absolute deviation: homework (solution). For L measuring the squared error, b is the expectation of X (proof using calculus). Loss functions also have a central role in decision theory. Model for decision making includes decision options, outcomes, subjective probabilities and a loss function. We model the task of decision making as finding the decision that minimises the expectation of a loss function on the product space of decision space and outcome space. Burglary insurance example.

Resources for week 3:

Study advice for week 3:

  • Study the extra examples about Dutch books in the lecture notes not covered in class.
  • Try to proof Theorems 3.1 and 3.2 yourself.
  • Construct decision models for examples in your own life.
  • Solve problems from exercise sheet 2.

Questions by students in week 3:

There were not really questions suitable to discuss here, but many thanks for comments that helped resolving typos on the board, and also for questions to clarify things - it’s very good for me to know whenever something needs to be explained better.

Also, I have some question which are not examinable at all, but may be good for passing your time while waiting for the U1 (or, maybe, for conversation with your housemates or at a party):

  • Can any rational person pretend to be irrational?
  • Can any irrational person pretend to be rational?