Can a probability measure adequately distinguish between the impossible and improbable?

To represent an impossible event, you exclude it from the event space. If an event is within the event space, with probability zero, that event will almost never happen.

See the concepts of "almost surely" and "almost never".

In probability theory, you start by defining a space of possible outcomes (the sample space), then you define a sigma algebra and probability measure over that sample space.

As I understand it, "impossible" means "not an element of the sample space." For example, if the sample space is the set of possible milliseconds that it takes me to run a mile, I will probably consider negative values to be not in the sample space. Thus my model takes negative running times to be "impossible". In contrast, a probability of zero for an event (set of outcomes in the sample space) does not mean "impossible". For example, given a normal distribution for a real-valued variable, any single possible value has a probability zero.

Of course, there's always some judgement involved in how you define the statistical model. In the running example, I could also define values less than, say, one minute to be "impossible" (not in the sample space) for my mile run. I could also define negative values to be in the sample space, but assign zero probability to any measurable subset comprised entirely of negative values. These distinctions probably ultimately don't materially effect the predictions of the statistical model.

In other words, probability values are not what distinguishes "impossible" from "zero probability"

Addendum (to capture some of the issues raised in the comments):

Any statistical model involves an initial judgement about what is "possible". If I model my mile run time as a real number, I am making a judgement that the outcome cannot, for example, be a non-real hyperreal number (even though we have no reason to know with absolute certainty that time isn't best modeled as a hyperreal number). Such an initial judgement of some things as impossible is completely unavoidable, even for Bayesian analysis. Whatever mathematical structure I choose to represent the set of possible outcomes, I am necessarily excluding as "impossible" outcomes that would be elements of some larger mathematical structure.

The question of zero probability for a "possible" outcome is related to the matter of countable vs. uncountable infinity. Not sure if you have the mathematical background that covers this, but: If the sample space is "discrete" (like the set of integers), we can meaningfully assign nonzero probability to every outcome because the sample space is a "countable" infinity. If the sample space is something like the set of real numbers (or even a finite interval of the real line), then the set of possible outcomes is an "uncountable" infinity and it is impossible to assign nonzero probability to every number in the set without the sum of probabilities being infinite. That's why we talk about probabilities for a real-valued random variable in terms of, for example, the probability that the outcome lies within a certain interval.

In the case of a probability model where the outcome is an n-tuple of real numbers, for example, I suspect the Bayesian approach would be something like the following: The initial probability distribution is one which assigns a nonzero probability to every subset of n-dimensional space which has nonzero Borel measure (see bullet point below). But this does not mean that every particular outcome, like say X=π in the one-dimensional case, has non-zero probability. Since we can never measure a real-valued variable with infinite precision, this is presumably sufficient for the Bayesian approach.

• Borel-measureable subsets of n-dimensional space are, roughly speaking, those which meet a certain Borel notion of "niceness" such that we can meaningfully ascribe a "measure" to them. For a single real variable "measure" means "length", if n=2 (pair of varibles) it's "area", generally for any n it's n-dimensional "volume."

You asked

In Bayesian thinking, no proposition is given a 0 probability given the possibility that one could be wrong. But if one uses the same measure to identify the probability of both an improbable and an impossible event, how can one distinguish between these events purely through probability?

Given the assumption that you might be in error, if you want complete correctness you simply do not attempt to declare things impossible. You assign them your best available estimate of their probability, including your confidence in the estimate as part of that number.

In many cases, the resulting probability will be close enough to zero to be discounted for practical purposes. Quantum teleportation of macroscopic objects or similar "miracles" are theoretically possible, but nobody wastes effort trying to anticipate something that unlikely. The odds of this affecting the result are so much lower than the odds that we make a measurement or computation error in trying to predict or measure the events that it makes more sense to simply reserve the right to say "oops, we missed something".

The mistake is in thinking that we can make a perfect estimate of the probabilities of real world events at all. We can't, and nobody who is dealing with reality pretends we can.

We can say what the exact odds would be if our model was perfectly accurate, but the map is not the territory. The best we can do is put confidence intervals around everything and carry those through the calculations.

Outside the domain of idealized philosophy, a difference that makes no difference is no difference. Errors in estimating how many angels can dance on the head of a pin, and what error that introduces in measuring the pin's weight, are simply not anything a sane person spends much time on.

If you are perfectly confident of your model and data, and absolutely certain that something is completely impossible, that is a mistake. And I'm perfectly confident of that.

TL;DR: You're starting with a version of Bayesianism so extreme that it has become a caricature. Don't do that.

It depends on what you mean by adequately. I find it entirely adequate to assign a probability of zero to impossible events, even though practically, I might also assign a probability of zero to possible but extremely improbable events. My running a mile in two minutes is impossible. My winning the lottery ten times in a row is possible but hugely improbable. In principle I can maintain the distinction between an impossible and highly improbable outcome by assigning a zero probability to the former and a small but non-zero probability to the latter. However, there comes a point at which the distinction is not worth maintaining for any practical reason.

In Bayesian thinking, no proposition is given a 0 probability given the possibility that one could be wrong. But if one uses the same measure to identify the probability of both an improbable and an impossible event, how can one distinguish between these events purely through probability?

I think you are referring here to 'priors', that is, your assumption of the likelihood of an event prior to Bayesian inference. You are free to choose 0 as your prior. If you choose that option, your inference is complete. There's no need to do any more calculations as they will always result in 0. Another way to look at it is that if you are sure something is impossible, there's no point in calculating the probability of that event. You already decided it was 0. Assigning a very small probability to an event is simply saying that you are aren't completely sure it's impossible.

I've noticed that people tend to get a little wrapped around the axel of priors in Bayesian inference. They aren't really that meaningful. That is, they don't determine the outcome of the inference. Bayesian inference should still march inexorably to a more correct result on each pass even if your prior is wildly wrong. The whole process is built around the assumption the prior is not (necessarily) correct.

What the prior does is affect how many iterations are required to reach a useful answer. Perhaps an analogy will help: if you are looking up a word in a large physical dictionary, if the word starts with 'M' you would likely open the dictionary up to some page near the middle of the book. It's not that you know that the word is on that page, you just guess it might be close. Then you look at the page and if the words on that page come before your word, you would guess again but ignoring the half of of the book that comes before your word. This process continues with finer and finer precision until you find the word.

Now, if your initial guess was way off and you opened it to the first page, the same process will still result in finding the correct page eventually. You will just be likely to need more steps to find it.

Sticking with the analogy, if you are sure the word is not in the dictionary (a prior possibility of 0), you wouldn't open it at all. If you are going to look at all for it, you are tacitly accepting that there's some possibility that it's in the book even if you don't expect to find it.

The way to indicate impossible events with probability would be to give it a probability of 0.

To do that with Bayesian reasoning, you'd have to explicitly assign it a value of 0 at some point. Most likely, this may happen upfront when one defines the possible set of outcomes (which excludes outcomes considered to be impossible).

Beyond that, with Bayesian reasoning, the probability of unseen events would tend towards 0, without ever reaching it. At any given point, such an event would have some probability ε. We can't say anything about where between 0 and ε the true probability lies (outside of gathering more data and getting closer to 0). Beyond decreasing as we get more data, the value of ε is also heavily influenced by the initial choice of probability for that event.

To put it more simply, Bayesian reasoning in itself can't tell you whether an unseen event is just highly unlikely or actually impossible. The actual probability it gives you for that event is meaningless in this regard (except in telling you that it's at the very least extremely unlikely). It might create a false sense that something actually impossible (or so extremely unlikely that it's functionally impossible) is actually just very unlikely.

Lotteries and resurrections

To determine the probability of winning the lottery 100000 times in a row, one could consider those to be independent events, at which point the probability is simply the probability of winning once to the power 100000. This would be a very, very, very small probability, but it's technically some precise number greater than 0.

Though if someone's winning many times in a row, that'd challenge our assumption of those being independent or about the probability of each lottery win, i.e. we might suppose it's rigged or that the results follow a predictable pattern.

Meanwhile, the probability of a resurrection* seems to just be tending towards 0. If one wants to say "random quantum fluctuations" can bring someone back to life, one would have to ask some physicists/biologists/whomever to estimate that. It'll probably also heavily depend on cause of death and time since death. From a layperson's perspective, based on all the people who've died and haven't resurrected, with zero verified accounts of resurrections, with it being contrary to our understanding of biology, and neuroscience in particular, I can only really put the probability at functionally 0.

* Some people have been revived after e.g. briefly having their heart stop, but this is compatible with our understanding of death being a process, where one's heart stopping is just one step. This is not what's typically meant when people speak of a "resurrection". Claims of a "resurrection" are also generally explicitly called a "miracle" that specifically defies our understanding of reality, which we can only really assign a probability of 0 until it meets the substantial burden of proof required to overturn how we understand reality to work. It would seem somewhat contradictory to say one can assign a meaningful probability to something which defies our understanding of reality, because we determine such probabilities based on our understanding of reality, which this defies.

Also, there are infinitely many conceivable things that defy our understanding of reality, so one can't coherently or mathematically assign some finite non-zero probability to all such things that are merely logically possible, no matter how small one makes that value. Though if one wants to say some of these are more likely than others, it's a different story, but it would still be an uphill battle to come with any meaningful (and justifiable) non-zero probability for any of those. Related answer.

If I may keep this as a purely formal and mathematical discussion, we have to remember is that a probability is by definition a ratio: the count of events we are interested in divided by the count of events that are possible: interesting-events/possible-events (E_i/E_p). For example, if we want to know the probability of rolling 4 in a pair of dice, we first note that there are three possible events that add up to 4 — 1,3; 2;2, 3;1 — and then we note that there are 6² (or 36) events we could possibly see (including those three events):

E_i/E_p = 4-events/all-events = 3/36 = 1/12 ≅ 0.0833 or 8.33%.

An improbable event is easy enough to understand here. 'Improbability' just means that E_i is small compared to E_p, or in other words, that there are very few events we are interested in within the whole range of events we might possibly see. That's always going to give us a concrete, east to interpret ratio.

An impossibile event, on the other hand, is confounding in math terms. 'Impossibility' means one of two things:

1. That out of all the events that might possibly occur, none of them occur: 0/E_p
2. That no event we're interested in occurs because no events could have occurred at all (E_i/0)

The first is confounding because either:

• it suggests that we listed out every possibly event that could have happened, and none of them did, which is either a methodological flaw or magic. Rolling 0 on a pair of dice can only be interpreted to happen when extraneous events intervene: we forget to roll the dice at all, or the dice shatter when they hit the table, or it's a magician's trick where he rolls the die and they turn into a pair of doves; it's a miracle where somehow the faces of two normal die add up to zero.
• it suggests that the probability of an impossible event is always the same, regardless of the possible number of events: the probability of rolling 0 in two dice is apparently the same as the probability of rolling 0 on a million dice, which seems statistically counter-intuitive.

The second is confounding because it's division by zero, and nobody likes that.

'Impossibility' does not fit into the formal mathematics we've created to define probability. We just can't use that term meaningfully.

Can a probability measure adequately distinguish between the impossible and improbable?

Probabiity measures are values between 0 and 1. Values of 0 and 1 are not probabilistic. They are deterministic and any assignment of 0 or 1 to an event should be provable. Otherwise it's a guess and can't be considered deterministic.

So the answer is no.