Local & Global Experts

Contemporary epistemology is replete with principles of expert deference. Epistemologists have claimed that you should treat the chances, your future selves, your rational self, and your epistemic peers as experts. What this means is that you should try to align your credences with theirs. There are lots of ways you might try to align your credences with those of some expert function. (That expert function could be the chances, or it could be your future credences, or something else altogether.

The Brier Measure is not Strictly Proper (as Epistemologists have come to use that term)

In recent years, formal epistemologists have gotten interested in measures of the accuracy of a credence function. One famous measure of accuracy is the one suggested by Glenn Brier. Given a (finite) set $\Omega =$ { $\omega_1, \omega_2, \dots, \omega_N$ } of possible states of the world, the Brier measure of the accuracy of a credence function $c$ at the state $\omega_i$ is

$\mathfrak{B}(c, \omega_i) = - (1-c(\{ \omega_i \}))^2 - \sum_{j \neq i} c(\{ \omega_j \})^2$

And formal epistemologists usually say that a measure of accuracy $\mathfrak{A}$ is strictly proper iff every probability function expects itself (and only itself) to have the highest $\mathfrak{A}$-value.

Strict Propriety
A measure of accuracy $\mathfrak{A}$ is strictly proper iff, for every probability function $p$ and every credence function $c \neq p$, the $p$-expectation of $p$'s $\frak{A}$-accuracy is strictly greater than the $p$-expectation of $c$'s $\frak{A}$-accuracy. That is: for every probability $p$ and every credence $c \neq p$,

$\sum_{i = 1}^N p(\{ \omega_i \}) \cdot \mathfrak{A}(p, \omega_i) \,\, > \,\, \sum_{i = 1}^N p(\{ \omega_i \}) \cdot \mathfrak{A}(c, \omega_i)$

(‘Weak propriety’ is the property you get when you swap out ‘$>$’ for ‘$\geq$‘.)

The point of today’s post is that, contrary to what I once thought (and perhaps contrary to what some others thought as well—though this could be a confusion localized to my own brain), the Brier score is not strictly proper.