The Brier Measure is not Strictly Proper (as Epistemologists have come to use that term)

In recent years, formal epistemologists have gotten interested in measures of the accuracy of a credence function. One famous measure of accuracy is the one suggested by Glenn Brier. Given a (finite) set $\Omega =$ { $\omega_1, \omega_2, \dots, \omega_N$ } of possible states of the world, the Brier measure of the accuracy of a credence function $c$ at the state $\omega_i$ is

$ \mathfrak{B}(c, \omega_i) = - (1-c(\{ \omega_i \}))^2 - \sum_{j \neq i} c(\{ \omega_j \})^2 $

And formal epistemologists usually say that a measure of accuracy $\mathfrak{A}$ is strictly proper iff every probability function expects itself (and only itself) to have the highest $\mathfrak{A}$-value.

Strict Propriety
A measure of accuracy $\mathfrak{A}$ is strictly proper iff, for every probability function $p$ and every credence function $c \neq p$, the $p$-expectation of $p$'s $\frak{A}$-accuracy is strictly greater than the $p$-expectation of $c$'s $\frak{A}$-accuracy. That is: for every probability $p$ and every credence $c \neq p$,

$ \sum_{i = 1}^N p(\{ \omega_i \}) \cdot \mathfrak{A}(p, \omega_i) \,\, > \,\, \sum_{i = 1}^N p(\{ \omega_i \}) \cdot \mathfrak{A}(c, \omega_i) $

(‘Weak propriety’ is the property you get when you swap out ‘$>$’ for ‘$\geq$‘.)

The point of today’s post is that, contrary to what I once thought (and perhaps contrary to what some others thought as well—though this could be a confusion localized to my own brain), the Brier score is not strictly proper.


All Posts by Category or Tags.