extreme value statistics

[phantom power]

I've been thinking about extreme value statistics recently.

If you repeatedly sample from a random variable, the (expected) time between achieving successive maximums grows exponentially. For example, here's some results I got from repeatedly sampling from a Normal distribution of standard deviation (1). Each number is the average number of trials to beat the previous maximum value:

{9.009, 61.989, 365.096, 1560.31, 4906.24}

If you plot these, you get a nice exponential curve. This is true for pretty much any distribution that's applied out in the "real world."

Why am I posting this in E/E? Consider climate change measurements. In the null-hypothesis that climate change isn't happening, we would expect weather "records" to be broken less and less frequently over time. In fact, the time between records would grow exponentially.

But that isn't what we're seeing. Weather records are broken on a fairly regular basis. The time between record-breakers appears to be constant, or maybe even accelerating.

And then there is the nature of the records themselves. In the "constant-climate" scenario (null-hypothesis), each time a record is broken, it's expected difference is less: the record is broken by a more narrow margin each time. Here are the differences between each maximum and the previous maximum:

{0.902897, 0.597499, 0.490741, 0.403293, 0.353109}

As we've seen in the last year, old weather records aren't just being broken, they are being smashed. 27 tropical storms. 143 days without rain.

I think that there are other statistical methods that are better for approaching these questions scientifically, but I thought it was interesting, since as humans we tend to be tuned in to extraordinary events, as opposed to less flashy statistical analysis.

[Dead_Parrot]

1. The problem with extraordinary events

is that if they keep happening, they are no longer extraordinary. The news that 2005 was the warmest year on record passed with barely a ripple, even here.

BTW, the easiest way to tackle exponetial curves in stats is to work with the log of the values, which will give you a nice flat line: you can then re-exponetialise when you've finished. :)

[phantom power]

2. Times between maximums are themselves random variables...

which may be log-normal. They clearly displayed an enormous variance, measuring in across several orders of magnitude. Doing it all in log-space might tame their behavior.

[Dead_Parrot]

3. With temp. data...

...you could even run an analysis of time intervals between anomalies in the log values, which I suspect would give another exp curve. I might poke around with this later if I get time: It would be fun¹ to get a "This is how much faster we are fucking ourselves each day" curve.

(¹In a "manic laughter" sort of way)

[bemildred]

4. A comment:

Weather is a chaotic system, hence in principle unpredictable over time, hence the assumption that there is a unitary population that can be sampled for its attributes seems itself questionable, particularly in the sense that if offers any predictive capacity. This is not to say that modern weather prediction systems are ineffective, but that their effective range is quite small, and that the regular smashing of records may be less anomalous than one might first think.

[hatrack]

5. Is weather truly a chaotic system?

There are regularly recurring patterns throughout the world system, whether it's the propensity towards snow in New England in the winter, towards cyclonic storms in the Western Pacific in late summer or towards tornadoes in the continental US in the spring.

I would suggest that there's a bit too much predictability involved to describe weather as "chaotic" and leave it at that.

:toast:

[phantom power]

7. It is truly chaotic, in the physics/math sense.

An arbitrarily small tweak to the system will cause an exponential deviation from a reference path, as the system runs forward in time. Chaotic systems can exhibit patterned behavior, but the details of the way those patterns play out is deeply unpredictable. There is no closed-form solution to the system state. The fastest way to compute the state at time (t) is to actually run the system. Computationally irreducible.

[bemildred]

8. Well, the question would be "How chaotic is it?"

It's not a either/or type of thing, it certainly is, that's where the original Lorenz strange attractor came from, studying weather:

The most famous strange attractor is undoubtedly the Lorenz attractor -- a three dimensional object whose body plan resembles a butterfly or a mask. The Lorenz attractor, named for its discoverer Edward N. Lorenz, arose from a mathematical model of the atmosphere.

Imagine a rectangular slice of air heated from below and cooled from above by edges kept at constant temperatures. This is our atmosphere in its simplest description. The bottom is heated by the earth and the top is cooled by the void of outer space. Within this slice, warm air rises and cool air sinks. In the model as in the atmosphere, convection cells develop, transferring heat from bottom to top.

http://hypertextbook.com/chaos/21.shtml

But thing is, as you say, weather is predictable within certain limits; however in the longer time range that seems doubtful, and in fact planetary climate history does show abrupt changes, at any scale that one cares to examine, so it seems to fair to at least question whether the assumptions that underly statistical analysis are met, and if so in what measure.

[phantom power]

6. I agree, the entire notion of a static climate is fictitious.

I just thought it was an interesting avenue for rejecting the "no-change" hypothesis. Of course, that hypothesis has been thoroughly rejected already (and with more rigor), for anybody paying attention.

It's interesting to speculate that if we had been taking records during a period of relative stability, we might have seen behavior more consistent with a static distribution. Weather records increasingly rare. I guess we'll never know.

[bemildred]

9. For that purpose, I expect you are right.

For people that assume stasis, the statistical argument, whether in fact valid or not, would carry weight, because it would still refute their false assumption.

And, in your second paragraph, you are also right that in one of the long equilibrium periods, the statistical argument for the assertion that change is starting to occur would in fact carry more weight. With recent climate history all one could say is that it still seems to be "variable".

[Dead_Parrot]

10. It depends on your timescale

If you pick a short enough sample not to be affected by long-term cycles, (like glaciation) and long enough to soak up the solar cycle, the temperature does seem to follow a fairly restricted distribution: Oddities like 535/536AD will stick out like a sore thumb.

The main problem with these data is that they are already subject to possible errors due to being reconstructed (from dendrochronology, ice cores, whatever) so it's difficult to draw any solid conclusions from them.

We do have some directly measured temps from the "hockey stick" handle, but not really enough to get a solid picture: Another thousand years of study would be in order...