Tuesday, April 26, 2011

Madrid is a very fine city, but it ain't no Rivendell

Despite the best efforts of the barrowights, also known as Air France, your two adventurers and their Great Burdens arrived in Madrid.



Ando argued mightily with the Wights that they leave us in Rivendell, aka Paris, but it was for naught. $1800 must be abandoned, p.p., for the privilege of getting off the plane early in Paris. That they left one of our bags to marinate overnight and closed the boarding of our plane 15 minutes before scheduled in Rivendell, I can only theorize, was to punish us for the asking.

But yes, Madrid. Quite nice.





Lots of parks, an absurdly extensive subway, and strangely desolate, due to Semana Santa (note foreshadowing)



We did some more investigating and found that Alsa would help carry our Great Burdens by auto-coach towards France, via way of Girona. They were intact, mostly. So we built them, mostly. Then we pushed them, cajoled them up stairs, held them while speaking in soothing tones on train and subway (they spook easily in crowds), did delicate ballet to corral them onto elevators. Then dumped them in the luggage hold of a bus.

Friday, April 22, 2011

Chapter One: In Which our Heroes Leave the Shire and Discover Their Great Burden

It was several weeks past the equinox in the shire, in what had been an unusually cool spring. The daffodils and tulips were finally, cautiously poking their heads out of the cold ground.

The Shire in Bloom
From bike1


From bike1


The mid April sun shone obliquely, but did little to warm Ando McDaggins, who was on his way to his post at the Cancer Prevention Mill. Ando had just crossed the bridge on his way to South Bywater when he came upon his dear friend Erwise.

“Good day, Erwise Gringee. I must admit I am surprised to see you this far from Hobbitown. Are not your services required in the Hobbitown Public Health Clinic?”

“Nay, Ando. I threw up my deuces and walked out this morning.”

“What brought on this change of heart, dearest Erwise?”

“I had a dream last night that a great journey was in my future, and I figured that I needed to be unencumbered by a job to take on this task."

"A dream? You quit your job because of a dream? I have always known you to be a level-headed, scientific type. I am just surprised you did such a thing.” Ando spoke, while scratching his scraggly bearded chin.

“You speak true. I usually do not heed such dreams, but I lately have been fed up with spending my days doing Hobbit footcare.”

“Tell me about this dream of yours. I took a class from Sir Gilbran on dream interpretation several years ago, and maybe I could help with interpretation.”

“Well, there was a large Elvin apparition that appeared to me and showed me two large boxes hidden in the bushes, and called this my 'great burden'. She told me I was to pick a partner for the adventure. And that was all. Except she was dressed strangely for an elf, in head-to-toe spandex and with a long, bright yellow cape. She also kept complaining about how much her taint hurt.”

“Oh, yes, I can definitely help interpret this type of dream,” replied Ando. “Just toss three sticks under one side of the bridge. When they appear on my side, I will use the pattern to decipher the meaning of this dream of yours."

Erwise quickly gathered three sticks and tossed them in the river.

"A-hah!" cried Ando. "Just what I thought. You are to take a bike journey to the hills of France, and your travel partner will be I, Ando."

Erwise looked down at the sticks - one stuck in the curve of the river, the other two seeming not to make a symbol at all. Sir Gilbran was known to teach bogus classes for extra cash to support his herbalism habit. This would not make for as good a story as her friend Frodo was able to brag about around the town bonfires. However, searching for a real answer to this obtuse dream seemed tedious, and the answer might be something that involved Dark Riders, which didn't sound tempting in the least. Giddy with the thought of free time and adventure with a dear friend, she accepted.

Ando tossed his cancer prevention bag into the Water, and they turned 90 degrees. They set out west, towards the Brandywine, the ends of the shire, and the land of the Euros.

Thursday, March 31, 2011

Empirical confidence intervals on scatter plots

Update: see this instead for a better solution.




Here's a little R-trick for you nerds out there in cyberland, but mainly for me, so I won't have to rediscover this little bit of code.

>xyplot(results3gr[,1]~results3gr[,2], xlim=c(0, 5000), cex=.1, ylim=c(-200, 200))



So you've got a dense scatter plot. Damn, there's points EVERYWHERE on this stupid thing, but how many more are being overplotted along the x axis versus some other blue-colored spot?

Well, how about using a 2D-kernel density estimator to get the relative density of points across a regular grid? Then you could find the ordinates correspond to 95% of the mass at a given slice of the grid.

So, the package KernSmooth should do the trick:

> ks = bkde2D(results3gr, bandwidth=c(10, 30), range.x=list(c(-200, 200), c(0, 1e4)))

Then we need to find the marginal mass at each abscissa of the grid, and the cumulative mass:

> ksmarg = apply(ks$fhat, 2, sum)
> kscum = apply(ks$fhat, 2, cumsum)

Then divide the cumulative mass by the marginal mass, and get a list of linear approximations


> kscond = as.data.frame((t(t(kscum)/ksmarg)))
> kscdf = list()
> for (i in 1:ncol(kscond)){kscdf[[i]] = approxfun(kscond[[i]], ks$x1)}

to get a cumulative distribution function for each abscissa of the grid. Now we get a list of 1% and 99% ordinates for each value in the abscissa:

> confbars = matrix(nrow=2, ncol = ncol(kscond))
> for ( i in 1:ncol(kscond)){confbars[1, i] = kscdf[[i]](.025); confbars[2, i] = kscdf[[i]](.975)}


Now do some lattice witchcraft and plot:

> xyplot(results3gr[,1]~results3gr[,2], panel=function(x, y, ...){panel.xyplot(x, y, cex=.3); panel.xyplot(ks$x2, confbars[1,], type="l", col="black"); panel.xyplot(ks$x2, confbars[2, ], type="l", col="black")}, xlim=c(0, 5e3), ylim=c(-200, 200))



The bars appear to be over-conservative, probably because of sparsity...but oh well. It'll have to do.


Hat tip to Yihui to for the reference to KernSmooth.