Check out R-bloggers for more excellent content!

Plotting Flows with riverplot

2014-08-14     R

I have been looking for an intuitive way to plot flows or connections between states in a process. An obvious choice is a Sankey Plot, but I could not find a satisfactory implementation in R… until I read the riverplot post by January Weiner. His riverplot package does precisely what I am need. Read more »

Commitments of Traders: Moves in the Last Week

2014-08-09     R

In my previous post I gave some background information on the Commitments of Traders report along with a selection of summary plots. Read more »

Charting culture


This is well worth watching. Good idea. Great presentation. Effective use of the media to publicise a scientific paper. Read more »

Whistlers and Volcanic Lightning


A paper authored with my very talented student, Claire Antel, entitled “Investigating Dunedin whistlers using volcanic lightning” has just been published in Geophysical Research Letters. The paper looks at electromagnetic signals (“whistlers”) received at Dunedin, New Zealand, caused by lightning over volcanoes thousands of km away near the Aleutian Chain. Read more »

Comrades Marathon: A Race for Geriatrics?

2014-07-22     R Running

It has been suggested that the average Comrades Marathon runner is gradually getting older. As an “average runner” myself, I will not deny that I am personally getting older. But, what I really mean is that the average age of all runners taking part in this great event is gradually increasing. This is not just an idle hypothesis: it is supported by the data. If you’re interested in the technical details of the analysis, these are included at the end, otherwise read on for the results. Read more »

Where to Put EAs and Indicators in New MT4 Builds


If you are creating an EA or indicator from scratch, then the MetaTrader editor places the files in the correct location and the terminal is automatically able to find them. However, if the files originate from a third party then you will need to know where to insert them so that they show up in the terminal. For older builds of MetaTrader 4 the directory structure was fairly simple. Everything was to be found in a folder under Program Files. Its contents looked like this: Read more »

Comrades Marathon Negative Splits: Cheat Strikes Again

2014-07-16     Running

Twins, Tripods and Phantoms at the Comrades Marathon

2014-06-12     R running

Having picked up a viral infection days before this year’s Comrades Marathon, on 1 June I was left with time on my hands and somewhat desperate for any distraction. So I spent some time looking at my archive of Comrades data and considering some new questions. For example, what are the chances of two runners passing through halfway and the finish line at exactly the same time? How likely is it that three runners achieve the same feat? Read more »

Concatenating a list of data frames

2014-06-06     R

It’s something that I do surprisingly often: concatenating a list of data frames into a single (possibly quite enormous) data frame. Until now my naive solution worked pretty well. However, today I needed to deal with a list of over 6 million elements. The result was hours of page thrashing before my R session finally surrendered. I suppose I should be happy that my hard disk survived. Read more »

Comrades Marathon Pacing Chart: Down Run

2014-05-28     Excel running

Although I have been thinking vaguely about my Plan A goal of a Bill Rowan medal at the Comrades Marathon this year, I have not really put a rigorous pacing plan in place. I know from previous experience that I am likely to be quite a bit slower towards the end of the race. I also know that I am going to lose a few minutes at the start. So how fast does this mean I need to run in order to get from Pietermaritzburg to Durban in under 9 hours? Read more »

What Can We Learn from the Commitments of Traders Report?


The Commitments of Traders (COT) report is issued weekly by the Commodity Futures Trading Commission (CFTC). It reflects the level of activity in the futures markets. The report, which is issued every Friday, contains the data from the previous Tuesday. Read more »

Race Statistics for Comrades Marathon Novice Runners: Corrigendum

2014-05-17     R running

There was some significant bias in the histogram from my previous post: the data from all years were lumped together. This is important because as of 2003 (when the Vic Clapham medal was introduced) the final cutoff for the Comrades Marathon was extended from 11:00 to 12:00. In 2000 they also applied an extended cutoff. Read more »

Race Statistics for Comrades Marathon Novice Runners

2014-05-16     R running

Most novice Comrades Marathon runners finish the race on their first attempt and the majority of them walk (shuffle, crawl?) away with Bronze medals. Read more »

Hazardous and Benign Space Objects: Orbits in the Solar-Ecliptic Reference Frame

2014-05-12     R

In two previous posts in this series I have wrangled NEO orbital data into R and then solved Kepler’s Equation to get the eccentric anomaly for each NEO. The final stage in the visualisation of the NEO orbits will be the transformation of locations from the respective orbital planes into a single reference frame. Read more »

Comrades Marathon Negative Splits: The Plot Thickens

2014-05-10     R running

I have been thinking a little more about those mysterious negative splits. Not too surprisingly, this thinking happened while I was out running along the Durban beachfront this morning. Read more »

Hazardous and Benign Space Objects: Solving Kepler's Equation

2014-05-08     R

Following on from my previous post about Near Earth Objects, today we are going to solve Kepler’s Equation to find the eccentric anomaly, which is the next step towards plotting the positions of these NEOs relative to Earth. Read more »

Comrades Marathon: Negative Splits and Cheating

2014-05-06     R running

With this year’s Comrades Marathon just less than a month away, I was reminded of a story from earlier in the year. Mark Dowdeswell, a statistician at Wits University, found evidence of cheating by some middle and back of the pack Comrades runners. He identified a group of 20 athletes who had suspicious negative splits: they ran much faster in the second half of the race. There was one runner in particular whose splits were just too good to be true. When the story was publicised, this particular runner claimed that it was a conspiracy. Read more »

Hazardous and Benign Space Objects: Getting the Data

2014-04-28     R

The recent story about a skydiver nearly being hit by falling meteor got me thinking about all the pieces of rock floating around in near-Earth space. Despite the fact that the supposed meteor was probably just a chunk of rock mistakenly packed in with a parachute, the fact that something like that could actually happen is quite intriguing. And not a little frightening. Read more »

R Interface to Myfxbook

2014-04-17     R

Myfxbook provides an interface to your FOREX trading accounts as well as an active trading community. It has a broad range of functionality including Read more »

Earthquakes: Land / Ocean Distribution

2014-04-13     R

Largest Volcanoes in Recorded History (and other statistics)

2014-04-11     R

Around 199 years ago the largest volcano in recorded history, Mount Tambora, erupted, spewing an enormous volume of molten rock and ash into the atmosphere and onto the surrounding land. Read more »

Earthquakes: Magnitude / Depth Chart

2014-04-07     R

I am working on a project related to secondary effects of earthquakes. To guide me in the analysis I need a chart showing the location, magnitude and depth of recent earthquakes. There are a host of such charts available already, but since I had the required data on hand, it seemed like a good idea to take a stab at it myself. Read more »

Daylight Saving Effect on Financial Indices

2014-04-01     R

Does the transition to and from Daylight Saving Time (DST) have a (significant) effect on the stock market? Read more »

Filtering Data with L1 Regularisation

2014-03-27     R

A few days ago I posted about Filtering Data with L2 Regularisation. Today I am going to explore the other filtering technique described in the paper by Tung-Lam Dao. This is similar to the filter discussed in my previous post, but uses a slightly different objective function: Read more »

Filtering Data with L2 Regularisation

2014-03-25     R

I have just finished reading Momentum Strategies with L1 Filter by Tung-Lam Dao. The smoothing results presented in this paper are interesting and I thought it would be cool to implement the L1 and L2 filtering schemes in R. We’ll start with the L2 scheme here because it has an exact solution and I will follow up with the L1 scheme later on. Read more »

Real Time Lightning Data


A map of real time lightning data from the World Wide Lightning Location Network (WWLLN) is now available here. The map is best viewed in Google Chrome. Try selecting the Cloud Overlay and Stroke Density options. Read more »

How Much Time to Conceive?

2014-01-12     R

This morning my wife presented me with a rather interesting statistic: a healthy couple has a 25% chance of conception every month [1], and that this should result in a 75% to 85% chance of conception after a year. This sounded rather interesting and it occurred to me that it really can’t be that simple. There are surely a lot of variables which influence this probability. Certainly age should be a factor and, after a short search, I found some more age-specific information which indicated that for a woman in her thirties, the probability is only around 15% [2,3]. Read more »

Processing EXIF Data

2013-12-16     R

I got quite inspired by the EXIF with R post on the Timely Portfolio blog and decided to do a similar analysis with my photographic database. Read more »

Updated Comrades Winners Chart

2013-12-14     running

On Friday I received my copy of The Official Results Brochure for the 2013 Comrades Marathon. Always makes for a diverting half an hour’s reading. And the tables at the front provide some very interesting statistics. Seemed like a good opportunity to update my Chart of Comrades Winners. Read more »

Contour and Density Layers with ggmap

2013-12-14     R