Check out R-bloggers for more excellent content!

What Can We Learn from the Commitments of Traders Report?


The Commitments of Traders (COT) report is issued weekly by the Commodity Futures Trading Commission (CFTC). It reflects the level of activity in the futures markets. The report, which is issued every Friday, contains the data from the previous Tuesday. Read more »

Race Statistics for Comrades Marathon Novice Runners: Corrigendum

2014-05-17     R running

There was some significant bias in the histogram from my previous post: the data from all years were lumped together. This is important because as of 2003 (when the Vic Clapham medal was introduced) the final cutoff for the Comrades Marathon was extended from 11:00 to 12:00. In 2000 they also applied an extended cutoff. Read more »

Race Statistics for Comrades Marathon Novice Runners

2014-05-16     R running

Most novice Comrades Marathon runners finish the race on their first attempt and the majority of them walk (shuffle, crawl?) away with Bronze medals. Read more »

Hazardous and Benign Space Objects: Orbits in the Solar-Ecliptic Reference Frame

2014-05-12     R

In two previous posts in this series I have wrangled NEO orbital data into R and then solved Kepler’s Equation to get the eccentric anomaly for each NEO. The final stage in the visualisation of the NEO orbits will be the transformation of locations from the respective orbital planes into a single reference frame. Read more »

Comrades Marathon Negative Splits: The Plot Thickens

2014-05-10     R running

I have been thinking a little more about those mysterious negative splits. Not too surprisingly, this thinking happened while I was out running along the Durban beachfront this morning. Read more »

Hazardous and Benign Space Objects: Solving Kepler's Equation

2014-05-08     R

Following on from my previous post about Near Earth Objects, today we are going to solve Kepler’s Equation to find the eccentric anomaly, which is the next step towards plotting the positions of these NEOs relative to Earth. Read more »

Comrades Marathon: Negative Splits and Cheating

2014-05-06     R running

With this year’s Comrades Marathon just less than a month away, I was reminded of a story from earlier in the year. Mark Dowdeswell, a statistician at Wits University, found evidence of cheating by some middle and back of the pack Comrades runners. He identified a group of 20 athletes who had suspicious negative splits: they ran much faster in the second half of the race. There was one runner in particular whose splits were just too good to be true. When the story was publicised, this particular runner claimed that it was a conspiracy. Read more »

Hazardous and Benign Space Objects: Getting the Data

2014-04-28     R

The recent story about a skydiver nearly being hit by falling meteor got me thinking about all the pieces of rock floating around in near-Earth space. Despite the fact that the supposed meteor was probably just a chunk of rock mistakenly packed in with a parachute, the fact that something like that could actually happen is quite intriguing. And not a little frightening. Read more »

R Interface to Myfxbook

2014-04-17     R

Myfxbook provides an interface to your FOREX trading accounts as well as an active trading community. It has a broad range of functionality including Read more »

Earthquakes: Land / Ocean Distribution

2014-04-13     R

Largest Volcanoes in Recorded History (and other statistics)

2014-04-11     R

Around 199 years ago the largest volcano in recorded history, Mount Tambora, erupted, spewing an enormous volume of molten rock and ash into the atmosphere and onto the surrounding land. Read more »

Earthquakes: Magnitude / Depth Chart

2014-04-07     R

I am working on a project related to secondary effects of earthquakes. To guide me in the analysis I need a chart showing the location, magnitude and depth of recent earthquakes. There are a host of such charts available already, but since I had the required data on hand, it seemed like a good idea to take a stab at it myself. Read more »

Daylight Saving Effect on Financial Indices

2014-04-01     R

Does the transition to and from Daylight Saving Time (DST) have a (significant) effect on the stock market? Read more »

Filtering Data with L1 Regularisation

2014-03-27     R

A few days ago I posted about Filtering Data with L2 Regularisation. Today I am going to explore the other filtering technique described in the paper by Tung-Lam Dao. This is similar to the filter discussed in my previous post, but uses a slightly different objective function: Read more »

Filtering Data with L2 Regularisation

2014-03-25     R

I have just finished reading Momentum Strategies with L1 Filter by Tung-Lam Dao. The smoothing results presented in this paper are interesting and I thought it would be cool to implement the L1 and L2 filtering schemes in R. We’ll start with the L2 scheme here because it has an exact solution and I will follow up with the L1 scheme later on. Read more »

Real Time Lightning Data


A map of real time lightning data from the World Wide Lightning Location Network (WWLLN) is now available here. The map is best viewed in Google Chrome. Try selecting the Cloud Overlay and Stroke Density options. Read more »

How Much Time to Conceive?

2014-01-12     R

This morning my wife presented me with a rather interesting statistic: a healthy couple has a 25% chance of conception every month [1], and that this should result in a 75% to 85% chance of conception after a year. This sounded rather interesting and it occurred to me that it really can’t be that simple. There are surely a lot of variables which influence this probability. Certainly age should be a factor and, after a short search, I found some more age-specific information which indicated that for a woman in her thirties, the probability is only around 15% [2,3]. Read more »

Processing EXIF Data

2013-12-16     R

I got quite inspired by the EXIF with R post on the Timely Portfolio blog and decided to do a similar analysis with my photographic database. Read more »

Updated Comrades Winners Chart

2013-12-14     running

On Friday I received my copy of The Official Results Brochure for the 2013 Comrades Marathon. Always makes for a diverting half an hour’s reading. And the tables at the front provide some very interesting statistics. Seemed like a good opportunity to update my Chart of Comrades Winners. Read more »

Contour and Density Layers with ggmap

2013-12-14     R

Amy Cuddy: Your body language shapes who you are...

2013-11-27     TED Talk

… but be careful with your plots: they might be misinterpreted. Amy Cuddy gives a great talk. Provided me with lots to think about and I will happily confess that I have struck a few power poses (but only after ensuring that I am quite alone)! Read more »

Deriving a Priority Queue from a Plain Vanilla Queue

2013-11-26     R

Following up on my recent post about implementing a queue as a reference class, I am going to derive a Priority Queue class. Read more »

Implementing a Queue as a Reference Class

2013-11-24     R

I am working on a simulation for an Automatic Repeat-reQuest (ARQ) algorithm. After trying various options, I concluded that I would need an implementation of a queue to make this problem tractable. R does not have a native queue data structure, so this seemed like a good opportunity to implement one and learn something about Reference Classes in the process. The Implementation We use setRefClass() to create a generator function which will create objects of the Queue class. Read more »

Lightning Activity Predictions For Single Buoy Moorings

2013-11-24     talk: standard

A short talk that I gave at the LIGHTS 2013 Conference (Johannesburg, 12 September 2013). The slides are a little short on text because I like the audience to hear the content rather than read it. The objective with this project was to develop a model which would predict the occurrence of lightning in the vicinity of a Single Buoy Mooring (SBM). Analysis and visualisations were all done in R. I used data from the World Wide Lightning Location Network (WWLLN) and considered four possible models: Neural Network, Conditional Inference Tree, Support Vector Machine and Random Forest. Of the four, Random Forests produced the best performance. The preliminary results from the Random Forests model are very promising: there is good agreement between predicted and observed lightning occurrence in the vicinity of the SBM. Read more »

Iterators in R

2013-11-14     R

According to Wikipedia, an iterator is “an object that enables a programmer to traverse a container”. A collection of items (stashed in a container) can be thought of as being “iterable” if there is a logical progression from one element to the next (so a list is iterable, while a set is not). An iterator is then an object for moving through the container, one item at a time. Read more »

Introduction to Fractals

2013-11-04     R

A short while ago I was contracted to write a short piece entitled “Introduction to Fractals”. The article can be found here. Admittedly it is hard to do justice to the topic in less than 1000 words. Both of the illustrations were created with R. Read more »

Percolation Threshold: Including Next-Nearest Neighbours

2013-11-01     R

In my previous post about estimating the Percolation Threshold on a square lattice, I only considered flow from a given cell to its four nearest neighbours. It is a relatively simple matter to extend the recursive flow algorithm to include other configurations as well. Malarz and Galam (2005) considered the problem of percolation on a square lattice for various ranges of neighbor links. Below is their illustration of (a) nearest neighbour “NN” and (b) next-nearest neighbour “NNN” links. Read more »

Percolation Threshold on a Square Lattice

2013-10-30     R

Manfred Schroeder touches on the topic of percolation a number of times in his encyclopaedic book on fractals (Schroeder, M. (1991) Fractals, Chaos, Power Laws: Minutes from an Infinite Paradise. Percolation has numerous practical applications, the most interesting of which (from my perspective) is the flow of hot water through ground coffee! The problem of percolation can be posed as follows: suppose that a liquid is poured onto a solid block of some substance. If the substance is porous then it is possible for the liquid to seep through the pores and make it all the way to the bottom of the block. Whether or not this happens is determined by the connectivity of the pores within the substance. If it is extremely porous then it is very likely that there will be an open path of pores connecting the top to the bottom and the liquid will flow freely. If, on the other hand, the porosity is low then such a path may not exist. Evidently there is a critical porosity threshold which divides these two regimes. Read more »

Eric Berlow and Sean Gourley: Mapping Ideas Worth Spreading

2013-10-25     TED Talk

Plotting Times of Discrete Events

2013-10-19     R

I recently enjoyed reading O’Hara, R. B., & Kotze, D. J. (2010). Do not log-transform count data. Methods in Ecology and Evolution, 1(2), 118–122. doi:10.1111/j.2041-210X.2010.00021.x. Read more »