Non-parametric detection of temporal order across pairwise measurements of time delays