Outlier detection in performance data of parallel applications

— When an adaptive software component is employed to select the best-performing implementation for a communication operation at runtime, the correctness of the decision taken strongly depends on detecting and removing outliers in the data used for the comparison. This automatic decision is greatly complicated by the fact that the types and quantities of outliers depend on the network interconnect and the nodes assigned to the job by the batch scheduler. This paper evaluates four different statistical methods used for handling outliers, namely a standard interquartile range method, a heuristic derived from the trimmed mean value, cluster analysis and a method using tatistics. Using performance data from the Abstract Data and Communication Library (ADCL) we evaluate the correctness of the decisions made with each statistical approach over three fundamentally different network interconnects, namely a highly reliable InfiniBand network, a Gigabit Ethernet network having a larger varianc...

• Type: Conference
• Year: 2008
• Where: IPPS
• Authors: Katharina Benkert, Edgar Gabriel, Michael M. Resch
• Keywords: Distributed And Parallel Computing, Gigabit Ethernet Network, IPPS 2008, Network Interconnects, Outliers