PARAFLOW: A Dataflow Distributed Data-Computing System

We describe the Paraflow system for connecting heterogeneous computing services together into a flexible and efficient data-mining metacomputer. There are three levels of parallelism: a dataflow paradigm provides functional parallelism, connecting services by channels; each service may be a SPMD parallel program; and each node of the SPMD service may be multithreaded. Paraflow is easily integrated with web-computing, using MIME types, HTTP forms, a Java GUI for setting up the network of services, and a simple API to allow web servers to be utilized in the computation. Emphasis is on providing a bridge from straightforward, public, unauthenticated browsing of data to collaborative, highperformance data-computing. Modern high-performance computing and communication is capable of providing enormous processing speeds and high-speed data transfer between processing engines, while RAID disk and tape robots can store the vast quantities of data that drive these systems. But effectively using...

• Type: Conference
• Year: 1998
• Where: HPCN
• Authors: Roy Williams, Bruce Sears
• Keywords: Heterogeneous Computing Services, HPCN 1998, HPCN 1999, SPMD Parallel Program, Web Servers