An Adaptive Synchronization Policy for Harvesting OAI-PMH Repositories

11 years 12 months ago
An Adaptive Synchronization Policy for Harvesting OAI-PMH Repositories
Metadata harvesting requires timely propagation of up-to-date information from thousands of Repositories over a wide area network. It is desirable to keep the data as fresh as possible while observing the overhead on the Harvester. An important dimension to be considered is that Repositories vary widely in their update patterns; they may experience different update rates at different times or unexpected changes to update patterns. In this paper, we define data Freshness metrics and propose an adaptive algorithm for the synchronization of the Harvester with the Repositories. The algorithm is based on meeting a desired level of Freshness while incurring the minimum overhead on the Harvester. We present a comparison between different policies for the synchronization within the framework devised. It is shown that the proposed policy outperform the other policies, especially for heterogeneous update patterns.
Noha Adly
Added 16 Aug 2010
Updated 16 Aug 2010
Type Conference
Year 2009
Authors Noha Adly
Comments (0)