Web object collection: here or there?

Ruth Kurniawati; Jesse S. Jin; John A. Shepherd

doi:10.1117/12.325835

5 October 1998 Web object collection: here or there?

Ruth Kurniawati, Jesse S. Jin, John A. Shepherd

Proceedings Volume 3527, Multimedia Storage and Archiving Systems III; (1998) https://doi.org/10.1117/12.325835
Event: Photonics East (ISAM, VVDC, IEMB), 1998, Boston, MA, United States

Abstract

Although the advantages to performing the collection of web objects at the site where the objects reside seem obvious, most of the popular search engines use their own centralized or coarsely distributed collectors (robots). Sending each object to the collector is almost always the worst option with respect to resource usage. An alternative is to distribute the collection process by sending the collector to the source site, which has the obvious advantage of distributing the significant computational load involved in cataloguing, as well as giving opportunities for summarizing and compression. In this paper, we propose a system for distributed object cataloguing over the world wide web via lightweight collector agents. This approach differs to previous approaches such as Harvest in using small Java- based collectors that can be very easily deployed on the site being indexed, thus allowing much finer grained distribution of the collection task.

Citation Download Citation

Ruth Kurniawati, Jesse S. Jin, and John A. Shepherd "Web object collection: here or there?", Proc. SPIE 3527, Multimedia Storage and Archiving Systems III, (5 October 1998); https://doi.org/10.1117/12.325835

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available