A novel mobile crawler system based on filtering off non-modified pages for reducing load on the network
Joint Authors
Source
The International Arab Journal of Information Technology
Issue
Vol. 8, Issue 3 (31 Jul. 2011), pp.272-279, 8 p.
Publisher
Publication Date
2011-07-31
Country of Publication
Jordan
No. of Pages
8
Main Subjects
Information Technology and Computer Science
Topics
Abstract EN
The studies in the literature show that about 40% of the current Internet traffic and bandwidth consumption is due to web crawlers that retrieve pages for indexing by the different search engines.
This traffic and bandwidth consumption will increase in future due to the exponential growth of the web.
This paper addresses the problem of bandwidth consumption by introducing an efficient indexing system based on mobile crawlers.
The proposed system employs mobile agents to crawl the pages.
These mobile agent based crawlers retrieve the pages, process them, compare their data to filter out pages that are not modified after last crawl, and then compress them before sending them to the search engine for indexing.
The experimental results of the proposed system are very encouraging.
American Psychological Association (APA)
Nath, Rajender& Bal, Satinder. 2011. A novel mobile crawler system based on filtering off non-modified pages for reducing load on the network. The International Arab Journal of Information Technology،Vol. 8, no. 3, pp.272-279.
https://search.emarefa.net/detail/BIM-265412
Modern Language Association (MLA)
Nath, Rajender& Bal, Satinder. A novel mobile crawler system based on filtering off non-modified pages for reducing load on the network. The International Arab Journal of Information Technology Vol. 8, no. 3 (Jul. 2011), pp.272-279.
https://search.emarefa.net/detail/BIM-265412
American Medical Association (AMA)
Nath, Rajender& Bal, Satinder. A novel mobile crawler system based on filtering off non-modified pages for reducing load on the network. The International Arab Journal of Information Technology. 2011. Vol. 8, no. 3, pp.272-279.
https://search.emarefa.net/detail/BIM-265412
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references : p. 278-279
Record ID
BIM-265412