Indexing: The information collected because of the crawlers is arranged and stored in a large database referred to as an index. These are utilized by search engines to find not long ago up-to-date materials as well as to index it making sure that users can find it when they do a search. Crawling