sharat7j / AlphaCrawler

Used open source crawler4j for web crawler functionality. Manipulated crawler4j api such that the priority set for crawling a url is based on a key value which is set to a predefined url priority+last modified time of the url+the doc id of the url.The lowest key value get crawled first.