summaryrefslogtreecommitdiff
path: root/crawler.js
AgeCommit message (Expand)Author
2018-11-10Improved performanceMarvin Borner
2018-11-09Fixed endless database existing checkMarvin Borner
2018-11-08Improved rate limit handling (probably need several proxies)Marvin Borner
2018-11-08Added indexing and improved crawling algorithmMarvin Borner
2018-11-07Implemented web crawlingMarvin Borner