todo.md - marvinborner/JsonSearchEngine - Basic web crawler, usable for search engines or for a searching feature of a website

summary refs log tree commit diff

path: root/todo.md

blob: e68cf4e170554e6378db65dc4192f2ee41e756f5 (plain) (blame)

# TODOs
- [ ] Respect robots.txt
- [ ] Clean up code (eg. external algorithm functions)
- [ ] Photo crawling