https://docs.crawlab.cn/zh/Spider/ConfigurableSpider.html
https://github.com/gocolly/colly https://github.com/crawlab-team/crawlab
https://github.com/JayBizzle/Crawler-Detect/blob/master/tests/crawlers.txt