[D] CrawlObservers
[D] CrawlProfiles
[D] CrawlQueues
[D] Exceptions
[D] Handlers
[F] Crawler.php
[F] CrawlerRobots.php
[F] CrawlUrl.php
[F] LinkAdder.php
[F] ResponseWithCachedBody.php