Some time ago I wrote an article,SayIt is time to use the whitelist to the wallIt's,But that has expired long whitelist,It is not so smooth with the,Then I say boasted:I want to implement your own a reptile,Come crawling China's domain,Good update whitelist。
Ok,All in all this crawler is written on the line and then climb took more than ten thousand,But in the end I found the former to do a better solution,So the project was abandoned reptile。In short,White List more powerful,Just do not use it this reptile。
Crawler is written in Python,Not used by[……]