Python is a good thing，Many times we like to use it to write small things... For example, many services on my server are written in Python ( Supervisor will use your Python program becomes Services ），Accumulate，Many dependency packages are actually used。
but，These Python dependencies will not follow your [crayon-67[……]
Recently doing text statistics，Implemented in Python，Encountered an interesting problem-how to save statistical results。
Writing directly into memory is really impossible，Memory exhausted after ten hours，The program was forced to close。If you write directly to the database，Each write is too slow，It's been more than ten hours，If you go on like this, you will have to count on the week，Not a solution。
At last，I thought of a solution that does both[……]
Before that，I have been so install and use pip on the macOS：
|
1 2 3 |
sudo easy_install pip3 pip3 install my_package |
later，This method is ineffective，Become such：
|
1 2 3 4 |
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py python get-pip.py pip install my_package |
Finally one day，I need to update it prompted pip，then：
[cray[……]
When using python，Often you need to download some third-party framework，Fortunately, python has a similar apt package management tool，pip。
but，Although management packs can pip，But no source switching mirror，And we downloaded package，Most of the large foreign code hosting server，This often leads to a few hundred KB of package to download one hour。
[……]
Some time ago I wrote an article，SayIt is time to use the whitelist to the wallGot it，But that has expired long whitelist，It is not so smooth with the，Then I say boasted：I want to implement your own a reptile，Come crawling China's domain，Good update whitelist。
Ok，All in all this crawler is written on the line and then climb took more than ten thousand，But in the end I found the former to do a better solution，So the project was abandoned reptile[……]
When using Python to do network programming will inevitably encounter convert a string to the byte stream，Here we record the following commonly used methods：
First byte array to a string，That is str：
|
1 2 3 4 5 |
b = b'some byte array' str(b, encoding = "utf-8") #or bytes.decode(b) |
Then the string to a byte array：
[crayon-673f7da7bcb9e0715[……]