When using Tensorflow，There has been a strange warning： Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA ，Although not affect the use，But looking annoying，You can use this command to turn it off： the.about['TF_CPP_MIN_LOG_LEVEL'] = '2' 。
but，Have you noticed it? " could speed up CPU computations"……Ok?!
background
In short，according toThe official，Tensorflow default does not support these advanced features to enhance the framework of compatibility Tensorflow，It can be used as much as possible more platforms，Thereby avoiding unnecessary compilation operation。But this is only the cost of using each platform has a CPU instruction set，The high-level commands can not add，After all, different CPU platforms they have different instruction sets of advanced technology。
Since I use macOS，GPU is an AMD，So I could not use Tensorflow accelerated with the GPU，With a very slow CPU，Can only run the test data (small)，But I still hope it faster。
So I Tensorflow Tips，For my CPU,，There are more advanced features can be turned on in order to accelerate the training - but how to open it? The answer is definitely a source code recompilation Tensorflow。
AVX2 and FMA
FMA is a modern CPU SupportAn advanced instruction set，Chinese called "fused multiply-add operation"；
AVX is supported by modern CPU "Advanced Vector Extensions instruction set”，Obviously，This means that AVX2 AVX advanced version of the meaning，It introduces the FMA arithmetic above，And floating point performance 2 Time。
In short，If Tensorflow able to directly use these advanced features，That training will be able to speed a lot faster，Then，We try to start from scratch to compile a Tensorflow。
Ready to work
Python
Obviously，We use Python to use Tensorflow，So you need to have Python environment：
|
1 2 3 |
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" export PATH="/usr/local/bin:/usr/local/sbin:$PATH" brew install python |
Here we use Brew installation Python3.
Next is to install the necessary dependencies：
|
1 2 3 |
pip3 install -U --user pip six numpy wheel setuptools mock 'future>=0.17.1' pip3 install -U --user keras_applications --no-deps pip3 install -U --user keras_preprocessing --no-deps |
If you have a problem using pip，I wish to refer to this article：Proper use PIP to install Python package to avoid TypeError: ‘module’ object is not callable
Source
Compiled from the source code Tensorflow，It would certainly have to download the source code，Do you like to find a directory location，Columns command execution source projects get Tensorflow： git clone https://github.com/tensorflow/tensorflow.git ，Enter the directory cd tensorflow ，We want to switch to the stable version of M.，First tohttps://github.com/tensorflow/tensorflow/releasesFind the latest stable version of the label，It is the time of writing this article，Stable tag is v2.1.0 ，Here we switch to Stable： git checkout v2.1.0 .
Basel
Tensorflow is to be used to compile the Bazel，Overall compilation process easy and simple，But it takes a little bit of Bazel installation tips。
First of all，Check your source code files Tensorflow tensorflow/configure.py ，Where about 53 Row position，There is a line _TF_MAX_BAZEL_VERSION = '0.29.1' ，This is the version we use in Bazel。
Please note：The latest version is Bazel 2.0，Directly brew installation bazel is not compile the Tensorflow。
First make sure the latest version of Xcode installed on your macOS，Then execute the command： sudo xcodebuild -license accept ，Then tohttps://github.com/bazelbuild/bazel/releasesDownload the installation package Bazel，Note that the file name should look like this： Basel-0.29.1-installer-darwin-x86_64.sh 。
Click the file name link to download directly above the specified version of the installation package bazel，Tensorflow future requirements may become the source code compiler，By the time the reader is the version number of the corresponding self-bazel。
Since macOS 10.15 Enhanced system security measures，So if you just execute the script， bazel is unable to complete the installation，Will prompt a similar "bazel is unknown developer release，Potentially harmful "kind of warning，Thus refuse to run。
Use the command： sudo spctl --master-disable This warning is temporarily closed，Allow the program to run arbitrary developer。
After closing the warning，You can install and use the Bazel： sh Basel-0.29.1-installer-darwin-x86_64.sh --user ，After installation is complete use Basel --version View，The result should be： Basel 0.29.1
Compile Tensorflow
Final，We can compile a，Back tensorflow directory，carried out ./configure Configure，It will ask you the location and Python library，If you do not use brew installed，You can use the following command to find the location of your Python：
|
1 2 3 |
$ where python3 /usr/local/bin/python3 /usr/bin/python3 |
If there are multiple results，The first election to。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
$ ./configure WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown". You have bazel 0.29.1 installed. Please specify the location of python. [Default is /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python]: /usr/local/bin/python3 Found possible Python library paths: /usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages Please input the desired Python library path to use. Default is [/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages] Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n No XLA JIT support will be enabled for TensorFlow. Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n No OpenCL SYCL support will be enabled for TensorFlow. Do you wish to build TensorFlow with ROCm support? [y/N]: n No ROCm support will be enabled for TensorFlow. Do you wish to build TensorFlow with CUDA support? [y/N]: n No CUDA support will be enabled for TensorFlow. Do you wish to download a fresh release of clang? (Experimental) [y/N]: n Clang will not be downloaded. Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n Not configuring the WORKSPACE for Android builds. Do you wish to build TensorFlow with iOS support? [y/N]: y iOS support will be enabled for TensorFlow. Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details. --config=mkl # Build with MKL support. --config=monolithic # Config for mostly static monolithic build. --config=ngraph # Build with Intel nGraph support. --config=numa # Build with NUMA support. --config=dynamic_kernels # (Experimental) Build kernels into separate shared objects. --config=v2 # Build TensorFlow 2.x instead of 1.x. Preconfigured Bazel build configs to DISABLE default on features: --config=noaws # Disable AWS S3 filesystem support. --config=nogcp # Disable GCP support. --config=nohdfs # Disable HDFS support. --config=nonccl # Disable NVIDIA NCCL support. Configuration finished |
The next parameter configuration is almost no need to set，All default to，We do not need GPU support (add, it's useless)，But in the end there is a support iOS，Here I chose y，After all by，Configuration，You can compile the。
Use the command Basel build --config=opt //tensorflow/tools/pip_package:build_pip_package Compile Tensorflow installation package，The default compile command is compiling for a local CPU，That is, to enable all current CPU instruction set support，Such is the translation of the Tensorflow accelerated it!
Compilation process is very slow，In my memory 16G 2.5 GHz Quad-Core Intel Core i7 CPU 下want 36301 Seconds to complete，full 10 hour(Given the wrong parameter，10Hours completely wasted) to 54,219.95 seconds to complete，A full 15 hours (15 years, 15-inch models rmbp)!
|
1 2 3 |
INFO: Elapsed time: 54219.950s, Critical Path: 1802.92s INFO: 22004 processes: 22004 local. INFO: Build completed successfully, 22776 total actions |
Another，I saw this warning at compile time：
|
1 |
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/util/ConfigureVectorization.h:300:10: warning: "Disabling AVX support: clang compiler shipped with XCode 11.[012] generates broken assembly with -macosx-version-min=10.15 and AVX enabled. " [-W#warnings] |
I thought it could not support the AVX want to do it again ...... but it does not seem to affect the final result。
Generating installation package
Use the command ./Basel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg To generate .whl Installation package：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg Sat Jan 18 03:59:17 AEST 2020 : === Preparing sources in dir: /var/folders/dn/p_qzcyss2m3gdnkkyl_bw_d00000gn/T/tmp.XXXXXXXXXX.MxFJDO4r ~/Downloads/tensorflow ~/Downloads/tensorflow ~/Downloads/tensorflow /var/folders/dn/p_qzcyss2m3gdnkkyl_bw_d00000gn/T/tmp.XXXXXXXXXX.MxFJDO4r/tensorflow/include ~/Downloads/tensorflow ~/Downloads/tensorflow Sat Jan 18 03:59:32 AEST 2020 : === Building wheel warning: no files found matching 'README' warning: no files found matching '*.pyd' under directory '*' warning: no files found matching '*.pd' under directory '*' warning: no files found matching '*.so.[0-9]' under directory '*' warning: no files found matching '*.dll' under directory '*' warning: no files found matching '*.lib' under directory '*' warning: no files found matching '*.csv' under directory '*' warning: no files found matching '*.h' under directory 'tensorflow_core/include/tensorflow' warning: no files found matching '*' under directory 'tensorflow_core/include/third_party' Sat Jan 18 04:00:00 AEST 2020 : === Output wheel file is in: /tmp/tensorflow_pkg |
Then we install it with pip： pip install /tmp/tensorflow_pkg/tensorflow-2.1.0-cp37-cp37m-macosx_10_15_x86_64.whl
At last，Do not forget to close before the open of the security warning： sudo spctl --master-enable
Test results of the installation
|
1 2 3 4 5 6 7 8 |
$ python3 Python 3.7.6 (default, Dec 30 2019, 19:38:26) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import tensorflow as tf >>> t = tf.constant('logcg') 2020-01-18 20:04:43.088104: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fefdc983db0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-01-18 20:04:43.088134: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version |
Now no longer prompt Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMAGot it，Try to run a model，Finally reached a long time can endure。
：）
References
- Building TensorFlow from source for SSE/AVX/FMA instructions: worth the effort?
- Build from source
- https://github.com/bazelbuild/bazel/issues/9304
Original article written by LogStudio：R0uter's Blog » On macOS compiled Tensorflow to open AVX2 and FMA
Reproduced Please keep the source and description link：https://www.logcg.com/archives/3283.html
