Textbook in PDF format Do you know C++ but not AI? Do you dream of writing your own AI engine in C++? From beginner to advanced, this book covers the internals of AI engines in C++, with real source code examples and research paper citations. As a programmer, your job is to harness the power of your AI platform and offer it up to your many users in top-level features. Whether your AI project is about writing sports content or auto-diagnosing X-ray images, your work as an AI developer is based on fundamentally the same architecture. And to do this at a scale that matches the capability of your workhorse models, you need a programming language to match its power. I'll give you three guesses which one I recommend. C++ is on the inside of all AI engines. Whereas Python is often on the outside wrapping around the various models, C++ is always closer to the machine and its hardware. PyTorch and Tensorflow have lots of Python code on the top layers, but the grunt work underneath runs in highly optimized C++ code. The main advantage of C++ is that it is super-fast, and has low-level capabilities, that makes its operations close to those of the hardware instructions. This is a perfect match, because AI engines need to run blazingly fast, with hardware-acceleration integrations direct to the GPU to handle literally billions of arithmetic calculations. And yet, C++ is also a high-level programming language with support for advanced features like classes and modularity, so it's great for programmer productivity. Key Features Transformer components in C++ Faster and smarter AI Play with an AI engine on your desktop Cutting-edge research optimizations Just C++ code without all the math Preface Part I: AI Projects in C++ Introduction to AI in C++ Transformers & LLMs AI Phones AI on Your Desktop Design Choices & Architectures Training, Fine-Tuning & RAG Deployment Architecture Part II: Basic C++ Optimizations Bitwise Operations Floating Point Arithmetic Arithmetic Optimizations Compile-Time Optimizations Pointer Arithmetic Algorithm Speedups Memory Optimizations Part III: Parallel C++ Optimizations Loop Vectorization Hardware Acceleration AVX Intrinsics Parallel Data Structures Part IV: Transformer Components in C++ Encoders & Decoders Attention Activation Functions Vector Algorithms Tensors Normalization Softmax Decoding Algorithms Tokenizer and Vocabulary Part V: Optimizing Transformers in C++ Deslugging AI Engines Caching Optimizations Vectorization Kernel Fusion Quantization Pruning MatMul/GEMM Lookup Tables & Precomputation AI Memory Optimizations Part VI: Enterprise AI in C++ Tuning, Profiling & Benchmarking Platform Portability Quality Reliability Self-Testing Code Debugging Part VII: Research on AI Optimization Overview of AI Research Advanced Quantization Knowledge Distillation Structured Pruning Early Exit and Layer Pruning Width Pruning Length Pruning Adaptive Inference Zero-Multiplication Models Logarithmic Models Arithmetic Optimization Research Ensemble Multi-Model Architectures Advanced Number Systems Neural Architecture Search Appendix 1: C++ Slug Catalog