Algorithms for Programmers

15 years 3 months ago
Algorithms for Programmers
"This is the (work in progress) book "Algorithms for Programmers". Publication as soon as possible, content is (modulo copy editing) close to final version. I will put a remark here as soon as the printed version is available. The electronic version of this book is identical to the printed (to come) version. NOTE: It will always stay online for free anonymous download. Your feedback is appreciated."
Jörg Arndt
Added 30 Mar 2009
Updated 27 May 2009
Authors Jörg Arndt
Click here for more information about the book as well as the fxt library.

Part I    Low level algorithms  p.1
1   Bit wizardry  p.3
    1.1    Trivia
    1.2    Operations on individual bits
    1.3    Operations on low bits or blocks of a word
    1.4    Extraction of ones, zeros, or blocks near transitions
    1.5    Computing the index of a single set bit
    1.6    Operations on high bits or blocks of a word
    1.7    Functions related to the base-2 logarithm
    1.8    Counting the bits and blocks of a word
    1.9    Words as bitsets
    1.10   Avoiding branches
    1.11   Bit-wise rotation of a word
    1.12   Binary necklaces *
    1.13   Reversing the bits of a word
    1.14   Bit-wise zip
    1.15   Gray code and parity
    1.16   Bit sequency *
    1.17   Powers of the Gray code *
    1.18   Invertible transforms on words *
    1.19   Space filling curves
    1.20   Scanning for zero bytes
    1.21   2-adic inverse and square root
    1.22   Radix -2 (minus two) representation
    1.23   A sparse signed binary representation
    1.24   Generating bit combinations
    1.25   Generating bit subsets of a given word
    1.26   Binary words in lexicographic order for subsets
    1.27   Fibonacci words *
    1.28   Binary words and parentheses strings *
    1.29   Permutations via primitives *
    1.30   CPU instructions often missed
2   Permutations and their operations  p.97
    2.1    Basic definitions and operations
    2.2    Representation as disjoint cycles
    2.3    Compositions of permutations
    2.4    In-place methods to apply permutations to data
    2.5    Random permutations
3   Special permutations  p.111
    3.1    The revbin permutation
    3.2    The radix permutation
    3.3    In-place matrix transposition
    3.4    Rotation by triple reversion
    3.5    The zip permutation
    3.6    The XOR permutation
    3.7    The Gray code permutation
    3.8    The reversed Gray code permutation
4   Sorting and searching  p.129
    4.1    Sorting algorithms
    4.2    Binary search
    4.3    Flavors of sorting methods
    4.4    Searching in unsorted arrays
    4.5    Determination of equivalence classes
5   Data structures  p.149
    5.1    Stack (LIFO)
    5.2    Ring buffer
    5.3    Queue (FIFO)
    5.4    Deque (double-ended queue)
    5.5    Heap and priority queue
    5.6    Bit-array
    5.7    Left-right array
    5.8    Finite state machines
Part II    Combinatorial generation  p.169
6   Conventions and considerations  p.171
    6.1    About representations and orders
    6.2    Ranking, unranking, and counting
    6.3    Characteristics of the algorithms
    6.4    Optimization techniques
    6.5    The implementations, demo-programs, and timings
7   Combinations  p.175
    7.1    Binomial coefficients
    7.2    Lexicographic and co-lexicographic order
    7.3    Order by prefix shifts (cool-lex)
    7.4    Minimal-change order
    7.5    The Eades-McKay strong minimal-change order
    7.6    Two-close orderings via endo/enup moves
    7.7    Recursive generation of certain orderings
8   Compositions  p.193
    8.1    Co-lexicographic order
    8.2    Co-lexicographic order for compositions into exactly k parts
    8.3    Compositions and combinations
    8.4    Minimal-change orders
9   Subsets  p.201
    9.1    Lexicographic order
    9.2    Minimal-change order
    9.3    Ordering with De Bruijn sequences
    9.4    Shifts-order for subsets
    9.5    k-subsets where k lies in a given range
10  Mixed radix numbers  p.217
    10.1   Counting (lexicographic) order
    10.2   Minimal-change (Gray code) order
    10.3   gslex order
    10.4   endo order
    10.5   Gray code for endo order
11  Permutations  p.231
    11.1   Lexicographic order
    11.2   Co-lexicographic order
    11.3   Factorial representations of permutations
    11.4   An order from reversing prefixes
    11.5   Minimal-change order (Heap's algorithm)
    11.6   Lipski's Minimal-change orders
    11.7   Strong minimal-change order (Trotter's algorithm)
    11.8   Star-transposition order
    11.9   Minimal-change orders from factorial numbers
    11.10  Derangement order
    11.11  Orders where the smallest element always moves right
    11.12  Single track orders
    11.13  Permutations with special properties
    11.14  Self-inverse permutations (involutions)
    11.15  Cyclic permutations
12  Multisets  p.291
    12.1   Subsets of a multiset
    12.2   Permutations of a multiset
13  Gray codes for strings with restrictions  p.301
    13.1   List recursions
    13.2   Fibonacci words
    13.3   Generalized Fibonacci words
    13.4   Digit x followed by at least x zeros
    13.5   Generalized Pell words
    13.6   Sparse signed binary words
    13.7   Strings with no two successive nonzero digits
    13.8   Strings with no two successive zeros
    13.9   Binary strings without substrings 1x1 or 1xy1
14  Parentheses strings  p.319
    14.1   Co-lexicographic order
    14.2   Gray code via restricted growth strings
    14.3   Order by prefix shifts (cool-lex)
    14.4   Catalan numbers
    14.5   Increment-i RGS and k-ary trees
15  Integer partitions  p.333
    15.1   Solution of a generalized problem
    15.2   Iterative algorithm
    15.3   Partitions into m parts
    15.4   The number of integer partitions
16  Set partitions  p.345
    16.1   Recursive generation
    16.2   The number of set partitions: Stirling set numbers and Bell numbers
    16.3   Restricted growth strings
17  Necklaces and Lyndon words  p.361
    17.1   Generating all necklaces
    17.2   The number of binary necklaces
18  Hadamard and conference matrices  p.373
    18.1   Hadamard matrices via LFSR
    18.2   Hadamard matrices via conference matrices
    18.3   Conference matrices via finite fields
19  Searching paths in directed graphs  p.381
    19.1   Representation of digraphs
    19.2   Searching full paths
    19.3   Conditional search
    19.4   Edge sorting and lucky paths
    19.5   Gray codes for Lyndon words
Part III    Fast transforms  p.401
20  The Fourier transform  p.403
    20.1   The discrete Fourier transform
    20.2   Summary of definitions of Fourier transforms *
    20.3   Radix-2 FFT algorithms
    20.4   Saving trigonometric computations
    20.5   Higher radix FFT algorithms
    20.6   Split-radix Fourier transforms
    20.7   Symmetries of the Fourier transform
    20.8   Inverse FFT for free
    20.9   Real-valued Fourier transforms
    20.10  Multi-dimensional Fourier transforms
    20.11  The matrix Fourier algorithm (MFA)
21  Convolution, correlation, and more FFT algorithms  p.435
    21.1   Convolution
    21.2   Correlation
    21.3   Weighted Fourier transforms and convolutions
    21.4   Convolution using the MFA
    21.5   The z-transform (ZT)
    21.6   Prime length FFTs
22  The Walsh transform and its relatives  p.453
    22.1   Transform with Walsh-Kronecker basis
    22.2   Eigenvectors of the Walsh transform *
    22.3   The Kronecker product
    22.4   Higher radix Walsh transforms
    22.5   Localized Walsh transforms
    22.6   Transform with Walsh-Paley basis
    22.7   Sequency-ordered Walsh transforms
    22.8   XOR (dyadic) convolution
    22.9   Slant transform
    22.10  Arithmetic transform
    22.11  Reed-Muller transform
    22.12  The OR-convolution and the AND-convolution
    22.13  The MAX-convolution *
    22.14  Weighted arithmetic transform and subset convolution
23  The Haar transform  p.493
    23.1   The `standard' Haar transform
    23.2   In-place Haar transform
    23.3   Non-normalized Haar transforms
    23.4   Transposed Haar transforms
    23.5   The reversed Haar transform
    23.6   Relations between Walsh and Haar transforms
    23.7   Prefix transform and prefix convolution
    23.8   Nonstandard splitting schemes *
24  The Hartley transform  p.511
    24.1   Definition and symmetries
    24.2   Radix-2 FHT algorithms
    24.3   Complex FFT by FHT
    24.4   Complex FFT by complex FHT and vice versa
    24.5   Real FFT by FHT and vice versa
    24.6   Higher radix FHT algorithms
    24.7   Convolution via FHT
    24.8   Negacyclic convolution via FHT
    24.9   Localized FHT algorithms
    24.10  2-dimensional FHTs
    24.11  Automatic generation of transform code
    24.12  Eigenvectors of the Fourier and Hartley transform *
25  Number theoretic transforms (NTTs)  p.533
    25.1   Prime moduli for NTTs
    25.2   Implementation of NTTs
    25.3   Convolution with NTTs
26  Fast wavelet transforms  p.541
    26.1   Wavelet filters
    26.2   Implementation
    26.3   Moment conditions
Part IV    Fast arithmetic  p.547
27  Fast multiplication and exponentiation  p.549
    27.1   Asymptotics of algorithms
    27.2   Splitting schemes for multiplication
    27.3   Fast multiplication via FFT
    27.4   Radix/precision considerations with FFT multiplication
    27.5   The sum-of-digits test
    27.6   Binary exponentiation
28  Root extraction  p.567
    28.1   Division, square root and cube root
    28.2   Root extraction for rationals
    28.3   Divisionless iterations for the inverse a-th root
    28.4   Initial approximations for iterations
    28.5   Some applications of the matrix square root
    28.6   Goldschmidt's algorithm
    28.7   Products for the a-th root *
    28.8   Divisionless iterations for polynomial roots
29  Iterations for the inversion of a function  p.589
    29.1   Iterations and their rate of convergence
    29.2   Schröder's formula
    29.3   Householder's formula
    29.4   Dealing with multiple roots
    29.5   More iterations
    29.6   Convergence improvement by the delta squared process
30  The arithmetic-geometric mean (AGM)  p.603
    30.1   The AGM
    30.2   The elliptic functions K and E
    30.3   AGM-type algorithms for hypergeometric functions
    30.4   Computation of ß
    30.5   Arctangent relations for ß *
31  Logarithm and exponential function  p.631
    31.1   Logarithm
    31.2   Exponential function
    31.3   Logarithm and exponential function of power series
    31.4   Simultaneous computation of logarithms of small primes
32  Computing the elementary functions with limited resources  p.645
    32.1   Shift-and-add algorithms for log b(x) and bx
    32.2   CORDIC algorithms
33  Numerical evaluation of power series  p.657
    33.1   The binary splitting algorithm for rational series
    33.2   Rectangular schemes for evaluation of power series
    33.3   The magic sumalt algorithm for alternating series
34  Recurrences and Chebyshev polynomials  p.673
    34.1   Recurrences
    34.2   Chebyshev polynomials
35  Hypergeometric functions  p.693
    35.1   Definition and basic operations
    35.2   Transformations of hypergeometric functions
    35.3   Examples: elementary functions
    35.4   The function xx
    35.5   Elliptic K and E
36  Cyclotomic polynomials, product forms, and continued fractions  p.713
    36.1   Cylotomic polynomials, Möbius inversion, Lambert series
    36.2   Conversion of series to infinite products
    36.3   Continued fractions
37  Synthetic Iterations *  p.733
    37.1   A variation of the iteration for the inverse
    37.2   An iteration related to the Thue constant
    37.3   An iteration related to the Golay-Rudin-Shapiro sequence
    37.4   Iteration related to the ruler function
    37.5   An iteration related to the period-doubling sequence
    37.6   An iteration from substitution rules with sign
    37.7   Iterations related to the sum of digits
    37.8   Iterations related to the binary Gray code
    37.9   A function encoding the Hilbert curve
    37.10  Sparse variants of the inverse
    37.11  An iteration related to the Fibonacci numbers
    37.12  Iterations related to the Pell numbers
Part V    Algorithms for finite fields  p.771
38  Modular arithmetic and some number theory  p.773
    38.1   Implementation of the arithmetic operations
    38.2   Modular reduction with structured primes
    38.3   The sieve of Eratosthenes
    38.4   The order of an element
    38.5   Prime modulus: the fields Z=pZ = Fp = GF  (p)
    38.6   Composite modulus: the ring Z=mZ
    38.7   The Chinese Remainder Theorem (CRT)
    38.8   Quadratic residues
    38.9   Computation of a square root modulo m
    38.10  The Rabin-Miller test for compositeness
    38.11  Proving primality
    38.12  Complex moduli: the fields GF  (p2)
    38.13  Solving the Pell equation
    38.14  Multiplication of hypercomplex numbers *
39  Binary polynomials  p.831
    39.1   The basic arithmetical operations
    39.2   Multiplication for polynomials of high degree
    39.3   Modular arithmetic with binary polynomials
    39.4   Irreducible polynomials
    39.5   Primitive polynomials
    39.6   The number of irreducible and primitive polynomials
    39.7   Transformations that preserve irreducibility
    39.8   Self-reciprocal polynomials
    39.9   Irreducible and primitive polynomials of special forms *
    39.10  Generating irreducible polynomials from Lyndon words
    39.11  Irreducible and cyclotomic polynomials *
    39.12  Factorization of binary polynomials
40  Shift registers  p.875
    40.1   Linear feedback shift registers (LFSR)
    40.2   Galois and Fibonacci setup
    40.3   Error detection by hashing: the CRC
    40.4   Generating all revbin pairs
    40.5   The number of m-sequences and De Bruijn sequences
    40.6   Auto-correlation of m-sequences
    40.7   Feedback carry shift register (FCSR)
    40.8   Linear hybrid cellular automata (LHCA)
    40.9   Additive linear hybrid cellular automata
41  Binary finite fields:  GF (2n )  p.897
    41.1   Arithmetic and basic properties
    41.2   Minimal polynomials
    41.3   Fast computation of the trace vector
    41.4   Solving quadratic equations
    41.5   Representation by matrices *
    41.6   Representation by normal bases
    41.7   Conversion between normal and polynomial representation
    41.8   Optimal normal bases (ONB)
    41.9   Gaussian normal bases
A   The electronic version of the book  p.933
B   Machine used for benchmarking  p.935
C   The pseudo language Sprache  p.937
D   The pari/gp language  p.939 
Comments (0)