Here is the current search code of the chess engine Stockfish:
Here are some useful notes:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 | # Stockfish Search Patch Development Insights
Here is a consolidated, comprehensive, and organized list of high-value insights for writing Stockfish search patches, focused on static code analysis and core search mechanics:
1. **Search Algorithm Fundamentals & Architecture**
* Stockfish uses a fail-soft alpha-beta search, parallelized with Lazy SMP, where each thread conducts an independent search and communicates primarily through the shared transposition table.
* The search is highly non-uniform due to aggressive pruning and reductions, resulting in a shallow search where most time is spent at low depths, prioritizing efficiency there.
* The search operates without tablebase knowledge during standard testing, so patches should not assume tablebases are available.
* The search is fundamentally an aspirated Principal Variation Search (PVS), where non-PV nodes are searched with a null window (`alpha == beta - 1`).
* The `alpha < beta` invariant is fundamental; any patch modifying alpha, beta, or aspiration windows must strictly ensure this invariant is never violated.
* The correctness of the search often emerges from the holistic interaction of its components, not from each function being naively "perfect" in isolation.
* A critical pitfall in pruning is eliminating the only good move in a position, which can lead to catastrophic performance drops.
* The search philosophy prioritizes a deeper, narrower search over a shallower, full-width one, justifying aggressive pruning techniques.
* The search near the root does not behave like a simple iterative deepening loop; due to aspiration windows and alpha-beta cutoffs, a node searched to depth `d` has likely not been fully searched to all shallower depths.
* The search process is inherently chaotic and sensitive to small changes; seemingly unrelated modifications can drastically alter search behavior.
* The engine has a fixed maximum search depth of 245 plies.
* The search's primary goal is finding the best move, not necessarily announcing a mate score; a high centipawn score is often practically equivalent to a mate-in-N.
* The search is highly dynamic; as depth increases, it can often correct for incorrect or suboptimal information from shallower plies.
* The `improving` flag, a key LMR component, indicates if the static evaluation has improved over the last two plies and is determined by comparing `staticEval` from `ss` and `ss-2`.
* The `value` variable inside a move loop is not initialized for the first move; it holds the search result from the *previous* move, making its use before the first move's search a common logical error.
* The `cutNode` flag's logic is highly non-intuitive, especially its interaction with Null Move Pruning (NMP), and attempts to "fix" it have historically failed.
* The `uciScore` at the root is not persistent but is overwritten during the search; its primary purpose is to indicate if a score from an aspiration window is a lower or upper bound.
* The search intentionally uses a smaller, faster 'smallnet' for positions with a very high `simpleEval`; this is a deliberate performance optimization.
* The `pos.capture_stage(move)` function identifies if a move is tactical or "noisy" (including promotions), not strictly if it's a capture.
2. **Transposition Table (TT) Usage and Semantics**
* The TT is a major bottleneck; adding more data per entry is infeasible without a fundamental structural change, as its memory layout is optimized for NUMA systems and cache-line alignment.
* When storing a pointer to a TT entry for later use, always store the position's Zobrist key as well, and verify it before accessing the entry's data, as the entry may have been overwritten.
* TT entries store the nominal search depth, ignoring extensions, and quiescence search entries are stored with a special constant (`DEPTH_NONE`).
* The TT replacement strategy is highly effective even if it seems illogical (e.g., disregarding depth for replacement), and attempts to "fix" it historically have failed.
* The `Bound` enum in TT entries specifies the nature of the stored score: `BOUND_UPPER` (fail-lows), `BOUND_LOWER` (fail-highs), and `BOUND_EXACT` (PV-nodes, meaning both upper and lower bound).
* The TT does not store information about the 50-move rule, which can lead to Graph History Interaction (GHI) issues, potentially turning a win into a draw.
* When adding new data to a TT entry, you must repurpose bits from existing fields (e.g., reducing the key size) due to strict size constraints for cache efficiency.
* The `DEPTH_NONE` constant in the transposition table is specifically offset to ensure all valid entries have a non-zero `depth8` value, critical for `hashfull()` occupancy checks.
* Transposition table lookups are used to prune in quiescence search, providing a cutoff before any moves are generated.
* If new information is introduced that influences search decisions, it must be stored in the TT entry; otherwise, a TT hit will revert to untuned logic, degrading performance.
* The `ttDepth > depth` condition in a zero-window search often implies a TT cutoff would have already occurred, meaning code under this condition may execute less frequently.
* The static evaluation stored in the TT (`tte->eval()`) is distinct from its `search score` (`tte->value()`); confusing them can lead to incorrect patches.
* Modifying logic for nodes that miss the TT cache, especially those without a cached static eval, is extremely sensitive and can cause catastrophic Elo loss.
* To prevent data races in multi-threaded search, copy a TT entry to a local variable immediately after a successful probe and use the copy for all subsequent operations.
* A non-null `ttMove` does not guarantee a valid TT hit (`ss->ttHit`), especially at the root where `ttMove` can be populated from `rootMoves`, which can lead to accessing garbage data if the original TT entry was overwritten.
* After a sub-search (like `qsearch`) or a fail-high, re-probe the TT to get updated information and a new `ttMove`, as the sub-search may have updated the entry, but `ss->ttHit` of the parent is not automatically updated.
* The primary benefit of the TT is providing the `ttMove` for singular extension checks, which is why hash collisions are less harmful than one might assume, as a random garbage move is often illegal and rejected.
* The search intentionally uses stateful evaluation concepts like `optimism`, meaning a value stored in the TT may not be "correct" on a subsequent visit, but this is a proven strength-gaining feature.
* The `ttValue != VALUE_NONE` check is a critical safeguard against TT access race conditions in multi-threaded search and should not be removed.
* Storing the raw, unscaled NNUE evaluation in the TT is a viable strategy to avoid storing stale data (like optimism or 50-move rule scaling), allowing for the final evaluation to be recalculated on the fly.
* In TT replacement logic, entries from PV-nodes (`b == BOUND_EXACT`) are considered extremely valuable and are often programmed to overwrite almost any existing entry, regardless of depth.
* A TT entry with an `EXACT` bound does not mean the value is the true score of the position, but rather that the score fell between the alpha and beta of the search that stored it.
* Tablebase results are stored in the TT, often without a static evaluation value (`ttEval`), primarily to enable fast TT cutoffs on subsequent visits to the position.
* When accessing a `TTEntry`, be cautious of using its members without verification; ensure the data is valid and initialized via a proper TT hit guard.
* Hard pruning based on a TT entry having a value below alpha (`ttValue < alpha`) is generally a bad heuristic; the only widely accepted form is razoring, which provides limited gains.
* A TT entry with a lower bound (`BOUND_LOWER`) and a value below the current search's alpha was actually a fail-high result from a previous, often shallower, search, meaning the move is likely strong and should not be prematurely discarded.
* The `ttValue != VALUE_NONE` check is essential for thread safety, safeguarding against race conditions where a TT entry might be accessed before its value is fully written.
* The transposition table key is adjusted based on the 50-move rule counter (`rule50`) to differentiate positions, typically by XORing the key in buckets of 8 plies after a certain threshold.
* A `ttMove` must always be checked for pseudo-legality before use, as it could be invalid due to a hash collision; if illegal, invalidate the TT hit (`ss->ttHit = false`).
* A valid TT entry can legitimately have no best move (`ttMove` is `MOVE_NONE`) if the node caused a beta cutoff before the move loop began or if the search completed without finding any move that improved alpha.
* The TT should not be polluted with speculative scores from techniques like null-move pruning; only results from "real" searches (even shallower ones like probcut) should be stored.
* In multi-threaded search, cache the `ttValue` in a local variable at the beginning of the search function to avoid race conditions where another thread updates the TT entry.
* To prevent race conditions in multi-threaded search, immediately copy all required TT entry data (depth, score, bound type, move) into local variables or a dedicated struct upon a successful TT probe.
* The `TTEntry` struct should be treated as a private implementation detail of `tt.cpp` and not exposed in public headers to the rest of the search code.
* Encapsulate related probed TT values (`ttMove`, `ttValue`, `ttDepth`, etc.) into a single struct (e.g., `ProbedTTEntry`) for clarity and to abstract away raw `TTEntry*` pointers, preventing accidental use of racy pointers.
* The 16-bit key used for TT lookups within a cluster is not a full guarantee of a position match; storing extra verification data (e.g., piece count) in unused bits of the TT entry can reduce hash collision impact.
* The TT replacement strategy balances entry depth and age; always preferring to replace lower-depth entries can be a bug if the age penalty (`relative_age`) causes useful deep entries to be discarded prematurely.
* When a search modification changes the effective search depth of a node, the depth stored in the corresponding TT entry must also be updated accurately.
* The constants used for depth offsets in the TT (e.g., `TTEntry::depth_adj`) are carefully chosen, and their sign has specific meaning.
* Artificially increasing the depth of a tablebase result (e.g., `depth + 6`) when storing it in the TT is a specific technique to force TT cutoffs and avoid repeated, expensive tablebase probes.
* The `hashfull` metric is calculated using a generation counter (`generation8`), ensuring it only counts entries written during the *current* search.
* When modifying the `TTEntry` struct, the containing `Cluster` size must be adjusted to remain a power of two (e.g., 32 or 64 bytes) by adding padding.
* To add a single bit of information to a `TTEntry` without increasing its size, "steal" a bit from an existing multi-bit field (e.g., reducing `generation` from 5 bits to 4).
* The `genBound8` field in `TTEntry` is a bitfield where different bits represent different flags, such as `bound()` (2 bits) and `is_pv()` (1 bit).
* The TT replacement strategy has specific rules; if a new search result is a fail-low (no best move found), the existing TT entry with a best move should be kept.
* A TT entry is typically only used for a cutoff if its searched depth is strictly greater (`>`) than the current search depth, not greater than or equal (`>=`).
* The TT does not store the alpha-beta window of a search; it only stores the resulting score and its type (exact, lower bound, or upper bound).
* The TT only stores a truncated portion of the full 64-bit Zobrist key (e.g., the lower 16 bits) for verification, which is the fundamental reason hash collisions are possible.
* The danger of a hash collision is not its mere existence, but that the search uses the incorrect value from the colliding entry to make a premature and incorrect pruning decision (a "cutoff").
* The `TranspositionTable::Cluster` has a strictly enforced, performance-critical size (e.g., 32 bytes); adding data to it will break static assertions and require a major redesign.
* The bits within a TT entry are a highly constrained resource; improving one feature (like collision resistance by adding key bits) may require sacrificing bits from another field (like the `generation` counter).
* Increasing the size of data stored per TT entry (e.g., using a 64-bit key instead of 16-bit) is often a net loss, as fewer total entries in a fixed-size hash table typically outweighs the benefit of reduced collisions.
* The TT size is not required to be a power of two; the hashing mechanism using `mul_hi64` allows for arbitrary table sizes.
* If a TT hit is from a very shallow depth, it can be beneficial to skip certain extensions (like singular extensions) and proceed to more forceful ones (like double extensions) to get a more reliable search.
* The result of a stand-pat evaluation in quiescence search is only saved to the transposition table if the entry is empty (`!ss->ttHit`); this prevents simple static evaluations from overwriting valuable entries.
* The effectiveness of certain search features, particularly extensions and pruning techniques, is highly sensitive to the size of the transposition table.
* Pay close attention to the logic of transposition table cutoffs; an upper bound entry (fail-low) implies the true score is at most `ttValue`, while a lower bound entry (fail-high) implies the score is at least `ttValue`.
* When updating heuristics keyed by the pawn hash, you must add a guard to prevent updates for a previous move if that move was a pawn move, as failing to do so means using a stale pawn hash key.
* A fundamental safety principle is to prevent scores from incomplete or aborted searches from being written to the transposition table. The `Threads.stop` flag is the key guard for this.
* Checks that seem redundant can be critical; a TT entry can be overwritten by a recursive call, making it necessary to re-validate the move.
* The TT move is checked *before* move generation to allow for fast cutoffs; storing it as an index into a generated move list would defeat this purpose.
* When using a TT entry, it is absolutely critical to verify the Zobrist key matches the current position's key (`entry->zobristKey == shrink(hash)`).
* A common condition for using a TT value involves checking if its depth is sufficient relative to the current search depth (e.g., `ttValue.depth > depth - 2`).
* A TT hit does not guarantee a valid TT value; code must check `ttData.value != VALUE_NONE`, not just `ttHit`.
* The TT replacement strategy prioritizes entries from a more recent search (`generation`) over those with a greater search depth, as raw depth is an unreliable indicator of search quality.
* A move ordering bonus can be given to a TT move even if it previously caused a fail-low, based on the heuristic that it might still be the best move.
* When a TT entry exists, its score is considered a better estimate than a fresh static evaluation if the static eval falls on the "wrong side" of the TT bound.
* Storing a second move in the transposition table is generally ineffective due to the nature of cutoffs in non-PV and PV nodes.
* Be aware of the distinction between `ttHit` (a TT entry exists) and `ttMove` (the move being searched is from the TT entry); confusing them is a common and subtle bug.
* Using mate scores directly from the transposition table without validation can lead to announcing incorrect mates, especially in PV-nodes where they are often ignored.
* Adding checks early in the search function (e.g., TT move legality) to prevent hash collisions is frequently too slow to be beneficial.
* The number of entries per transposition table cluster is a quality-of-search parameter, not just a cache performance optimization.
* A TT hit can be used to infer that a position was likely visited in a previous, probably reduced, search, leveraging existing knowledge.
* TT cutoffs are disabled in PV-nodes; if a line is promoted to PV, it must be re-searched to correct earlier mistakes or shallow cuts.
* Modifying TT replacement policies is a low-probability path for improvement; successful changes usually involve differentiating for PV-nodes.
* A simple depth extension for non-PV nodes can be effective because it causes moves that fail high to be stored in the TT with greater depth, improving TT move quality.
* If the TT move consistently causes a fail-high, the search may not explore other promising moves, highlighting a challenge in ensuring sufficient exploration.
* When a bonus is applied to an already strong move found in the TT, it can be beneficial.
* TT cuts based on shallower search depths can be surprisingly powerful when the stored score is close to the current alpha-beta window, not just when it is far outside of it.
* Pruning conditions based on transposition table scores can be asymmetric; a condition like `tt.value() <= alpha + C` might be beneficial, while `tt.value() >= beta - C` can be detrimental.
* The MovePicker handles the TT move in a specific sequence: it's passed to the MovePicker, returned immediately if valid, and then explicitly filtered out during subsequent move generation stages to prevent it from being processed twice.
* When implementing ideas at singular extension nodes, be extremely careful about state; reusing TT static evaluation can fail if related state is not properly saved and restored.
* The transposition table is still probed during the tree search that follows a static exchange evaluation (SEE).
3. **Search Depth, Extensions, and Reductions**
* `rootDepth` is a thread-level variable tracking the current iteration of iterative deepening at the root, while `depth` is a parameter passed down recursive calls indicating remaining search ply.
* Search optimizations should be heavily focused on shallow depths, as the vast majority of nodes (over 99%) are at low depths (e.g., depth <= 10).
* The nominal `depth` variable is merely the iterative deepening iteration number, not the true ply depth, which is heavily modified by extensions and reductions.
* When implementing depth-dependent logic, consider the edge case of very large depths; cap values or use gentler scaling.
* Modifying the `depth` variable directly within the main move loop is difficult to reason about; it is often preferable to adjust move-specific reductions or extensions instead.
* Search explosion guards, like the cap on `ss->doubleExtensions`, are often more robust than simple `rootDepth` checks, preventing infinite loops in complex recursive paths.
* The `ss->doubleExtensions` counter tracks the total number of double extensions from the root to the current node, used as a safety mechanism.
* Some search parameters, like extension depth margins, are extremely sensitive; even seemingly logical changes have been tried and failed repeatedly.
* A change that slows down shallow search can, paradoxically, improve performance at very high depths (e.g., lowering singular depth threshold).
* Heuristics based on absolute search depth (e.g., `rootDepth < 10`) are discouraged as they do not scale well; using depth differences is more robust.
* LMRs are fundamentally based on the logarithm of the move index (`log(i)`), reflecting that later moves are exponentially less likely to be good.
* When a depth reduction is applied, a subsequent `if (depth <= 0)` check can trigger a transition to quiescence search; if the condition implies a PV-like context, call `qsearch<PV>`.
* Historically, attempts to introduce depth-dependent parameters into the search have almost always failed to produce consistent gains.
* The search tree in Stockfish is not a classic, broad tree but is heavily shaped by singular extensions, creating a very deep, narrow search focused on the presumed principal variation.
* Search changes that add extensions or alter reduction logic must be carefully guarded to prevent "search explosions" and are often controlled by existing mechanisms like the `doubleExtensions` counter.
* Counters for depth-increasing features must track the total accumulation down a search path, as the risk of search explosion comes from the combined effect.
* When programmatically altering search depth, it is crucial to use guards (e.g., `if (newDepth > d)`) to ensure the resulting depth does not violate intended search boundaries.
* Clamping a new depth's minimum value to 1 can prevent the search from ever reaching `depth <= 0`, thus skipping quiescence search and causing an infinite loop.
* The effectiveness of search heuristics, particularly extensions like singular extensions, can vary significantly with search depth, and their parameters may need to be scaled.
* Pruning should be less aggressive in PV-nodes to avoid missing tactical refutations; a common implementation is to apply significantly smaller Late Move Reductions (LMR) in PV-nodes.
* The established method for treating PV nodes differently in reductions is to use the `delta` margin, which naturally makes reductions less aggressive for PV nodes.
* A common and critical pitfall is misinterpreting logical conditions in reduction logic (e.g., `!PvNode`), which can inadvertently increase reductions for important nodes.
* Weakening aggressive pruning techniques like Late Move Pruning (LMP) requires a high-value trade-off, ensuring the added exploration is highly valuable.
* Pruning is disabled when `bestValue` remains at its initial `-VALUE_INFINITE` to ensure the search explores all moves to find a legal one before declaring checkmate.
* Move count pruning is somewhat self-regulating because its threshold grows rapidly with depth, making it naturally less active in deep searches.
* Before implementing a new pruning technique, check if its conditions are already covered by an existing one, as new, similar checks can be redundant and inefficient.
* Understand the distinction between similar pruning methods: Razoring verifies a low static eval with an expensive `qsearch`, while Futility Pruning uses a high static eval to assume a fail-high without verification.
* The strength of the engine allows for very aggressive pruning; heuristics like razoring are pushed to high depths because the underlying evaluation and search are strong enough.
* A `qsearch` call is computationally expensive; it is justified in a precise heuristic like razoring because the static check alone is not perfectly reliable.
* Futility pruning conditions, particularly the depth check (e.g., `if (depth < 8)`), are critical for mate-finding performance.
* The futility pruning formula `margin * (depth - improving)` is effective because it correctly reduces the pruning margin to zero when `depth` is 1 and the side to move is `improving`.
* Pruning thresholds (e.g., for move count pruning) do not need to be static; they can be made dynamic by incorporating other heuristics, such as a move's history score.
* LMR are scaled by the number of threads (e.g., using `std::log(Threads.size())`) to counteract the search-widening effect of Lazy SMP.
* Pruning techniques like ProbCut or Futility Pruning operate on reduced depths (e.g., depth - 3), so their results, including potential mates, are "unproven" and should be handled with more caution.
* When a pruning heuristic at a reduced depth finds a win, use `VALUE_KNOWN_WIN` rather than `VALUE_TB_WIN_IN_MAX_PLY` to avoid propagating an unproven, shallow result as a certain mate.
* Reductions and extensions are fundamentally attempts to estimate how evaluation accuracy might change with more depth, allocating resources to more promising lines.
* LMR is the primary mechanism for selective deepening; it applies a baseline reduction that is then modified by other heuristics.
* LMR and pruning are often less aggressive or disabled entirely in PV-nodes; applying more LMR to captures in PV-nodes has historically failed.
* When the engine's position is improving (`improving` is true), it is more likely to cause a fail-high (beta cutoff), so pruning techniques that rely on causing a fail-high (futility, null move) should be made *more* aggressive.
* Futility pruning is one of the few search heuristics that directly uses static evaluation, making its margins and behavior highly sensitive to changes in the NNUE.
* Negative extensions are a specific pruning technique that can be applied conditionally, for instance, only when the evaluation is significantly higher than `singularBeta`.
* Aggressive pruning is a fundamental design choice for strength, prioritizing finding a win over the optimal path, and can cause the engine to miss shortest mates.
* Reducing pruning can boost positional understanding but may introduce shallow tactical holes by making the search narrower and longer.
* Consider disabling or reducing pruning heuristics at very low search depths (e.g., depth < 4) to improve history quality.
* Futility pruning conditions are highly sensitive and must account for mate scores (e.g., `abs(beta) < VALUE_MATE_IN_MAX_PLY`) to prevent incorrect pruning of forced mates.
* Reductions are generally safer and more effective when applied only to non-PV nodes.
* LMR have built-in safeguards, ensuring reductions do not reduce depth below a minimum threshold (e.g., depth 1).
* Futility pruning is not based on static evaluation alone; a move with a very poor history score can be aggressively pruned even if the static evaluation is above alpha.
* The order of operations is critical: performing pruning *before* applying extensions is more effective.
* The combined strength of multiple pruning heuristics is often significantly greater than the sum of their individual contributions, as they compensate for each other's weaknesses.
* NNUE's higher evaluation accuracy allows the search to prune branches more heavily with less risk of tactical errors, enabling more aggressive reductions.
* New pruning and reduction logic should typically be disabled for PV nodes (`!PvNode`), captures, and promotions, as these are critical parts of the search where aggressive pruning is too risky.
* When implementing new pruning or reduction heuristics based on static evaluation, add a `!ss->inCheck` condition, as checks often invalidate the assumptions behind such heuristics.
* Futility pruning, which relies heavily on static evaluation, has become more powerful relative to other heuristics with NNUE.
* Futility margins are dynamic, often depending on remaining depth (`d`) and whether the side to move is improving (`improving`), e.g., `Value(147 * (d - improving))`.
* The absolute evaluation cap for futility pruning can be set very high (e.g., `eval < 22266`) to avoid pruning positions with overwhelming material advantages that are not yet mate.
* Razoring margins are complex functions of depth, such as a quadratic formula (`alpha - C1 - C2 * depth * depth`), making them more aggressive at deeper plies.
* Razoring can effectively use an asymmetrical window like `[alpha - 1, alpha]`.
* Aggressive NMP can be a liability in complex positions, causing the engine to miss tactical wins; restricting NMP in certain situations can be beneficial.
* NMP is less reliable when the current evaluation comes from a TT entry that previously caused a beta cutoff (fail-high), suggesting NMP should be more conservative in this situation.
* Applying depth reductions at PV-nodes and cut-nodes is a valid and powerful technique, even aggressive ones like `depth -= 2` under certain conditions.
* LMR is far more effective at cut nodes compared to PV nodes.
* LMR formulas are complex, often incorporating the score's relation to beta, current depth, and game complexity.
* Applying aggressive LMR at very low depths (e.g., starting from depth 2) can be surprisingly effective, potentially leading to deeper selective searches in key lines.
* Reducing LMR for PV nodes, especially at low depths, is a known technique to improve tactical performance.
* A negative reduction value (`r < -1`) can be used to implement a targeted extension under very specific conditions, such as for early moves in a PV-node when not already double-extended.
* The `see_ge(0)` check is a fundamental heuristic used to quickly determine if a capture is non-losing based on Static Exchange Evaluation (SEE), often as a condition for pruning or move ordering.
* Pruning in Stockfish is a complex, multi-faceted system incorporating various factors beyond evaluation scores.
* The order of pruning checks and other search operations is critical; performing a pruning check *after* a recursive search call can lead to flawed logic.
* Different pruning techniques have vastly different accuracy profiles (e.g., NMP ~99.5%, Razoring ~98%, Futility ~94%).
* Aggressive pruning changes (e.g., reducing futility pruning depth) often show gains at short time controls but perform poorly in deep, long time control searches.
* Hard pruning heuristics (like razoring) should generally include checks like `beta < VALUE_KNOWN_WIN` and `alpha > -VALUE_KNOWN_WIN` to prevent incorrectly pruning sacrificial mate lines.
* Pruning changes can have non-linear effects with time control; a patch reducing pruning might pass at short time controls but fail at longer ones, or vice-versa.
* Aggressive pruning can scale poorly in SMP, as it might remove nodes beneficial for other threads.
* Pruning margins that rely on `staticEval` or `PieceValue` are implicitly tied to the scale of the neural network's output; if the network produces larger evaluations, existing thresholds become more aggressive.
* Many pruning conditions are a finely tuned balance of multiple components; modifying one part can unbalance the entire heuristic.
* Futility pruning is extremely aggressive, cutting significantly more nodes than NMP, highlighting its importance and sensitivity.
* Futility pruning conditions are not always redundant; checks like `eval >= beta` are essential even with other margin-based conditions, as terms like `statScore` can be negative.
* A fundamental weakness of NMP is its failure in zugzwang positions; patches improving zugzwang detection can fix tactical oversights.
* NMP depth reduction is a critical and complex formula, often exhibiting non-linear scaling based on depth, `eval` relative to `beta`, and position complexity.
* NMP is handled differently at high depths (`depth >= 14`), requiring a verification search; simply enabling NMP at high depths without this is a major bug.
* The check preventing consecutive null moves is a significant, historical safeguard; removing it relies entirely on the aggressive depth reduction formula for null moves.
* An alternative to forbidding consecutive null moves is to permit them but apply extra, harsher reductions on the second null move in a sequence.
* NMP is less reliable when the current evaluation is much higher than the static evaluation, as this disparity can indicate a temporary advantage that disappears after a null move.
* The primary defense against infinite loops from repeated null moves is the aggressive depth reduction formula, designed to deplete search depth very quickly.
* The verification search in NMP does not need to be at the same reduced depth as the main NMP search; experimenting with a deeper verification search is a valid approach.
* Verification search has its own search rules that can act as a failsafe against pathologies, such as disabling NMP for the side to move.
* Unlike singular searches, Null Move Pruning (NMS) searches use the next stack entry (`ss+1`), meaning NMS does not corrupt the state of the current node's stack entry.
* In Null Move Pruning (NMP), when the null move search fails high (`nullValue >= beta`), the returned score must be capped below tablebase win scores to prevent returning an unproven mate score.
* The application of major pruning techniques like NMP is often gated by depth or other search state (e.g., disabled at very high depths or under specific conditions).
* NMP is based on the assumption that having an extra move is always an advantage, which fails in zugzwang positions (approx. 4.5% of cases), necessitating verification searches.
* NMP is one of the less precise pruning techniques (approx. 96% accurate), lower than futility pruning, probcut, and razoring.
* Verification search is performed at the same depth as the initial null-move search it verifies, but with a very large depth reduction (`R`), often resulting in a `qsearch` call.
* The standard way to prevent recursive null move verification is to check `thisThread->nmpMinPly` and return early, as it signals a verification search is in progress.
* When guarding against recursive verification, returning `nullValue` is the established practice; alternatives have performed worse.
* NMV is a critical safety feature; removing it can introduce permanent tactical blind spots unrecoverable with more depth.
* The sole purpose of verification search is to catch zugzwangs, which are a known weakness of NMP; it is historically Elo-neutral.
* Verification search is skipped if `abs(beta) < VALUE_KNOWN_WIN` and `depth < 14`, as zugzwangs are less likely in these scenarios.
* The search techniques are often ordered by relative cost; null move search is performed before its verification because it prunes a smaller, cheaper tree.
* A flawed recursive verification search might devolve into a chain of repeated null move searches.
* When saving a Transposition Table (TT) entry from a search that has already made a move (e.g., quiescence search inside ProbCut), the depth must be adjusted to reflect the consumed ply (e.g., `depth - 3` instead of `depth - 4`).
* A search at `depth <= 0` is often an error condition; if a reduction or other logic could lead to this, the depth should be explicitly clamped (e.g., `depth = std::max(depth, 1)`).
* Be aware that search can reach very high ply counts due to extensions; a simple linear dependency on `depth` might have counter-intuitive effects in deeply extended lines.
* The conditions for applying search extensions are critical; a simple but powerful condition, such as requiring a move to be a non-capture from the TT (`!ttCapture`) for triple extensions, can be key to Elo gain.
* The history of Stockfish search shows a clear trend toward more aggressive and layered extensions (e.g., double, triple, quadruple); patches exploring new conditions for extending singular or promising moves are fruitful.
* Extension margins (e.g., for double or triple extensions) are very sensitive; setting a margin to zero can be highly detrimental.
* Counter-intuitive ideas, like very large, sudden extensions (e.g., extending by 4 plies instead of the usual 2), can be surprisingly effective and improve time control scaling.
* Singular Extensions are highly sensitive to non-linear scaling; decreasing `singularBeta` (e.g., by increasing the divisor) is known to improve performance at longer time controls but often hurts at shorter ones.
* Deeper extensions (e.g., moving from double to triple or quad extensions) are a known strategy for gaining strength at very long time controls, but are expected to lose Elo at short time controls.
* Completely unlimited extensions are known to be harmful; while the trend is to increase extension limits for Elo, a limiting factor is necessary to prevent search explosions.
* The `ss->multipleExtensions` counter is the main defense against search explosions; its logic is subtle, and changes to extension amounts must be considered in tandem with this limiting mechanism.
* Unconditional, global extensions are extremely dangerous and can cause an exponential explosion in search time (nodes), making them almost certain to fail.
* Extensions must be made conditional to be viable, often limited by depth (e.g., `depth < 7`) to prevent them from triggering in deep search where they cause the most explosion.
* If a simple constant extension (e.g., `depth += 1`) is too aggressive, a common and effective alternative is to make it conditional (e.g., `depth += (depth < X)`), limiting its application to shallower parts of the search tree.
* The current extension framework is hierarchical (e.g., a quadruple extension requires a triple extension to be true); making these extension conditions independent has been attempted and was not successful.
* The search balances pruning and extensions; Stockfish's development often oscillates between adding more aggressive pruning and adding more extensions.
* There is a critical balancing act between extensions and pruning/reductions; when new extensions are added, it creates opportunities for new pruning or reduction patches to succeed by restoring search speed.
* Aggressive extensions can mask other improvements; if the search is configured with very aggressive extensions, it can "dilute" the effect of patches in other areas.
* Unconditional or overly broad depth reductions (e.g., `depth--` or `depth -= 2`) are a common pitfall and historically perform poorly at longer time controls.
* The reduction variable `r` is subtracted from the new search depth (`newDepth - r`), so a larger positive value for `r` corresponds to a larger (deeper) reduction.
* When modifying reductions, a change that results in a larger value for the reduction variable `r` means the move is being reduced *more*, not less.
* The amount of depth reduction applied in techniques like Null Move Reduction (NMR) is dynamic, often depending on factors like the current search depth and how much the static evaluation exceeds the search window's bound.
* The historical trend in Stockfish development has been to make negative extensions (reductions) less aggressive, suggesting that overly harsh reductions are a common anti-pattern.
* Aggressively changing extensions or reductions for PV-nodes, especially those with a TT move, is very risky and can lead to large, unexpected Elo losses.
* Be cautious with heuristics that use depth-squared terms (e.g., `depth * depth`); these values can grow so rapidly that they make a pruning or reduction technique completely non-functional at moderate to high depths.
* A reduction of `r--` (e.g., for `singularQuiet`) effectively acts as a one-ply extension for promising moves when the main `ttMove` fails to cause a cutoff.
* The depth reduction in ProbCut (e.g., `depth - 4`) is tightly coupled with the depth stored in the transposition table (`depth - 3`) to accurately reflect that a move has been made.
* Extensions are not a single mechanism and are implemented in multiple places, including explicit extensions, post-LMR re-searches, and allowing LMR'd depth to exceed the original `newDepth`.
* Patches that add or increase the number of extensions tend to scale well with longer time controls, whereas patches that reduce extensions are more likely to scale negatively.
* Triple extensions scale more linearly with longer time controls, while double extensions tend to plateau in effectiveness, suggesting a focus on triple extensions for VVLTC gains.
* The reason depth-scaling for extensions fails is that longer time controls require extending moves at *shallower* search depths to build a deep and accurate principal variation.
* Double extensions and post-LMR extensions (e.g., `goDeeper`) are very strong but are historically known to be a primary cause of search instability and getting "stuck."
* Hard-capping the number of double extensions based on depth has historically failed, suggesting that allowing more extensions at greater depths is generally better.
* The `depth` variable is not a reliable measure of how deep the search will actually go; true search depth can be up to twice the nominal depth due to extensions.
* Increasing Internal Iterative Reductions (IIR) or general depth reductions tends to perform poorly at longer time controls.
* Reducing extension margins generally scales well.
* The `singularBeta` margin is known to have non-linear scaling; pushing its value closer to the TT value tends to scale better at longer time controls.
* Recursive search calls (e.g., `qsearch` from main search) require using the correct search stack frame (`ss + 1` only after a move is made).
* Selective extensions are often better than universal ones; even if a condition for an extension is rarely met, rejecting it can be critical.
* Avoid over-extending PV nodes; patches that extend PV nodes too aggressively are likely to perform poorly.
* Changes to search extensions (e.g., singular, multi-PV) have very strong scaling effects; a patch good at shallow depths can be detrimental in deep searches.
* Be extremely cautious with changes that reduce the frequency of multi-move extensions; this is a common trap for deceptively good-looking patches at short time controls.
* When modifying a heuristic affecting extensions, consider adding a compensating value to keep the average effect similar and normalize scaling behavior.
* A valid search extension strategy is to search deeper on sacrificial moves (negative SEE), potentially combined with other signals like LMR failure.
* The `depth` parameter in quiescence search is not used by the move picker for scoring captures or evasions.
* Ensure that the search depth cannot be floored at 1 (e.g., `max(1, depth - R)`), as this would prevent entry into quiescence search (depths <= 0).
* If a search modification changes the effective search depth of a node (e.g., through a new extension), the depth stored in the corresponding transposition table entry must also be updated to reflect this change accurately.
* When a search extension is applied, it's important to guard against propagating unproven mate scores from the deeper search. A non-PV search returning a mate score can incorrectly propagate to a PV node, flagging a short or incorrect PV at the root.
* Extending the search when there is only one legal move is a valid idea to explore, even if standard singular extension conditions are not met.
* The search has "gigabloated" extensions; modifying extension logic or activation margins is a powerful way to alter search behavior.
* A search heuristic that is beneficial globally may not be beneficial when restricted to a specific condition (e.g., only at high depths), and vice-versa.
4. **Quiescence Search (Qsearch)**
* Quiescence search has a defined minimum depth limit (e.g., `DEPTH_QS_RECAPTURES` at -5 plies), which is a key parameter controlling its termination.
* Quiescence search uses a partial insertion sort for ordering moves, indicating that highly optimized, simple sorting methods are preferred in performance-critical code paths.
* Be extremely cautious when modifying quiescence search (`qsearch`); it has sensitive and non-obvious pre- and post-conditions, and even logical changes can introduce instability.
* Replacing NNUE with classical evaluation in quiescence search (`depth < 0`) or at very shallow search depths (`depth <= 3`) results in a massive performance loss.
* The quiescence search assumes it is entered from a position where the side to move is not in check; modifications must respect this.
* Applying a qsearch directly to a static evaluation is a known technique to significantly improve its quality.
* A common pitfall is calling `qsearch` under conditions where it will be a no-op (e.g., `staticEval >= beta`).
* When evaluating a node for the first time (no TT hit), the search first calls `evaluate()` for a static score, then `qsearch` or recursive `search`. Calling `qsearch` on this initial static eval without safeguards can lead to double-evaluating every new node.
* The bounds of a quiescence search (`qsearch`) are a parameter for experimentation; a non-standard window like `(alpha - 1, alpha + 1)` could be used.
* Since moves in quiescence search (`qsearch`) are now partially sorted, it may be beneficial to update move statistics (like history heuristics) within `qsearch`.
* There is a known inconsistency between the main search (full evaluation) and quiescence search (simpler static evaluation), which can cause issues for features like null-move search.
* The `qsearch` implementation is considered "ancient" and is a promising area for improvements that deviate from canonical qsearch algorithms.
* A key limitation in the current `qsearch` is a termination rule that restricts it to recaptures on the last active square, preventing it from solving certain deep tactical sequences.
* Extending quiescence search by adding quiet moves, such as killer moves, is a known, advanced technique that fundamentally changes qsearch from purely tactical.
* In `qsearch`, it is acceptable to approximate the static evaluation after a null move by negating the previous evaluation, but this is detrimental in the main `search` function.
* A potential optimization in `qsearch` is to generate only recaptures directly when below `depth_qs_recaptures`, avoiding overhead.
* Heuristics successful in the main search may also be applicable to `qsearch`, but often require adaptation due to different trade-offs.
* In `qsearch`, pruning heuristics should be ordered from the computationally cheapest to the most expensive to maximize efficiency.
* In `qsearch`, the `depth` variable is negative; pruning heuristics should account for this, pruning less aggressively at higher (less negative) depths.
* A key exception in `qsearch` is that transposition table moves can be quiet moves, so logic cannot assume all moves considered are captures or promotions.
* A heuristic that works in `qsearch` may fail in the main search because the main search requires higher precision.
* The behavior of quiescence search is controlled by the depth value; making the depth more negative can trigger more aggressive pruning.
* The `qsearch` (`qsearch`) assumes it will not be called on a position where the side-to-move is in check, and calling it from contexts that bypass main search checks can lead to crashes.
* Do not use static evaluations from parent states (`(ss-1)->staticEval`) that might be `VALUE_NONE` (e.g., due to the parent being in check), as negating it results in `-VALUE_NONE`, which fails assertions.
* Reusing a parent's static evaluation after a null move by simple negation (`-(ss-1)->staticEval`) is incorrect because evaluation terms like `optimism` are side-to-move dependent.
* `qsearch` uses specific depth constants like `DEPTH_QS_RECAPTURES` (`-5`) in conditional logic; a null move search entering `qsearch` with `depth=0` will satisfy `depth > DEPTH_QS_RECAPTURES`.
* `qsearch` does not have a hard depth limit; it continues as long as there are recaptures or *any* type of transposition table move available.
* `qsearch` can be entered from the main search loop if depth drops to zero or below due to reductions, bypassing the rest of the main loop's logic for that move.
* In `qsearch`, `depth == 0` includes quiet checks, while progressively more negative depths restrict the search to captures and then only recaptures.
* When in check, `qsearch` will search all legal evasions (both quiet and tactical), regardless of the current (potentially negative) search depth.
* Pruning in `qsearch` is disabled when `bestValue` is a mate score (i.e., less than `-VALUE_MATE`) to prevent pruning a move that could avert mate.
* The pruning logic in `qsearch` may contain outdated or illogical code inherited from older versions.
* When pruning a move in `qsearch`, avoid code that artificially raises `bestValue` (e.g., to `futilityBase`), as `bestValue` should reflect a searched move.
* `qsearch` does not probe tablebases and is capped by static evaluation for new nodes.
* `qsearch` uses negative depth values (e.g., `DEPTH_QS_CHECKS = 0`, `DEPTH_QS_NO_CHECKS = -1`) to control which types of moves are generated.
* `qsearch` does not generate all possible checks; "quiet check" generation is an optimization at `depth == DEPTH_QS_CHECKS`.
* Qsearch and the main search have different characteristics; TT entries generated in qsearch have very shallow depths and should be treated with caution when probed by the main search.
* The `SearchStack` (`ss`) is not fully initialized at each ply for performance; any stack values used in `qsearch` must be explicitly initialized within `qsearch` itself.
* Any mate score returned from qsearch is potentially unreliable because, when not in check, it doesn't generate quiet moves that could be a valid defense.
* A potential solution to unsound mates from qsearch is to prevent it from returning definitive mate scores by clamping the return value to just below a proven mate.
* Qsearch cannot prove true mates; if all moves in qsearch lead to a loss, it should return a very low score (e.g., `VALUE_TB_LOSS_IN_MAX_PLY + 1`) rather than a `mated_in()` score.
* Applying main search pruning techniques to qsearch often fails; directly porting history pruning from the main search and blending it into qsearch's futility pruning has historically "sucked."
* Quiet TT moves are always considered in qsearch, though they are still subject to the usual qsearch pruning heuristics.
* In qsearch, a mate score can only propagate up from a position that is already in check; attempting to add quiet move generation in qsearch to avoid a mate found via captures is logically flawed.
* In qsearch, raising alpha immediately based on a static evaluation is correct, even if the score is a proven loss, because if alpha is already a mate score, the goal is simply to find a *faster* mate.
* The ordering of stages in qsearch is critical; ensure all recaptures are generated and searched before other captures to avoid missing tactical sequences.
* Be extremely careful with control flow in qsearch; a condition like `if (eval >= beta)` can cause an immediate return, preventing subsequent intended logic from executing.
* Trying the TT move in qsearch is delicate; if it failed to cause a cutoff previously, it's likely to fail low, so some engines guard this by only trying the TT move if it has a non-negative SEE score.
* Variables like `bestValue` can have different semantics and represent different concepts in `qsearch` versus the main `search` function.
* `qsearch` can be called on the root node, particularly through razoring, a rare but critical edge case that must be handled correctly as there is no valid previous move on the stack.
* Qsearch can have different characteristics than the main search. A surprisingly large number of qsearch PV-nodes can occur with a search window of exactly 1 (`beta - alpha == 1`).
* Quiescence search is not limited to captures; it also processes specific quiet moves like checks (at depth 0), check evasions, and moves from the transposition table. These quiet moves can be pruned using continuation histories.
* Quiescence search can benefit from its own dedicated history tables, separate from the main search. An "approximated depth" based on `seldepth - ply` can be used to index these q-search specific tables.
* When manipulating scores within `qsearch`, especially for PV-nodes, special care must be taken to not alter proven mate scores or tablebase win scores. A check like `abs(bestValue) >= VALUE_TB_WIN_IN_MAX_PLY` is robust.
5. **Terminal Node Handling**
* The search handles terminal nodes by first checking if any legal moves exist for a position.
* If zero legal moves, the search determines if the king is in check to distinguish checkmate (mate score) from stalemate (draw score).
* The main static evaluation function is not called for terminal nodes; a definitive score (mate or draw) is returned directly.
* A position with no legal moves is a stalemate (`VALUE_DRAW`) unless the king is in check, in which case it is mate (`VALUE_MATE`).
* Helper functions like `Position::is_draw()` are intentionally designed to ignore stalemates because their call sites in the search handle mate/stalemate detection separately.
6. **Position State and Move/Undo Operations**
* The `ply` variable indicates the current search depth from the root.
* The `StateInfo` stack correctly handles restoring the `rule50` (halfmove clock) counter after `do_move`/`undo_move`.
* The `pliesFromNull` variable in `StateInfo` tracks plies since the last null move, used to control null-move pruning.
* Be extremely careful with `do_move()` and `undo_move()` calls inside the search to check properties of a future position; this pattern is very slow and should be avoided.
* When programmatically making and unmaking moves, use the correct inverse functions (e.g., `undo_null_move()` for `do_null_move()`).
* To access the move that led to the current search node from within the `search` function, use `(ss-1)->currentMove`, where `ss` is the pointer to the current `SearchStack` entry.
* The `Position` object (`pos`) passed through the search is the same as `rootPos` and is modified; to access initial root data (like starting ply), cache it in the `Thread` structure at the beginning of the search.
* To programmatically create a move from its components, use the `make<MOVE_TYPE>(from_sq, to_sq, pieceType)` function template (e.g., `make<NORMAL>(...)`).
* The `move` variable is only valid within the context of the move generation loop; accessing it before this loop leads to unstable behavior.
* When implementing changes involving `do_move` and `undo_move`, it is a common and critical pitfall to forget to correctly handle all special move types, particularly promotions.
7. **Heuristic Design and Application**
* The effectiveness of a search heuristic or pruning margin can be highly dependent on the search context; consider making thresholds dynamic based on depth or material.
* To make a heuristic active only in the early stages of a search, conditioning it on `thisThread->rootDepth` is often more appropriate than `thisThread->nodes`.
* A viable search idea is to apply heuristics probabilistically rather than deterministically (e.g., a "probabilistic reduction increase").
* A good pattern for creating new heuristics is to combine a specific tactical event (e.g., a contact check or capture) with a statistical measure (e.g., a high history score) to trigger an action like an extension.
* Search heuristics can be successfully scaled based on the number of moves played in the game (`moveCount`), making them more or less aggressive in different phases.
* For depth-dependent logic with a floor and ceiling, prefer `std::clamp(value, min, max)` for readability.
* A complex, semantically clear metric like "distance from PV" can be less effective than a simpler, more pragmatic heuristic like checking the reduction count (e.g., `r < -1`).
* Heuristics based on reduction counts (e.g., `r < -1`) can be a powerful proxy for "value lost relative to the PV."
* A common pattern for improving a search heuristic is to rebalance its conditions, making one stricter while making another more permissive.
* The logic for "countermove based pruning" is not based on the simple countermove heuristic but also incorporates more complex "followup continuation history."
* There can be strong correlations or redundancies between different search heuristics (e.g., `distanceFromPv` and `r`).
* A rarely-triggered condition can still be critically important and resist simplification; for example, a `statScore` check in null-move pruning might be essential even if rarely hit.
* Explore simple, concrete positional features to guide search decisions (e.g., bonus for creating an open file or gaining tempo on queen).
* The search struggles to correctly identify and evaluate fortresses; patches attempting explicit fortress detection have historically failed.
* The value of a highly accurate evaluation is questionable for nodes that are likely to be futility pruned, as their scores primarily influence pruning decisions.
* Predicting the node cost of searching a branch deeper is extremely difficult; it is easier to predict the magnitude of the *change in evaluation*.
* Test stricter versions of existing pruning; for example, testing for strictly positive SEE (`see > 0`) instead of non-negative (`see >= 0`).
* Many search heuristics (reductions, pruning) are depth-dependent, often disabled or curtailed at high depths for stability.
* A viable strategy for introducing a new or risky heuristic is to limit its application to specific, well-defined scenarios (e.g., only with tablebase wins or mate scores).
* Search heuristics that are currently simple functions of depth (LMR, pruning) are promising areas for improvement.
* A simple, general condition can often outperform a more complex and specific one, even if the latter seems more logical.
* Search ideas that failed in the past are worth revisiting, as changes to other parts of the engine might make them viable again.
* Avoid scaling search parameters by `rootDepth`; instead, scale by current ply or depth.
* The statistical properties of nodes change dramatically after pruning stages; heuristics must be calibrated for the specific stage.
* When creating new pruning or extension formulas, be extremely careful about edge cases that violate search logic (e.g., `beta` must not be lower than original `beta`).
* When developing a patch that modifies a specific search heuristic, build it on top of other recent, related changes due to likely interactions.
* Prioritize general search improvement and maximizing TT size over complex on-the-fly learning schemes.
* Search parameters can be dynamic functions of game state (e.g., `rootDepth`, remaining time).
* Aspiration windows can be dynamic and asymmetric (e.g., wider at high evaluations).
* Search development requires balancing patches that expand the search (increasing nodes) with those that contract it to maintain efficiency.
* Search algorithm changes can perform differently on various position types (sharp vs. quiet).
* The 50-move rule is scored as a draw during search, but only under specific conditions (e.g., "strictly after the root") to avoid premature draw claims.
* The `counterMoves` heuristic is expected to generate many illegal moves; functions using its output must perform legality validation.
* Seemingly simple changes to pruning parameters can have complex and unexpected negative side effects.
* Interaction effects between different search components are common and subtle.
* Counter-intuitive changes can sometimes work, potentially by preventing search explosions or introducing beneficial noise.
* Old ideas can become effective again after other engine parts change.
* Isolate the core of a new idea by testing simpler versions without all conditions.
* In lost positions, it can be beneficial to select moves that complicate the game or push towards the 50-move rule.
* Search heuristics are sometimes adjusted in response to poor performance in high-profile games.
* A promising idea is to make history tables more context-aware by indexing them with the king's position (using a compressed representation).
* The same history heuristic is used in multiple, separate parts of the search, suggesting a potential for unification into a single, more principled formula.
* When implementing depth-dependent formulas, pay special attention to their behavior at shallow depths like `depth = 1`.
* To make the engine stronger at very short time controls, consider aggressive changes that reduce search overhead (e.g., removing history-based pruning).
* The `sigmoid()` utility function must not be called with a final parameter of 0, as it causes a division-by-zero error that can lead to a guaranteed crash.
* Search explosions, where the engine gets stuck in a loop of extensions and fails to deepen, are critical bugs. Patches fixing these are valuable even without direct Elo gain.
* To prevent search explosions, either proactively add guards before every recursive call or reactively check the state after a recursive call returns (e.g., checking if the previous call was a double extension).
* When implementing a new search feature that might cause instability, apply guarding logic similar to that used in established, stable parts of the search (e.g., singular extensions).
* A known search pathology is the evaluation failing to increase with depth in certain positions, indicating a search space exploration problem, not an evaluation bug.
* A low `seldepth` relative to the main search `depth` in a complex tactical position can be a symptom of a search pathology, where the engine fails to explore critical lines due to overly aggressive pruning or insufficient extensions.
* In quiet or drawish endgame positions, a constant evaluation score across increasing depths is expected behavior and does not necessarily indicate a "stuck" search, especially if `seldepth` continues to increase.
* Search logic should be robust without relying on tablebases, as tablebases can mask underlying search pathologies that only surface when they are not present.
* There is no logical reason to continue searching a branch after a tablebase probe returns a decisive result like a draw (when not at the root), as no better outcome can be found.
* A powerful but complex patch type involves fundamentally reshaping the search tree, e.g., reducing average depth in favor of searching a few promising lines much deeper via extensions.
* A more restrictive or conservative search heuristic can be preferable to a more aggressive one; even if it costs a small amount of ELO, a robust heuristic that avoids pathological behavior is often better.
* A complex feature with many lines of code for only a few Elo points (like ProbCut) is a good target for replacement by a simpler, more elegant mechanism.
* Simplifying search logic is a valid development goal, e.g., removing LMR dependencies on `statScore` if other changes make it redundant.
* For clarity and better organization, consider breaking down search parameters into logical sections like "early pruning," "LMP," "extensions + LMR," and "time management."
* Don't be afraid to test unconventional or seemingly "bizarre" ideas for search heuristics; they can sometimes lead to breakthroughs.
* The `delta` variable (`beta - alpha`) can become extremely large (e.g., `VALUE_INFINITE - (-VALUE_INFINITE)`), which can cause overflow or unexpected behavior in formulas that use it.
* Search bounds (alpha and beta) can be negative; patches that manipulate these bounds must handle negative values correctly to avoid creating invalid states like `beta < alpha`.
* A critical and common search bug to avoid is confusing fail-low nodes (value <= alpha) with fail-high nodes (value >= beta); ensure your logic correctly distinguishes between these outcomes.
* Some heuristics commented as providing `~0 Elo` gain cannot be removed without causing a performance loss, indicating complex, non-obvious interactions.
* Historically, attempts to create complex heuristics that "normalize" search behavior across different time controls have a very high failure rate; simpler, fixed heuristics often prove more robust.
* Search heuristics that work in other engines may fail in Stockfish due to its highly optimized existing components.
* Search development is often cyclical; heuristics simplified or removed in the past can become effective again as other parts of the engine evolve.
* Tactical oversights are rarely caused by a single pruning method but by a harmful interaction between several (e.g., multiple LMRs followed by NMP cutting off refutation).
* Search heuristics are highly interdependent and finely tuned; porting individual features in isolation can cause severe performance drops.
* A heuristic that is generally beneficial may be harmful in specific contexts.
* Search heuristics can have non-linear effects that do not scale predictably with time control or search depth.
* Search components are highly interactive; a change to one heuristic can have complex and non-obvious effects on others.
* Statistical information (e.g., `stat bonus`) is used to directly modulate pruning and reductions; worse stats lead to narrower/shallower search.
* Many "obvious" simplifications to complex search logic have likely been attempted and failed; complexity often exists to handle subtle edge cases.
* Avoid adding new features to a heuristic that are already captured by existing, stronger ones (e.g., small penalty for sign mismatch if large absolute difference penalty exists).
* Using the sign of evaluation components as a signal in search heuristics has historically not been successful; magnitude of difference is generally more robust.
* The engine's tendency to seek or avoid draws from repetitions or the 50-move rule can be controlled by adjusting `value_draw`, a distinct mechanism from static evaluation.
* Position complexity can be used to dynamically adjust other search parameters, such as the number of moves exempt from futility pruning.
* The decision to switch between NNUE and classical evaluation is a carefully balanced heuristic, depending on game phase, material imbalance, and shuffling detection.
* A key condition for using classical evaluation is a high piece-square table (PSQ) imbalance, indicating a decisive advantage where less precise classical eval is sufficient.
* In search heuristics, "material" often specifically refers to `pos.non_pawn_material()`.
* The `psq` score represents the *difference* between White's and Black's piece-square values, not the total value of pieces.
* When adding a condition with a constant (e.g., `pos.rule50_count() > 30`), understand the typical range and distribution of that variable's value.
* The search benefits from a large transposition table even when it is not close to being full.
* Extensions can have complex interactions with move ordering heuristics (e.g., quiet TT move extension tied to history heuristic threshold).
* The `optimism` term, which directly modifies the evaluation score, is adjusted by a hybrid complexity metric (blending NNUE complexity with NNUE/classical eval difference).
* The `searchAgainCounter` is intended to make the engine re-search the same depth, particularly when it's near the end of its allocated time.
* A single patch must represent a single, atomic logical change. Combining multiple, unrelated search ideas into one commit is a critical mistake.
* When a root move fails high, the engine performs a re-search at a reduced depth to quickly verify the new score, which is why reported depth can appear to go backward.
* Patches modifying classical evaluation terms can still provide gains, but success often hinges on correctly adjusting the scaling formula that integrates the classical score with NNUE.
* Improving cache locality in the TT is a major area for optimization, as `TT::probe` is a significant bottleneck.
* Be extremely cautious with aggressive pruning ideas near the root; pruning the second-best move or its direct replies is high-risk.
* The engine used to have a 2-fold repetition draw rule but it was removed; it was Elo-neutral and removed to make the engine more useful for analyzing standard human games (3-fold rule).
* Simple threat detection heuristics effective in other engines may not be effective in Stockfish due to stronger history heuristics and deeper search.
* A general heuristic can have unexpected (and sometimes beneficial) side effects on specific move types, such as evasions for particular piece types.
* For performance, it can be acceptable to omit logically "correct" but expensive checks if the edge case is rare and has a benign, predictable effect.
* Be aware of logic that carries state across the move loop; a boolean variable might be set based on the first move and then then used for subsequent moves.
* Time management formulas can be unstable if they divide by `timeLeft` (near zero) or `movestogo` (near one).
* Instamoves should only occur when the engine is truly out of time, not when remaining time is simply less than `moveOverhead`.
* `maximumTime` can be less than `optimumTime` because `maximumTime` subtracts `moveOverhead` while `optimumTime` does not.
* Search parameters can take on counter-intuitive values that work in practice (e.g., a `stat_bonus` becoming a penalty) because their effect is relative to the entire set of tuned parameters.
* Heuristics tuned for standard chess may not generalize well to variants like Fischer Random Chess (FRC); a "trapped bishop" penalty might be too aggressive in FRC.
* A persistent Transposition Table (TT) is superior to a traditional static opening book as the engine can continue searching and update the "book."
* A persistent TT or book can guide search in positions where evaluation is known to be flawed.
* Disabling `Syzygy50MoveRule` is a valid strategy to find tablebase wins earlier.
* An opening book integrated directly into the search is more powerful than a simple polyglot book.
* The `SyzygyProbeDepth` parameter is crucial to defer expensive tablebase probes until deeper in the search.
* The classical pawn hash table is almost entirely unused in the main search due to NNUE strength.
* MultiPV is implemented by iteratively re-searching the root after removing the previously found best move.
* MultiPV-like search (broadening the root search) is generally too slow.
* Be extremely cautious with pruning at the root node, as it permanently discards moves.
* When designing heuristics for `go mate`, consider disabling pruning techniques like SEE pruning.
* The contempt feature is historically known to be ineffective and a likely dead-end for new patches.
* There is a specific constant, `VALUE_KNOWN_WIN`, for decisive but non-mating advantages, distinct from mate values.
* You do not need to understand the entire search to contribute; focusing on one component is a valid approach.
* Designing search patches specifically to solve famous tactical puzzles is a bad practice; solutions rarely generalize well.
* When needing information about threatened squares, it is more robust to use the full attack map rather than inferring from a bitboard of just threatened pieces.
* There are very subtle and deep interactions between search components; a null move can alter the evaluation context for a subsequent quiescence search.
* A general change to a core heuristic, like mixing the alpha bound into history scores, can have severe, unintended consequences in specific search states.
* Enabling move pruning at the root node is extremely dangerous and can lead to severe search blindness; it must be exceptionally conservative if ever implemented.
* A patch that significantly harms mate-finding capabilities is often treated as a bug and is likely to be reverted.
* Be cautious of unintended side effects when passing core state objects by reference; a search function might modify the `rootPosition` object.
* When writing complex conditional logic, be aware that the condition might trigger far less frequently than anticipated.
* Do not rely on the compiler to optimize away unclear or potentially buggy code; prioritize clarity and correctness.
* Designing patches to fix a single, specific position is a common pitfall that rarely generalizes well; successful patches should address a broader search principle.
* Focus expensive logic on critical parts of the tree, such as nodes at a very low distance from the Principal Variation.
* Use the engine's internal PRNG for deterministic behavior for any logic requiring random numbers.
* The `ss->staticEval` value is always from the perspective of the side to move; `(ss-1)->staticEval` is from the opponent's perspective with an opposite sign.
* Avoid complex parallel search algorithms like PV-splitting; LazySMP is the established and superior method.
* Data initialized only at the root of the search can still have a significant impact, particularly at shallow depths or in short time controls.
* The search has evolved to be highly complex and counter-intuitive; what seems like a "reduction" can lead to an extension, and moves that skip LMR can still be reduced later.
* When using depth-indexed arrays for parameters, ensure accesses are properly bounded, as search depth can exceed the array's intended maximum size.
* Be cautious when refactoring search logic into new functions; many critical parameters are local variables within the main search functions.
* Question the purpose of existing variables (e.g., `maxNextDepth`); some may be remnants of older logic.
* Seemingly independent search parameters can be implicitly linked; a change to one might break the intended logic of another.
* The idiomatic way to add a bonus to a `StatsEntry` object is by using the overloaded `<<` operator.
* Avoid hard-coding special logic for specific endgames, as the general search and evaluation are expected to handle them.
* Creating opponent-specific parameter tunes is avoided because it makes search parameters brittle and interconnected.
* In PV-nodes, search logic can be optimized based on the static evaluation, as the primary goal is to find a move that improves upon the current `bestValue`.
* Logic in Late Move Reductions (LMR) is known to have a high rate of branch misprediction, making it a prime candidate for simplification.
* A common technique is to add guards for critical nodes (e.g., `!PvNode || depth < N`) to a pruning or reduction formula.
* A common pitfall is making incorrect assumptions about what a data structure contains, such as assuming a `threatenedPieces` bitboard also contains information about the squares those pieces are attacked on.
* A general change to a core heuristic, like mixing the alpha bound into history scores, can have severe, unintended consequences in specific search states, such as mate-finding.
* Be aware of historical, difficult-to-solve issues like "stuck search"; patches that manipulate the search window or reductions can inadvertently worsen these.
* When implementing conditional logic, carefully check for redundant conditions.
* To make a search parameter dynamic based on the overall search depth, the variable `thisThread->previousDepth` can be used.
* Search heuristic parameters do not have to be static; they can be made dynamic by depending on variables like search depth or material count.
* When pruning based on a counter (e.g., `quietCheckEvasions`), you can rely on the counter's specific increment conditions to simplify the pruning logic.
* Calculating complex positional properties like king mobility from scratch is very expensive; such checks are more feasible inside functions like classical evaluation.
* A concrete method to check if a king has legal moves involves generating the king's attack bitboard and removing squares occupied by its own pieces or attacked by the opponent.
* Historically, many attempts to add explicit logic for "no legal king moves" (in non-check situations) to detect zugzwang have failed to show improvement.
* Effective fortress detection is a very deep problem; the search must identify the potential for a fortress 20-30 moves in advance.
* The Graph History Interaction (GHI) problem is addressed by decaying the evaluation for non-relevant moves and using distinct hash slots.
* The logic for assigning statistical bonuses should be consistent with the actual search depth performed for a move.
* A single line of code can have multiple, intertwined effects, such as simultaneously preventing a re-search, correcting statistical bonus logic, and applying an extension.
* For logical consistency, if a heuristic is applied based on the results of a reduced-depth search, consider if it should also be applied based on full-depth search results.
* History heuristics should only be updated on statistically significant events; fail-lows are the vast majority of outcomes and updating on them would skew statistics negatively.
* The `rootDepth` variable is critical for UCI communication and internal state; it must be monotonically increasing and should not be modified directly.
* Asymmetric search modifications between threads, such as reducing depth for even-numbered threads, are a valid technique to decorrelate work in parallel search.
* Understand the precise context of a search modification; a change to a search *after* a failed LMR is fundamentally different from changing the initial LMR reduction.
* When using the `bestValue` variable from a search, always account for the fact that it can be negative, as using it directly in depth or reduction calculations can incorrectly result in a `newDepth` of zero or less.
* Search heuristics can be sensitive to the material balance on the board; a static parameter or heuristic may fail if it doesn't adapt to unusual material counts.
* Integer Arithmetic for Heuristics: Floating-point concepts for heuristics must be translated into pure integer arithmetic, often using scaling factors (e.g., `3 * depth / 2`) or more complex formulas to achieve the desired effect without using floats, as they are slow in the hot path.
* StateInfo Management: The search maintains a history of board states via a linked list of `StateInfo` objects (`st->previous`), which is crucial for reasoning about state-dependent features like incremental calculations and ensures proper state updates.
* Check Evasions and Counter-Checks: Moves that are both evasions from check and deliver a counter-check are rare but powerful, making them strong candidates for extensions or reduced reductions.
* `psq` vs. `psq_stm` Distinction: Be aware that `Position::psq` is a color-absolute piece-square table value, while the search requires a color-relative evaluation, necessitating conversion based on the side to move (e.g., `(sideToMove == WHITE ? 1 : -1) * eg_value(psq)`).
* Strict `StateInfo` Update Ordering: The `StateInfo` stack (`ss`) must be updated in a precise order; for example, `ss->currentMove` must be updated *after* a singular extension search to maintain search state integrity.
* NNUE Accumulator Hinting: Calling `Eval::NNUE::hint_common_parent_position()` is crucial for efficiency, as it pre-populates the accumulator, significantly speeding up evaluation of multiple child nodes.
* `Score` Type for MG/EG Values: The `Score` type is a packed data structure containing both 16-bit midgame and endgame values; use `mg_value()` and `eg_value()` helpers to correctly extract scores.
* Cycle Detection in `search`: Repetition/cycle detection logic belongs in the main `search` function, not `qsearch`, due to its complexity and computational expense in the high-node-count quiescence search.
* Mate Distance Pruning in `search`: The main `search` function contains mate distance pruning logic that is absent in `qsearch`; calling `qsearch` directly from the move loop bypasses this critical check.
* Boolean Logic and Chess Rules: When modifying boolean logic, remember that chess rules create implicit dependencies (e.g., a move cannot be both a promotion and castling), so seemingly different logical expressions may be functionally identical.
* MovePicker Stage Awareness: The `MovePicker` processes moves in distinct, ordered stages (e.g., good captures, then quiets); information generated in later stages (like `threatened` bitboard) is not available in earlier ones.
* `MOVE_NULL` Consequences: A `MOVE_NULL` on the search stack can affect systems like countermove or continuation history updates, as they might incorrectly process an invalid "previous square" (`b1`).
* Cautious Selectivity/Extensions: Be extremely cautious with changes that increase search selectivity or extensions (e.g., widening margins), as they often provide gains at shallow depths but fail at deeper searches due to poor scaling.
* Quiet Move Scoring Logic: The scoring for quiet moves is a complex heuristic combining multiple continuation history tables with carefully chosen, non-arbitrary weights.
* PV-Node Extension Cost: Extending PV-nodes is much more costly than extending non-PV nodes, and aggressive PV-node extensions have historically failed.
* Aspiration Window `delta`: The `delta` value in aspiration windows is surprisingly small (e.g., 70 centipawns initially), not on the same order of magnitude as `VALUE_KNOWN_WIN`.
* "Tropism" (King Flank Attacks): The `kingFlank` attacks concept involves counting squares attacked by the opponent in the king's flank, with a quadratic scaling component for multiply-attacked squares.
* Phase/Progress Dependent Parameters: Making search parameters dependent on game phase (`is_endgame`) or search progress (`completedDepth`) to balance short-term gains vs. long-term scalability often creates "resource blackholes" and is generally discouraged.
* History Update Formula: The standard history update formula (`entry += bonus - entry * abs(bonus) / D`) is not a simple running average; it ages old values towards zero while allowing saturation at `+D` and `-D`.
* TT Cutoff and Negative Extensions: When a TT entry's value is high (`ttValue >= beta`), applying a negative extension (reduction) is valid, but immediately continuing the loop (`continue`) is poor practice as it aggressively prunes potentially good moves.
* Guard Against Zero/Negative Depth: When applying reductions, always guard against the resulting search depth becoming zero or negative (e.g., `depth > 0`) to prevent critical implementation errors.
* Continuation History with Killer Moves: The combination of continuation history (`contHist`) with killer moves is effective because it confirms a move's strength in two separate contexts.
* Less Aggressive Reductions for Shallow TT Entries: TT entries from shallower searches (`tte->depth() < currentDepth`) are less reliable; heuristics should be less aggressive (e.g., smaller reductions) when using information from them.
* `ss->currentMove` Can Be `MOVE_NULL`: Be aware that `ss->currentMove` may be `MOVE_NULL` in certain code paths (e.g., after a TT cutoff); functions using it must handle this case.
* Avoid `to_sq()` on `MOVE_NULL`: Never call functions like `to_sq()` on `MOVE_NULL`; any logic processing moves must have guards like `is_ok(move)` or `move != MOVE_NULL`.
* Subtle Differences in Logical Checks: Seemingly equivalent logical checks (e.g., `prevSq != SQ_NONE` vs. `is_ok((ss-1)->currentMove)`) are not always interchangeable and can have subtle behavioral or performance differences.
* Singularity Extension Sensitivity: Parameters controlling core search features like singularity extensions (`singularDepth`) are extremely sensitive, with even a change of 1 having significant impact.
* Root Node `currentMove`: At the root node, the previous stack frame (`ss-1`) has `currentMove` set to `MOVE_NONE`; code accessing `(ss-1)->currentMove` must handle this special case.
* `Root` Node is a `PvNode`: The search function is templated by `NodeType` (`Root`, `PV`, `NonPV`), and a `Root` node is considered a `PvNode` (`constexpr bool PvNode = nodeType != NonPV`), which is critical for PV-specific logic.
* PV vs. Non-PV Tuning: A common pattern for tuning search parameters is to differentiate behavior in PV vs. non-PV nodes (e.g., using `!PvNode` as a multiplier or condition).
* Null Move State Implications: After a null move, the parent node's `moveCount` is not incremented, and a null move cannot become a killer move; conditional logic checking these properties in a null move context is likely redundant.
* `thisThread->completedDepth` as Time Control Proxy: `thisThread->completedDepth` is a fragile "hack" used as a proxy for time control to adjust search behavior; expanding its use to new areas is generally discouraged.
* Extensions and Time Control Scaling: The extensions logic is a primary area where time-control scaling behavior is implemented; changes here are highly likely to have performance impacts that vary significantly with search time.
* Historical Dummy Squares: Historically, `MOVE_NONE` and `MOVE_NULL` were sometimes mapped to dummy squares (e.g., A1); if changing this, be aware that existing code may implicitly rely on `prevSq` always being a valid square.
* Avoiding Re-searches in IIR: Using re-searches within Internal Iterative Reductions (IIR) has been found to not scale well with longer time controls and should likely be avoided.
* `seldepth` Definition: `seldepth` is defined by the depth of Principal Variation (PV) nodes; LMR and double extensions primarily apply to non-PV nodes and thus do not directly increase reported `seldepth`.
* PV Node Extensions: PV nodes can be extended via `doDeeperSearch` and `doEvenDeeperSearch` if a reduced-depth search returns a value significantly better than alpha, allowing `seldepth` to grow.
* `singularBeta` and `ttValue`: When using `singularBeta`, be aware of its relationship to `ttValue`; a condition like `singularBeta >= beta - 4` is safe because the depth-dependent term guarantees `ttValue >= beta`.
* Non-PV Null Window Context: A critical mistake is forgetting that non-PV nodes (`!PvNode`) are searched with a null window (`beta == alpha + 1`); `ttValue < beta` is functionally equivalent to `ttValue <= alpha` in non-PV nodes.
* `qsearch` After TT Cutoff: Avoid calling `qsearch` immediately after a TT hit suggests a cutoff; `qsearch` also probes the TT and will likely hit the same entry, resulting in a non-functional slowdown.
* Integer Arithmetic in Null Windows: In null-window contexts, `(alpha + beta) / 2` simplifies to `alpha` due to integer division.
* Hard Prune vs. Negative Extension: There is a crucial difference between a hard prune (e.g., `singularBeta >= beta` leading to a return) and a negative extension (e.g., `ttValue >= beta` causing a reduction).
* PV-Node and Cut-Node Mutually Exclusive: A node cannot be both a PV-node and a cut-node, allowing for logical simplifications (e.g., `PvNode && !cutNode` is identical to `PvNode`).
* Integer Division Refactoring: Be extremely cautious when refactoring formulas involving integer division; the distributive property does not apply (`(A + B) / C` is not equivalent to `(A / C) + B / C`).
* Stateful Variables Across Move Loop: Be aware of stateful variables that persist across the move loop within a single node search (e.g., `singularQuietLMR`), as they can affect subsequent moves.
* Move Generation Stage and Handling: The stage where a move is generated (e.g., captures vs. quiets) is critical to its handling in search; underpromotion captures, for instance, have implications for ordering and reductions.
* Recapture Search Optimization: Drastically reducing search on simple recaptures has historically failed; the time spent populates the TT and maintains search continuity.
* Node Count and Pruning: Increased node count in a test often signifies "less pruning" rather than "more extensions."
* `rootDepth` / `completedDepth` Scaling: Scaling search parameters (like extensions) based on `rootDepth` or `completedDepth` is a common idea that has historically failed to provide gains at longer time controls.
* Slow Integer Division: Integer division is a very slow CPU instruction; avoid dividing by a variable if possible, often favoring `if` statements to branch between division by a constant or no division.
* TT Access Guards: Any access to TT entry fields (e.g., `tte->depth()`) must be guarded by a preceding check for a TT hit (`ss->ttHit`) to prevent accessing invalid memory.
* Null Move Stack State: After a null move, `ss-1` refers to the null move itself, and `ss-2` refers to the last move made by the *current* side; looking at the wrong stack entry is a common mistake.
* Integer Division Truncation: Be extremely careful with how integer division truncates towards zero; a complex formula might be non-functional if the numerator rarely exceeds the divisor.
* Clamping `complexity` Values: When modifying values that feed into complex formulas like time management, consider edge cases (e.g., `complexity` becoming negative) and clamp values if necessary.
* Depth Variable Distinction: Distinguish between `ss->ply` (search depth from current root), `rootDepth` (target depth for current iteration), and `pos.game_ply()` (absolute game ply count).
* Initializing `Stack` Fields: When manually initiating a recursive search call, it is critical to initialize all necessary `Stack` fields (e.g., `Stack::currentMove`, `Stack::continuationHistory`) to prevent crashes.
* Futility Pruning in PV-Nodes: Aggressive pruning techniques like futility pruning on captures must often be disabled in PV-nodes (`!PvNode` check) to avoid pruning the best line.
* Runtime Variables as Divisors: Avoid using runtime variables as divisors in performance-critical search code, as this forces slow integer division instructions.
* Templating Performance Impact: Templating search functions or creating multiple specialized versions can paradoxically decrease performance due to increased executable size and instruction cache misses.
* `ttMove` Without `ttHit`: A transposition table move (`ttMove`) can exist without a full table hit (`ttHit`), but this is an exceptional state, typically only at the root.
* Integer Division vs. Bit-Shifting for Negatives: Integer division and bit-shifting behave differently for negative numbers; replacing division with bit-shifting for performance requires careful consideration of `depth`'s potential negative range in `qsearch`.
* Division by `depth` Expressions: Formulas involving division by `depth` expressions (e.g., `C / (k + depth)`) are highly risky; reductions can cause `depth` to become negative, leading to division-by-zero crashes, so guard with `depth > -k`.
* `moveCount` Indexing: The `moveCount` variable is incremented at the start of the move loop, so the first move processed corresponds to `moveCount == 1`, not `0`.
* TT Move and LMR: The principal variation (PV) move from the transposition table (`ttMove`) should not be subject to Late Move Reductions (LMR); applying reductions to the most promising move is a conceptual error.
* Initializing New Stack Frames: When calling the next ply of search (`search(ss+1)` or `qsearch(ss+1)`), critical state variables on the new stack frame (e.g., `ss->currentMove`, `ss->continuationHistory`) must be correctly initialized.
* Boolean Algebra Simplification: Applying boolean algebra can make complex conditional logic in search more readable and potentially easier for the compiler to optimize.
* Function Argument Semantics: Be aware of function argument semantics, especially for `out` parameters passed by reference/pointer; using such a variable in a condition before it's assigned is a critical bug.
* "Complexity" Metric Flavors: The "complexity" metric is not unified; different versions exist (classical eval, NNUE, TT hits) and are not always interchangeable.
* Ternary Operator vs. Bit-Shifting: A ternary operator (`condition ? a : b`) is not always compiled into the most efficient branchless code; explicitly using bit-shifting (e.g., `val >> (!improving)`) can sometimes force a more optimal instruction sequence.
* Move Count Pruning in `qsearch`: Move count pruning is also used in `qsearch`, particularly for pruning quiet check evasions.
* `std::clamp` Arguments: Carefully verify the arguments of standard library functions like `std::clamp(v, lo, hi)`, where `lo` is the fixed lower bound, to avoid impossible conditions.
* TT Handling of Fail-Lows: For nodes that fail low (score <= alpha), the transposition table stores `MOVE_NONE` as the best move.
* Search Parameter Cleanup: If a variable was a tunable search parameter but is no longer used, refactor it into a constant for code cleanup.
* Constant Depth Resets: When an algorithm temporarily changes search depth, restoring it to a small constant (e.g., 1 or 2) is a valid experimental idea, suggesting a shallow re-search.
* Search Code Modularity: Search code is not always modular; a change in one area (e.g., `qsearch`) may require updates to seemingly unrelated components (e.g., continuation histories).
* LMR Reduction Formula: LMRs are heavily tuned, but simple changes to the reduction formula based on a single heuristic like `statScore` can still be effective.
* Classical Eval vs. NNUE Thresholds: Modifying the activation thresholds for when to use classical evaluation instead of NNUE is a valid and impactful search patch.
* NMP Edge Cases: Modifying search depth reductions without considering edge cases like null move pruning (NMP) can lead to `qsearch` being called with very low depths and a `MOVE_NULL` context, breaking recapture logic.
* Aggressive NMP Formula: The null move reduction (NMP) formula is aggressive and can reduce depth by a large, variable amount (e.g., `depth - 10`), leading to immediate `qsearch` calls with highly negative depths.
* Negative Depth Values: Using negative depth values is a deliberate design choice to elegantly manage quiescence search extensions without passing extra state parameters.
* `optimism` Initialization: Be cautious when inserting new logic before the main `NNUE::evaluate` call; variables like `optimism` may not be initialized at that point.
* Counter-Intuitive Patches: Counter-intuitive or seemingly illogical patches can be successful (e.g., applying a *negative* stat bonus to a move that previously failed high in LMR).
* Convex Combination for Thresholds: When creating thresholds based on the alpha-beta window, use a convex combination (`(a * alpha + b * beta) / (a + b)`) rather than arbitrary denominators.
* Non-Obvious Interactions: Be aware of non-obvious interactions between search components; e.g., changing `qsearch` depth constants can unexpectedly affect `probcut` logic.
* Identifying Extensions: The condition `newDepth >= depth` inside the main search loop is a reliable way to identify moves that have received an extension.
* LMR Fail-High Stat Bonuses: Stat bonuses from `update_continuation_histories` for LMR fail-highs can be "reverted" later with a negative bonus to compensate for strength differences in zero-window vs. full-window searches.
* `ttDepth` in `qsearch`: The `ttDepth` stored in the TT for `qsearch` (e.g., `DEPTH_QS_CHECKS`) is distinct from the actual recursive search depth, which can be much lower.
* "Magic Number" Patches: Avoid patches that introduce numerous "magic number" constants and scattered, seemingly unrelated changes, as they are "hacky" and difficult to maintain.
* LMR Success and PV Re-search: A move's success in an LMR search (zero-window) does not guarantee its success in the subsequent full-depth PV re-search; patches can exploit this by adjusting bonuses.
* Mixing Search Depth with Static Eval: Avoid adjusting static evaluation based on current search `depth`, as evaluation can be reused at different depths or in `qsearch` where `depth` can be negative.
* Null Move Verification (NMV) Purpose: NMV is critical for correctness, not just Elo; it prevents the engine from being blind to zugzwang positions, even if its direct Elo contribution is neutral.
* Aggressive Pruning and Mate Finding: Aggressive pruning techniques, if not depth-limited, can cause the engine to fail to find mates in positions with high scores, as it may require a very deep search.
* MovePicker Stages and Contexts: Understand the `MovePicker`'s distinct stages and contexts (e.g., `MAIN_TT`, `EVASION`, `PROBCUT`); ProbCut, for instance, is not performed when the king is in check.
* Rarely Triggered Heuristics: A change will have little effect if it's in a code path that is executed very infrequently (e.g., only at the root node).
* Singular Extension Beta Values: `singularBeta` often involves subtracting a depth-scaled penalty from the TT value, with a smaller penalty for PV-nodes.
* Non-Linear Scaling Parameters: Some scaling parameters in the search are notoriously sensitive and can produce unexpected results at different time controls.
* Lazy SMP and TT Sharing: In Lazy SMP, threads share information via the transposition table; a later thread benefits from information left by the first.
* Reduction Heuristics: A reduction can be triggered when a move's score is significantly better than the best score found so far (e.g., `(value - bestValue) > (7 * (beta - alpha)) / 8`).
* Logical Contradictions in Conditions: A condition like `(value - bestValue) > ...` will always be false if it comes immediately after `bestValue = value;`, highlighting the importance of checking for static logical contradictions.
* Complex Patch Component Contribution: When creating a complex patch, ensure every logical condition can actually be met and contributes to the intended effect, rather than relying on other bundled changes.
* Large Depth Reductions After Fail-High: Applying large depth reductions after a fail-high scales poorly and can severely degrade performance in critical situations like mate finding.
* "Safety Net" Code: Certain search components, like verification search, may not provide direct Elo gain but handle specific engine weaknesses (e.g., Zugzwang); removing or weakening them can reintroduce old behaviors.
* Redundant Conditional Logic: Be cautious when simplifying or removing conditional logic that appears redundant; it might protect against rare but important edge cases.
* Eval Function Preconditions: Ensure `Eval::evaluate` is not called on a position in check; the search logic must guard against this with a `!pos.checkers()` check.
* Thread/MainThread State Persistence: State in `Thread` or `MainThread` objects can persist across different search commands within the same game, influencing heuristics that modify these values.
* Quiet Move Extension Failures: Simplifying or removing quiet move extensions has historically failed at longer time controls, suggesting their criticality for deep search stability.
* TT Entry Depth and Reductions: Logic depending on `tte->depth()` must account for how search reductions affect it; after a fail-high, depths are often reduced, making `tte->depth() >= depth` less selective.
* GHI Mitigation: The Graph History Interaction (GHI) problem is handled by downgrading potential mate scores from the TT when the 50-move rule count is high; simply disabling TT cutoffs is an Elo loss.
* Rationale for Downgraded TT Scores: A downgraded TT score is trusted because it's more likely to represent a legitimate, long mating sequence (with 50-move rule resets) than a false mate due to GHI.
* Preserving Functionality: A patch may be rejected if it degrades the engine's ability to handle specific, known problems (like zugzwangs), even if generally Elo-neutral.
* Indirect Benefits of Pruned Searches: Searches that are ultimately pruned (e.g., verification search confirming NMP cutoff) still populate the TT and update history heuristics.
* Clarity in Search Logic: Refactoring complex `if` conditions into separate boolean variables can improve code readability and maintainability.
* Cascading Effects of Shallow Changes: Changes at shallow search depths can have cascading effects on deeper parts of the search (e.g., early extensions leading to higher TT depths and more extensions later).
* Thread-Local State in Recursive Calls: Be extremely careful with thread-local data in recursive search calls; a flawed patch allowed inner calls to reset `thisThread->nmpMinPly`, breaking outer verification state.
* Logical Consistency of Heuristics: Maintain logical consistency; if a weaker move ordering heuristic is removed, it's illogical to keep a stronger, related one.
* Asserts for Invariants: Use asserts not just to validate current logic, but as a safeguard for future developers, catching regressions and ensuring fundamental search invariants are not violated.
* Returning `nullValue` for Recursive Verification: When guarding against recursive verification, returning `nullValue` is the established practice; alternatives have performed worse.
* Singular Extensions in `qsearch`: Avoid introducing complex search mechanics like singular extensions into simpler, more specialized contexts like `qsearch`.
* Cumulative Pruning Degradation: The cumulative effect of many individually Elo-positive pruning/reduction changes can lead to significant degradation in deep tactical ability and mate-finding.
* Degenerate Search Behavior: Be aware of degenerate behavior; a flawed recursive verification search might devolve into a chain of repeated null move searches.
* Pruning and NNUE Interaction: The effectiveness of pruning heuristics is tightly coupled with NNUE evaluation; a stronger net might make previously safe pruning unsound.
* Conditional Application of Safety Features: Manage the cost of safety features like verification searches by applying them conditionally (e.g., only in non-PV nodes after a primary heuristic indicates cutoff).
* "Death by a Thousand Cuts": A series of individually Elo-positive patches can cumulatively lead to significant regression in specific domains (e.g., tactical ability) by eroding search robustness.
* Robustness vs. Over-tuning: Avoid creating fragile search heuristics by over-tuning them to their absolute limit; a system pushed to the edge may lack robustness for unexpected positions or interactions.
8. **Search State and Threading**
* The search stack (`ss`) and state information (`StateInfo`) are intentionally decoupled; `StateInfo` is tied to the board state, `ss` to the recursive search.
* The node count (`thisThread->nodes`) is frequently used as a source of deterministic, pseudo-random variation, often with bitwise AND (e.g., `nodes & 3`).
* `ss->ply` is the variable for the current search depth (distance from the root in ply).
* Variables influencing search behavior must be reset at the start of every search to prevent unintended behavior.
* Always explicitly initialize variables on the search stack (`ss`); `goto` paths for in-check positions can bypass normal initialization.
* Accessing data from ancestor nodes via `(ss-N)` is valid for historical search information, but ensure current ply is deep enough to prevent out-of-bounds access.
* Not all members of the stack struct (`ss`) are initialized on every node (e.g., `ss->pieceCount` is not set in check nodes).
* Cache root-specific data in the `Thread` struct rather than relying on accessing `rootPos` deep within the recursive search.
* Stockfish uses Lazy SMP because it scales well to a high number of cores, unlike older algorithms.
* A core architectural requirement for Lazy SMP is a shared transposition table, allowing threads to benefit from each other's search results.
* LazySMP inherently widens the search tree compared to single-threaded search; effective SMP patches often compensate by making the search narrower (e.g., more aggressive reductions scaling with threads).
* Reductions should often be increased as the thread count grows, as single-core strategies may not be optimal for multi-threaded search.
* When implementing a heuristic that scales with threads, `std::log(Threads.size())` is a robust method, naturally handling the single-threaded case.
* Multi-threaded (Lazy SMP) search explores a wider tree than a single-threaded search, meaning more threads do not guarantee a faster time-to-depth.
* State that persists between searches (e.g., `previousDepth`) must be stored on a per-thread basis (in the `Thread` class) to avoid race conditions.
* Updating state across all threads must be done at a proper synchronization point, often after the main thread starts the search but before worker threads begin.
* A critical pitfall is failing to initialize local variables within search functions, leading to undefined behavior when specific code paths are skipped (e.g., due to a TT hit).
* To access the root `StateInfo` object from a deeper ply, use pointer arithmetic like `(ss - ss->ply)`.
* Search must be deterministic for a given input and state; non-determinism can arise if state is not properly cleared between searches or if the UCI protocol is violated.
* In a multi-threaded search, the final reported depth can be lower than a previously reported depth if the helper thread that found the high-depth PV is not chosen as the final `bestThread`.
* When altering multi-threaded search logic, consider edge cases, such as requiring a PV length of at least 2 to avoid incorrect behavior.
* A common SMP failure mode is a single "stuck" thread exploring a difficult line while others finish quickly, causing imbalance.
* The `searchAgainCounter` makes the engine re-search the same depth, particularly near the end of allocated time, to improve lazy SMP and move ordering.
* To manage runaway counters, adjust their effect non-lineraly (e.g., `rootDepth - 3 * (counter + 1) / 4`) to periodically force effective search depth increases.
* Modifying search logic for individual threads can create unintended interactions with other systems, like "best thread voting" and TT contents.
* For deterministic search behavior, stateful components like history tables must be properly cleared between searches (e.g., via `ucinewgame`).
* Introducing true randomness into the search is a significant anti-pattern; deterministic "random-like" behavior can be achieved using node counts.
* Each thread requires its own `StateInfo` object and associated tables, imposing a practical limit on the maximum number of threads due to memory consumption.
* For speculative or "read-only" searches, prevent writes to global structures like the TT and preserve all thread-local state, including history tables and variables like `doubleExtensionAverage`.
* When dealing with root-level values dependent on the side to move (like `rootStateValue`), handle the sign correctly, often by using an array indexed by color.
* Be aware of known, subtle inaccuracies in the search state (`ss`), such as `ss->moveCount` after a singular extension or incorrectly inherited `statScore`; "fixing" these has historically not been beneficial.
* Avoid passing many individual state variables as arguments; keep state on the search stack (`ss`) for readability and access to previous plies.
* When accessing data from previous plies (e.g., `(ss-1)->moveCount`), the `Stack` array must be oversized and the `ss` pointer initialized with an offset to prevent out-of-bounds memory access.
* A variable set on the search stack for one move's search can persist and incorrectly influence the search of the next sibling move if not cleared before pruning techniques are evaluated.
9. **Mathematical Formulas and Scaling**
* Be extremely careful with constants in mathematical formulas; using an integer literal instead of a float can cause integer division, truncating results and disabling intended scaling.
* For performance, prefer division by powers of two (e.g., `/ 512`) over arbitrary constants (e.g., `/ 500`), as compilers optimize this to faster bit-shift operations.
* While compilers optimize signed integer division by powers of two, a simple right bitshift (`>>`) is not arithmetically equivalent for negative numbers.
* When designing depth-dependent heuristics, choose scaling factors (e.g., `depth / X`) carefully to ensure they activate at reasonable search depths.
* Formulas for search heuristics can incorporate multiple factors such as TT value, ply, depth, and TT entry's PV status.
* Effective heuristics often combine multiple search parameters in non-obvious ways (e.g., `depth`, `moveCount`, `PvNode`).
* The `statScore` (statistical bonus from move history) is a known non-linear component; attempts to make it scale linearly have historically failed at long time controls.
* Pruning margin formulas can contain quadratic terms (e.g., `int(prev) * prev / C`), deliberately making the pruning effect scale non-linearly.
* Simple mathematical curve-fitting (e.g., linear, logarithmic) for search parameters often fails; relationships are typically more complex.
* The `nnueComplexity` variable, which includes terms for material imbalance and bishop pairs, is used to adjust search intensity.
* Be vigilant about integer overflow, especially when multiplying `Value` types; decompose complex calculations into safer, sequential operations.
* The `Value` type is limited and generally does not exceed `VALUE_INF` (32001), which is about 15 bits; this constraint is critical for multiplications or large bonuses.
* Algebraic transformations valid for floating-point math are often invalid for integer arithmetic due to truncation; the order of operations is critical.
* When implementing quadratic or polynomial adjustments to a value, break the calculation into sequential steps to avoid overflow and precision issues with integer arithmetic.
* When implementing mathematical operations, use the correct C++ standard library functions for type safety, e.g., `std::abs()` for `int64_t`.
* When manipulating evaluation scores, ensure the final value remains clamped within the `VALUE_TB_LOSS_IN_MAX_PLY + 1` and `VALUE_TB_WIN_IN_MAX_PLY - 1` range.
* Heuristics should be logically sound; for example, a comparison involving a score delta and a static evaluation likely needs to use `abs()` to be meaningful.
* Pay close attention to C++ operator precedence in complex reduction and extension formulas; a missing parenthesis can completely change the logic.
10. **Special Cases and Edge Cases**
* Nodes where the king is in check (`ss->inCheck`) are a special case and are assigned a complexity of 0; new logic using complexity should account for this.
* The first move of a game is a significant edge case where search often reaches much higher depths; heuristics may need special handling.
* When implementing a new heuristic relying on state from a previous search (like `previousDepth`), carefully consider its initial value for analysis mode or the first move of a game.
* During a null-move search, the ply is incremented (`ss+1`), so standard ply-based logic remains valid within the recursive sub-search.
* Ensure search functions have robust exit conditions for low or non-positive depths; allowing `depth <= 0` can lead to incorrect behavior.
* The `searchStack` (`ss`) object is passed down through recursive calls, so its state (e.g., `ss->moveCount`) can be inherited from a deeper sub-search and may not reflect the immediate node.
* The `ss` points to the `Stack` entry for the current ply; `ss-1` refers to the parent node's stack information.
* A search change might only be beneficial at shallow depths because at greater depths, the engine often finds the correct move through brute force.
* In clearly winning positions, the search is not optimized to find the fastest mate or highest evaluation, but reliably find *a* winning continuation.
* The search must distinguish between tablebase (TB) win/loss scores and tactical checkmate scores; TB scores are scaled below `VALUE_MATE_IN_MAX_PLY`.
* Avoid adding hard-coded logic for specific endgame scenarios (e.g., insufficient material draws) if the static evaluation already handles them reasonably well, as it adds overhead.
* A critical implementation pitfall is violating the `alpha < beta` invariant in the search; any logic modifying alpha must ensure this.
* Extremely subtle implementation bugs, such as an off-by-one error or `==` instead of `<=`, can be the sole reason a patch passes or fails.
11. **PV-Node Specifics**
* PV update logic is notoriously difficult and a common source of bugs; incorrect PV handling can easily lead to critical errors.
* There is a specific, non-obvious rule for fail-highs in zero-window sub-searches at PV nodes: they are allowed to produce a fail-high, but this behavior is disabled for the root node.
* Reductions at PV nodes are handled as a special case; being in check at a PV node can decrease the reduction applied.
* An "All-Node" is technically defined by the condition `!PvNode && !cutNode`, which is important for understanding when certain extensions or reductions are applied.
* When a node causes a cutoff (fail-high), the move that led to this state is located at `(ss-1)->currentMove`, and its statistics can be updated accordingly.
* Captures and checks are fundamentally special move types; the search architecture treats them differently, and a valid path for new ideas is to extend this concept to other types of direct threats.
* The evaluation of internal (non-leaf) nodes primarily serves to guide search heuristics (move ordering, pruning), while the evaluation at leaf nodes determines the line's value.
* Heuristic or simplified evaluations performed at internal nodes can be stored in the TT, potentially influencing subsequent searches.
* The `evaluate()` function is most frequently called on nodes deep in the tree that were *not* found in the TT, often because they belong to lines pruned by earlier, shallower searches.
* Reductions are decreased for PV-Nodes to search the main line more deeply, typically using a depth-dependent formula.
* Non-PV nodes are fundamentally for zero-window searches, only trying to prove if a move is better than alpha.
* The reliability of the TT move is significantly higher in PV-nodes compared to non-PV (e.g., cut) nodes.
* The PV array for the *next* ply (`ss+1`) must be cleared (`(ss+1)->pv[0] = MOVE_NONE;`) before a recursive search call to that ply.
* The PV array for the *next* ply (`ss+1`) must be correctly set up and initialized *before* making a recursive call for a PV node.
* In `qsearch`, it's possible to prune all moves at a PV node, requiring initialization of the *current* ply's PV (`ss->pv[0] = MOVE_NONE;`) before the move loop to prevent stale data.
* In the main `search`, PV initialization happens *inside* the move loop, assuming at least one move will be searched at a PV node, which is not a safe assumption for `qsearch`.
* The logic for initializing the PV array in `qsearch` should mirror the logic in the main `search` function for consistency.
* A PV node in `qsearch` only recursively calls `qsearch` on other PV nodes; passing a special flag to disable pruning for descendants of a PV node is incorrect.
* The condition to re-search a move in a PV-node is `value > alpha`, not `value >= beta`, and should be guarded by `if (PvNode ...)`.
* Negative pruning on PV-nodes is dangerous; positive pruning (based on a high score) is safer.
* Reducing pruning in PV-nodes is a common idea that seems logical but has historically failed to provide consistent ELO gains.
12. **Parameter Dynamics & Dependencies**
* Search components are tightly coupled; a change in one area (e.g., `LMRdepth`) will directly impact another (e.g., capture futility pruning).
* Search parameters are highly interdependent; a value optimal in isolation may be suboptimal when many other parameters are changed simultaneously.
* Be mindful of logical implications between different search state variables; e.g., `distanceFromPV <= 4` can imply `moveCount <= 5`, allowing for simplification.
* Understand the order of operations and short-circuiting; adding a condition already implied by a preceding check in an `if` statement will be a no-op (pure slowdown).
* A property like `complexity` can influence several distinct parts of a single pruning technique, such as both the evaluation threshold and the depth reduction formula in NMP.
* Seemingly "magic numbers" in the search (e.g., `depth - 3` conditions) often have a logical basis.
* Reason by analogy from existing successful patches; if reducing LMR for PV nodes is effective, test if making quiescence search more thorough for PV nodes also helps.
* Combine information from different search components to create more robust heuristics; e.g., if singular search indicates a move is strong but its TT entry is poor, apply a reduction.
* Attempts to make search parameters change dynamically based on the root depth have historically failed to produce gains.
* Basing dynamic search parameters on the number of nodes searched is more robust and hardware-independent than basing them on elapsed time.
* Scaling search parameters (like pruning thresholds) based on simple metrics like node count or root depth has a history of failure, often regressing at longer time controls.
* Attempts to create a smooth, linear scaling of TC-sensitive parameters based on node count or depth have historically failed.
* Some search heuristics (e.g., `conthist`) are detrimental at very low node counts and only become beneficial after a certain threshold of nodes.
* The ideal search tree shape can depend on position type; tactical positions benefit from wider search, quiet positions from more depth.
* The perceived value and optimal tuning of singular extensions can change significantly as other parts of the search and evaluation evolve.
* Having different threads use different search parameters (e.g., less NMP) has been attempted frequently but rarely successful.
* Optimal values for search parameters are not always strongly dependent on the specific NNUE network; a good search heuristic tends to remain good across different networks.
* Heuristics like "optimism" can have opponent-specific effectiveness; parameters tuned against dev Stockfish may not be optimal against older versions.
* Some search parameters are co-dependent; e.g., `SyzygyProbeDepth` has no effect unless `SyzygyProbeLimit` is also set to match the cardinality of loaded tablebase files.
13. **Time Control Sensitivity**
* Two of the most TC-sensitive areas are countermove history (`conthist`) based pruning and extension margins, particularly the depth at which extensions begin.
* Increasing `conthist` pruning strength tends to improve performance at longer TCs, while reducing it is better for shorter searches, as history scores need more nodes to become reliable.
* For singular extensions, a smaller depth threshold is better for longer TCs, while a larger threshold is better for shorter TCs.
* LMR depth thresholds are critical and TC-sensitive; lowering the depth requirement can be very strong at short TCs but horrible at long TCs.
* Aggressively increasing search extensions often has a strong, non-linear dependency on time control, potentially hurting performance at short TCs while improving it at long TCs.
* Patches that aggressively extend "promising" lines can create a "tunneling" search behavior, effective for deep tactics but brittle when search time is limited.
* For some TC-sensitive parameters to be effective, they must be applied from the very first iterations of the search.
* Code sections or parameters known to have non-linear performance scaling with time/nodes should be commented as such.
* A successful search patch often trades nominal depth for selective depth (`seldepth`); a lower average nominal depth is acceptable if critical variations are explored much more deeply.
* Optimal parameters for singular extensions may depend on search state; they could be stricter at short TCs when the TT is less populated.
* Continuation history-based pruning and extensions are highly tuned for standard and long TCs and are detrimental to performance at very short TCs.
14. **Code Structure and Best Practices**
* Be aware of subtle, non-obvious ordering dependencies in the main search function; refactoring code can cause regressions even if it seems logically equivalent.
* When considering a change, favor modifying a central definition over patching all call sites if it improves code simplicity, even if it seems riskier.
* When implementing position-dependent logic, pay meticulous attention to `Us` vs `~Us` with functions like `relative_square()`.
* Before adding a new conditional check to a pruning heuristic, verify that the condition is not already implicitly guaranteed by the code path.
* The location of variable initialization is critical; a variable not reset correctly between searches can carry over state from one position to the next.
* Be cautious with copy-paste logic, especially in move ordering; applying value calculations intended for one move type to another can lead to incorrect scores.
* When quantizing values, prefer division by powers of two (e.g., 4, 8) over other integers (e.g., 7), as this compiles to faster bit-shift operations.
* When simplifying or removing old code, consider that its associated parameters may be out of tune; a small parameter tweak on the simplified version could turn a neutral change into a gain.
* Some search heuristics are intentionally designed to have the most impact at low ply (near the root).
* Simplifying complex code is a valuable goal in itself, even without a clear performance gain; cleaner code is easier to understand and maintain.
* Be cautious when inserting new code within complex conditional logic; a new statement can unintentionally affect the condition of a subsequent `else` block.
* The performance of code simplifications involving bitboards can be counter-intuitive; an explicit `if (bitboard)` check can be faster if the bitboard is frequently empty.
* Simplifying two different parts of the search can have negative interaction effects, even if each simplification is logical in isolation.
* When a patch removes the use of a search-related variable, ensure its declaration, initialization, and any remaining increments are also removed.
* Refactoring complex search logic for clarity (e.g., moving aspiration search into its own function) is acceptable if it enhances readability without performance loss.
* Avoid using slow floating-point functions like `sqrt()` in performance-critical parts; use integer arithmetic or pre-calculated lookup tables.
* Approximate complex functions with faster integer-based operations (e.g., piecewise linear functions or bit-shifts).
* Be wary of simplifying logical conditions; a seemingly equivalent change might fail to capture the original intent and miss edge cases.
* Avoid adding new UCI options for internal search parameters; the engine's search should be robust and self-contained.
* Reducing lines of code (LOC) is not a goal in itself and should not be prioritized over clarity; complex ternary expressions are an anti-pattern.
* The original purpose of "simplification" was to reduce algorithmic complexity for easier future strength improvements.
* Be deeply aware of implicit properties within specific code paths (e.g., a move generation path only producing captures/promotions that give check).
* CPU-specific instruction sets (PEXT, AVX-512) must be handled carefully, as they can degrade performance on certain architectures.
* Manual prefetch directives can become obsolete "junk code" if modern compilers handle prefetching better.
* The codebase contains necessary performance "hacks" and hardware-specific workarounds accepted for performance reasons.
* Patches with very complex or "black box" code are generally discouraged unless weakly coupled and provide large strength gains.
* Avoid scaling search parameters by `rootDepth` as it's a known failed approach.
* Explicitly cast `Stockfish::Value` in mixed-type arithmetic to prevent ambiguous overload errors.
* Avoid floating-point math in search for performance and determinism; use scaled integer arithmetic.
* Use node count, not time, for hardware-independent heuristic logic.
* Understand the intent of a heuristic before changing it (e.g., 50-move rule damping for fortresses).
* Beware of uninitialized stack variables; certain code paths can bypass normal initialization.
* When a patch modifies multiple distinct search components, separate them to understand individual effects.
* Even seemingly unrelated search patches can have minor, unexpected interactions; true non-interaction is rare.
* While Quiescence Search (qsearch) is critical, it can be successfully modified.
* Avoid unnecessary variable initializations if a variable is guaranteed to be assigned a value on all code paths before read.
* Simplifying complex, illogical, or historically confusing conditions in the search is a valuable goal in itself, as cleaner code is preferred even without immediate Elo gain.
* To avoid redundant computations within a single ply, cache calculated values on the search stack (`ss`) early in the node's processing.
* The order of operations at a search node is highly sensitive; even logical refactoring has failed due to subtle dependencies.
* Group code blocks by the logical condition being checked (e.g., `is_pv_node`), rather than by the action being taken, to improve readability and maintain logical structure.
* Refactoring search code for clarity and efficiency is encouraged (e.g., combining `if (PvNode)` and `if (!PvNode)` into `if/else`).
* Question legacy code constructs; a function like `std::clamp` might be redundant if the value never reaches its boundaries with current tunings.
* Avoid implementing a zero-window search on a non-PV node by manually setting `beta = alpha + 1`; use the `search<NonPV>(...)` template.
* Avoid "optimizations" that try to pre-calculate values already exposed by simple, inlined getter functions (e.g., `Position::rule50_count()`).
* A simple but effective optimization is to add checks to avoid redundant function calls.
* Special-cased search code can often be simplified or removed entirely as the NNUE evaluation function becomes stronger.
* Seemingly minor or non-functional code simplifications can have significant negative interactions with other, unrelated search patches.
* Integer division (`idiv`) is computationally expensive; prefer branchless arithmetic using boolean-to-integer conversion over formulas with multiple divisions.
* Many seemingly complex calculations like `pos.non_pawn_material()` are inexpensive as they read pre-computed values from the `Position` object.
* When implementing logic based on pawn structure, use efficient bitboard operations like `pos.pieces(c, PAWN) & Rank6BB`.
* For performance, prefetching operations in any search function should be placed *after* all pruning checks to avoid fetching data for nodes that will be immediately cut.
* Optimize move loops by identifying calculations that are constant for all moves in the loop and computing them once before the loop.
* When implementing a series of checks for a pruning or extension heuristic, order the conditions by their probability of causing a cutoff to improve performance.
* When implementing a block of multiple pruning conditions, they should be ordered by their probability of triggering to maximize the benefit of logical short-circuiting.
* In a move-pruning loop, if a condition is met that guarantees all subsequent moves will also be pruned, use `break` to exit the loop entirely instead of `continue`.
* For code readability and maintainability, functions in `search.cpp` should be ordered to reflect the call graph.
* A long chain of negative logical ANDs (e.g., `!a && !b && !c`) can often be refactored into a more readable and potentially more efficient inverted OR (e.g., `!(a || b || c)`).
* Leverage template parameters like `PvNode` with `if constexpr` to generate specialized code for different node types at compile time, eliminating runtime branching.
* For parameters that depend on a small, fixed range of shallow depths, a `switch` statement can be more efficient than an array lookup.
* Avoid introducing frequent memory lookups from arrays into tight loops where the original code used direct calculations.
* Do not manually replace standard library functions like `std::min` or arithmetic like division-by-a-power-of-two with `if` statements or bitshifts; compilers optimize these automatically.
* Rewriting simple `if` statements into branchless code (e.g., with ternary operators) is a complex trade-off; it avoids branch misprediction but can introduce more instructions.
* Avoid storing derived data in core search structures (like `RootMoves`) if it can be computed on-the-fly from the main `Position` object without significant performance cost.
* When refactoring or introducing complex search heuristics, prefer using intermediate variables to break down formulas for readability and maintainability.
* When writing complex boolean conditions, use explicit parentheses to group `&&` and `||` clauses to prevent ambiguity, silence compiler warnings, and reduce logic errors.
* Logical simplification is a valuable contribution; remove redundant checks or code that was explicitly removed in previous simplification commits.
* Be aware of historical code cruft; a seemingly redundant line may be a remnant from previous logic and can often be safely removed.
* `search.cpp` is generally considered a more approachable file for new contributors compared to `evaluate.cpp`.
* Boolean-to-Integer Arithmetic: Replacing small `if` statements that apply a constant modification with direct boolean-to-integer arithmetic (e.g., `r += cutNode * 2;`) can improve performance by eliminating conditional branches.
* Slow Integer Division: Integer division is a very slow CPU instruction; avoid dividing by a variable if possible, often favoring `if` statements to branch between division by a constant or no division.
* Ordering Logical Conditions: Place computationally cheapest checks first when ordering logical conditions (e.g., simple booleans before `pos.has_game_cycle()`).
* Compile-Time Template Parameters: Template parameters like `PvNode` and `cutNode` are resolved at compile time; checks involving them have zero runtime cost and don't need ordering for performance.
* "Branchless" Code Optimization: Replacing `if (boolean)` with multiplication by the boolean value may not improve performance, as compilers can convert it back to conditional moves.
* Compile-Time Constant Divisors: Structure conditional logic (e.g., using ternary operators) to ensure divisors are compile-time constants, allowing the compiler to replace slow runtime division with faster multiplication by a magic number.
* Boolean Logic Negation: Be extremely careful with boolean logic, especially when negating complex conditions, as misapplication of De Morgan's laws can lead to fundamentally different conditions.
* Exact Bitwise Flag Checks: When checking for an exact combination of multiple bitwise flags (e.g., `BOUND_EXACT`), use direct equality (`tte->bound() == BOUND_EXACT`), not a simple bitwise AND.
* Boolean Algebra Simplification: Applying boolean algebra can make complex conditional logic in search more readable and potentially easier for the compiler to optimize.
* Ternary Operator vs. Bit-Shifting: A ternary operator (`condition ? a : b`) is not always compiled into the most efficient branchless code; explicitly using bit-shifting (e.g., `val >> (!improving)`) can sometimes force a more optimal instruction sequence.
* When a function has non-obvious behavior (like `pos.capture()` including queen promotions), add a concise comment in the header file explaining this.
|
Here are 15 examples of how previous versions of the Stockfish search code have been improved:
Example patch number 1:
search.cpp:
Improved search.cpp:
Example patch number 2:
search.cpp:
Improved search.cpp:
Example patch number 3:
search.cpp:
Improved search.cpp:
Example patch number 4:
search.cpp:
Improved search.cpp:
Example patch number 5:
search.cpp:
Improved search.cpp:
Example patch number 6:
search.cpp:
Improved search.cpp:
Example patch number 7:
search.cpp:
Improved search.cpp:
Example patch number 8:
search.cpp:
Improved search.cpp:
Example patch number 9:
search.cpp:
Improved search.cpp:
Example patch number 10:
search.cpp:
Improved search.cpp:
Example patch number 11:
search.cpp:
Improved search.cpp:
Example patch number 12:
search.cpp:
Improved search.cpp:
Example patch number 13:
search.cpp:
Improved search.cpp:
Example patch number 14:
search.cpp:
Improved search.cpp:
Example patch number 15:
search.cpp:
Improved search.cpp:
Can you create a single new patch for the Stockfish search code shown above?
It should be definitely more than just changing parameters.
Generally avoid changing parameters, since they are already optimized/tuned for long time controls.
You can take also take inspiration from the provided examples.
When applicable, prefer scalers (i.e. patches that scale well in LTC (long time control)).
A good workflow to find a new patch:
- Take example number 15 and select the roughly same search technique step that is targeted in this example to improve upon. To be clear, this is only about the area to improve, not the patch itself.
- Describe what the current implementation of the selected code snippet does.
- Deeply and critically analyze which improvements to the selected part make sense.
At the end, please provide the final patch in same format as above examples, meaning:
"search.cpp:
Improved search.cpp:
cpp
// ... new code"
And finally the last line should look like this:
"Branch name: a_suggested_branch_name_for_this_patch"
Where "a_suggested_branch_name_for_this_patch" should be adapted to fit the final patch.