Always commute strategy when preprocessing DESC keys.

A recently added nbtree preprocessing step failed to account for the
fact that DESC columns already had their B-Tree strategy number commuted
at this point in preprocessing.  As a result, preprocessing could output
a set of scan keys where one or more keys had the correct strategy
number, but used the wrong comparison routine.

To fix, make the faulty code path that looks up a more restrictive
replacement operator/comparison routine commute its requested inequality
strategy (while outputting the transformed strategy number as before).
This makes the final transformed scan key comport with the approach
preprocessing has always used to deal with DESC columns (which is
described by comments above _bt_fix_scankey_strategy).

Oversight in commit commit b3f1a13f, which made nbtree preprocessing
perform transformations on skip array inequalities that can reduce the
total number of index searches.

Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Natalya Aksman <natalya@timescale.com>
Discussion: https://postgr.es/m/19049-b7df801e71de41b2@postgresql.org
Backpatch-through: 18

diff --git a/src/backend/access/nbtree/nbtpreprocesskeys.c b/src/backend/access/nbtree/nbtpreprocesskeys.c
index 21c519cd108edbefadda3cfb74e9168798e0fe9d..4c9389c781a57379f51dcb7a1a83023cb5ca82b2 100644 (file)
--- a/src/backend/access/nbtree/nbtpreprocesskeys.c
+++ b/src/backend/access/nbtree/nbtpreprocesskeys.c
@@ -1410,6 +1410,7 @@ _bt_skiparray_strat_decrement(IndexScanDesc scan, ScanKey arraysk,
    Datum       orig_sk_argument = high_compare->sk_argument,
                new_sk_argument;
    bool        uflow;
+   int16       lookupstrat;
 
    Assert(high_compare->sk_strategy == BTLessStrategyNumber);
 
@@ -1431,9 +1432,14 @@ _bt_skiparray_strat_decrement(IndexScanDesc scan, ScanKey arraysk,
        return;
    }
 
-   /* Look up <= operator (might fail) */
-   leop = get_opfamily_member(opfamily, opcintype, opcintype,
-                              BTLessEqualStrategyNumber);
+   /*
+    * Look up <= operator (might fail), accounting for the fact that a
+    * high_compare on a DESC column already had its strategy commuted
+    */
+   lookupstrat = BTLessEqualStrategyNumber;
+   if (high_compare->sk_flags & SK_BT_DESC)
+       lookupstrat = BTGreaterEqualStrategyNumber; /* commute this too */
+   leop = get_opfamily_member(opfamily, opcintype, opcintype, lookupstrat);
    if (!OidIsValid(leop))
        return;
    cmp_proc = get_opcode(leop);
@@ -1462,6 +1468,7 @@ _bt_skiparray_strat_increment(IndexScanDesc scan, ScanKey arraysk,
    Datum       orig_sk_argument = low_compare->sk_argument,
                new_sk_argument;
    bool        oflow;
+   int16       lookupstrat;
 
    Assert(low_compare->sk_strategy == BTGreaterStrategyNumber);
 
@@ -1483,9 +1490,14 @@ _bt_skiparray_strat_increment(IndexScanDesc scan, ScanKey arraysk,
        return;
    }
 
-   /* Look up >= operator (might fail) */
-   geop = get_opfamily_member(opfamily, opcintype, opcintype,
-                              BTGreaterEqualStrategyNumber);
+   /*
+    * Look up >= operator (might fail), accounting for the fact that a
+    * low_compare on a DESC column already had its strategy commuted
+    */
+   lookupstrat = BTGreaterEqualStrategyNumber;
+   if (low_compare->sk_flags & SK_BT_DESC)
+       lookupstrat = BTLessEqualStrategyNumber; /* commute this too */
+   geop = get_opfamily_member(opfamily, opcintype, opcintype, lookupstrat);
    if (!OidIsValid(geop))
        return;
    cmp_proc = get_opcode(geop);