C++: Improve performance of Printf::callsVariadicFormatter.

[geoffw0]

Improve performance of cpp/cleartext-storage-buffer, which regressed a month or so ago. The difference isn't huge, though the fix is in library code so it could affect other queries as well.

Locally on wireshark (with cache warmed up by cpp/system-data-exposure) this change improves performance from 55s -> 45s. It appears to have eliminated entirely the following pre-computation:

[2022-06-01 10:56:39] Evaluated non-recursive predicate Printf::callsVariadicFormatter#b0c2256f#bfff#join_rhs#1@3c8b9cg6 in 1308ms (size: 993161).
Tuple counts for Printf::callsVariadicFormatter#b0c2256f#bfff#join_rhs#1@3c8b9cg6:
         650340  ~1%    {4} r1 = SCAN Function::Function::getParameter#dispred#f0820431#bff OUTPUT In.2, -1, In.0, In.1
        1428098  ~6%    {4} r2 = JOIN r1 WITH Access::Access::getTarget#dispred#f0820431#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, -1, Lhs.2, Lhs.3
         996581  ~0%    {5} r3 = JOIN r2 WITH Call::Call::getArgument#dispred#f0820431#fff_201#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.2, -1, Lhs.3, Rhs.2
         996566  ~2%    {5} r4 = JOIN r3 WITH Expr::Expr::getEnclosingFunction#dispred#f0820431#bb ON FIRST 2 OUTPUT Lhs.0, -1, Lhs.1, Lhs.3, Lhs.4
         993161  ~5%    {6} r5 = JOIN r4 WITH Call::FunctionCall::getTarget#dispred#f0820431#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.4, -1, Lhs.2, Lhs.3, Lhs.0
                        return r5

seemingly without affecting the recusion - before:

[2022-06-01 10:56:51] Completed evaluation of a recursive layer containing 2 different predicates (took 70ms). Predicates evaluated:
Printf::callsVariadicFormatter#b0c2256f#bfff@0950fw3s (took 19ms) (size: 127)
Printf::variadicFormatter#b0c2256f#bfff@0950fx3s (took 48ms) (size: 54)

after:

[2022-06-01 11:42:23] Completed evaluation of a recursive layer containing 2 different predicates (took 52ms). Predicates evaluated:
Printf::callsVariadicFormatter#b0c2256f#ffff@0b44ewig (took 19ms) (size: 127)
Printf::variadicFormatter#b0c2256f#ffff@0b44exig (took 32ms) (size: 54)

Confirmed on kamailio 103s -> 96s. I'll do a DCA run as well to check nothing is impacted negatively...

[MathiasVP]

As far as I can see, the join_rhs materialization is now gone because we're starting the pipeline with a join between Printf::variadicFormatter and getTarget, and only then do we look at the arguments/parameter pairs. That's indeed a much better join order than what we did before.

There's a lot going on in that pipeline, so I can't immediately see if this is the right way to enforce this join order, but if DCA is happy it's a clear win.

[geoffw0]

There's a lot going on in that pipeline, so I can't immediately see if this is the right way to enforce this join order, but if DCA is happy it's a clear win.

I was expecting to have to try pragmas in a few different places, such as tightly around just fc - but it worked well in the first place I tried and experience has taught me to accept that. The thinking was that I want to force it to explore the recursion first because in this case its overwhelmingly likely to reach only a very small proportion of all functions (ones that are essentially variadic formatters) and we don't want to do any work on other functions.

I've started the DCA run (after a bit of a fiddle).

[MathiasVP]

The thinking was that I want to force it to explore the recursion first because in this case its overwhelmingly likely to reach only a very small proportion of all functions (ones that are essentially variadic formatters) and we don't want to do any work on other functions.

And that's exactly what ended up happening after your change, so that's excellent!

Note that the safest way to enforce a join to be first in the pipeline is to unbind all its columns (this pattern is also mentioned in the join order document). It works in this case because the optimizer picked an ordering that only joined on the first column (which you correctly pragma'd), but it could in theory also start with some of the other conjuncts and end off joining on the recursive case using a combination of the last three columns. So if you unbind those three columns as well you avoid this potential issue.

[geoffw0]

Note that the safest way to enforce a join to be first in the pipeline is to unbind all its columns

Noted. My instinct at present is to prefer a minimal intervention (as few unbinds as reasonably possible), but this might just be a remnant of the way we used to think when the optimizer was young. I wonder if there have been cases where we've had to go back and add more unbinds for a fix that has regressed?

In any case this is approved and working and not worth spending hours on, so I'm merging it.

[MathiasVP]

Noted. My instinct at present is to prefer a minimal intervention (as few unbinds as reasonably possible), but this might just be a remnant of the way we used to think when the optimizer was young. I wonder if there have been cases where we've had to go back and add more unbinds for a fix that has regressed?

I can't think of a case off the top of my head, no.