[pcaspers]

We use this kind of speed optimization for quite a while, especially for "heavier" processes like our CrossAssetStateProcess, where the caching of quantities that are path-independent makes a huge difference. So far we have not touched the StochasticProcessbase interface though.

This PR is the attempt to implement the concept in the base class to simplify our code.

We als add caching to GeneralizedBlackScholesProcess and enable the caching in MonteCarloModel via McSimulation if the derived pricing engine enables this. For now we are enabling the caching only for the mc asian option engines. We need to check every engine one by one, because a non-standard access pattern to the stochastic process's drift / diffusion methods can lead to an exception from PathIndependentCache. For example, the mc barrier engines have an extra call into diffusion() after each generated path. We should easily fix this by suspending the caching for these calls. Will come back to this and go through all classes derived from McSimulation.

Seeing a speedup of around 5x in testMCDiscreteGeometricAveragePrice. This is using trivial yield and vol term structures (constant), I would expect even higher speedups when real-world term structures are used where the lookup is more expensive.

Also fixing #2951 and removing the evolve() override from GeneralizedBlcakScholesProcess which is not necessary, and reinstating the discretization's drift() implementation in expectation() if exact resolution is not applicable.

[pcaspers]

@lballabio the checks are not running, do you have to approve them for this branch?

[lballabio]

@pcaspers Yes, there's been a noticeable increase of PR lately, I had to turn off automated runs off in order to have some hope to manage them in an ordered way. I'm running CI for a PR manually after I review it.

[pcaspers]

@lballabio ah I see, makes sense.

[kp9991-git]

Hi @lballabio @pcaspers , I thought I'd add to the point on PRs. Clearly producing code has become easy with AI and the bottleneck is now review and validation. I myself spend most of my time validating rather than coding. Looking at PRs, there is no guarantee how deep the validation has been.

So hypothetically, if previously it took maybe a couple of weeks to fix some model, and the validation was to some extent natural because the dev process was manual (so you had to go into every detail), then now it takes half a day with AI, the validation is within unit tests (which is fine but not everything) and it still takes Luigi about the same time to review.

Something to think about I guess. Just wanted to share.

[pcaspers]

Hi @lballabio @pcaspers , I thought I'd add to the point on PRs. Clearly producing code has become easy with AI and the bottleneck is now review and validation. I myself spend most of my time validating rather than coding. Looking at PRs, there is no guarantee how deep the validation has been.

So hypothetically, if previously it took maybe a couple of weeks to fix some model, and the validation was to some extent natural because the dev process was manual (so you had to go into every detail), then now it takes half a day with AI, the validation is within unit tests (which is fine but not everything) and it still takes Luigi about the same time to review.

Something to think about I guess. Just wanted to share.

I think it should not be this way. AI is a great tool and I use it a lot myself. But in the end the human contributor is responsible for a PR, same as in the time before AI, i.e., the they have to understand each line of the code, be prepared to answer questions (not forwarding questions to the ai), defend the code and design, make sure the code is high quality and compact, as if they had written it themselves. I know it is hard to just reject a PR due to insufficient quality, it is often a grey area, and we had bad PRs before ai, too. But if people can not use ai responsibly, it is the only way. It is not Luigi's or any maintainers task to sort this out and waste their time on it.

[pcaspers]

Having said that, this PR is 100% ai-free. I made it a rule for me to not use ai for QuantLib PRs, in order not to unlearn coding. :-)