feat: adding stochastic positional embeddings

[kir-7]

Part of #1900

Summary

• Adds add_stochastic_positional_noise as a utility function to lightly/models/utils.py.
• Adds optional use of StoP to:
  * lightly/models/modules/ijepa.py and
  * lightly/models/modules/ijepa_timm.py
• Keeps StoP disabled by default to preserve existing behaviour.
• Adds a new test tests/utils/test_stochastic_positional_embedding.py to test the behaviour of the StoP function while disabled and enabled.

Additional details:

• StoP can be used for masked image modelling (MIM) models and this PR uses it for IJEPA. It can be used for MAE as well, but currently MAEDecoder in lightly/models/modules does not provide functionality to modify the masks directly, these are modified while creating the MAE class (examples/pytorch/mae.py). These examples can be modified to evaluate StoP.

[gabrielfruet]

/review

[codecov]

Codecov Report

❌ Patch coverage is 69.23077% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.33%. Comparing base (9c0346f) to head (9c27f5f).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
lightly/models/modules/ijepa.py	0.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1963      +/-   ##
==========================================
- Coverage   86.36%   86.33%   -0.04%     
==========================================
  Files         172      172              
  Lines        7358     7371      +13     
==========================================
+ Hits         6355     6364       +9     
- Misses       1003     1007       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kir-7]

@gabrielfruet The failing check seems to be regarding test coverage for ijepa.py for from_vit_encoder, should I add tests for this? And is there anything else I need to modify?

[gabrielfruet]

@kir-7 Thank you for your contribution! No need to worry, we are planning to remove it #1966 . Either me or Lionel are going to review your PR as soon as we can : )

[liopeer]

One last thing to address, otherwise looking good!

[kir-7]

Hi @liopeer, a quick reminder for this PR. I have finished resolving the previous comments. When you have a chance, could you please take another look and let me know if any further changes are needed? Thanks!

[liopeer]

It looks mostly correct now, sorry that it took a long time to review. One thing that we need to address though: Adding noise should only happen if the module is in training mode!

[liopeer]

LGTM, thanks a lot!

[kir-7]

I also noticed that IJEPAPredictor.from_vit_encoder does not properly initial noise_std, so model created through this will default noise_std=0.0. Should we add an optional noise_std: float = 0.0 argument to from_vit_encoder and pass it to cls(...)?

[liopeer]

I think it's fine to leave it as it is, no need to cover this imo.