QMK Firmware Keyboard and Keymander 2 3Play

I hope someone else with a QMK firmware based keyboard or a similar error can share their experience here, so I have a better start for my troubleshooting

I'm using an XD75 Keyboard with QMK firmware, and the Keymander 2 3Play as a KMM Switch for two PCs and a PS5.

The console hand-through and emulation work fine, but actually typing on my computer doesn't.

When I press caps lock (not always on the first press but within 5 key presses at most) keyboard and mouse freeze and it seems that the Keymander restarts. After a couple of seconds mouse and keyboard come back and now every caps lock press will directly cause the same behaviour.

When I use another mechanical keyboard everything seems fine, when I use the XD75 keyboard directly on the same device or any other device I don't see any issues.