Answering myself. It was an "random" hardawe problem BOTH on the
network card and switch (I changed one and kept the other in my first
tests, that's what I didn't notice it).
For anyone interested, by lowering the MTU from 1500 to an small 512
value, samba connection could "hardly" work with the broken hardware.
This could be a quick test to check for random network hardware