If you mean zooming out when the map is opened during a service, just press the button that zooms out to stop it. Similarly, if it starts panning without input, use the control to pan in that direction then return to centre. I'm not sure why any of this happens.
Joystick drift. As controllers get older, they wear in the thumbstick ball-joints and don't hold zero especially well.
If zoom was on the sticks, maybe. But zoom on the map is on the triggers on XBox. And you cannot remap that ingame, so unless the OP remapped it at the system level with a unique controller profile for TSW2....