Xbox controller lose settings between games

I have an Xbox One and a regular xbox controller. When I play Multiplayer, it often happens that my controller loses its settings. Above all, my thumbstick becomes extremely sensitive. A small movement gives large effects. If I leave the game and go into another game, my controller often return to normal.

Does anyone else experience the same thing? Why is this happening? How should I be able to fix it?