Can't fix lag, not ISP, troubleshooted everything

So my roommate and I both bought Xbox ones and halo guardians to be able to play together without split screen, but the game will only work for one game before we both get the most ridiculous lag. Party chat will work fine, can hear each other and party members loud and clear, but after one game we take turns getting massive lagspikes that run everyone into walls until 3min later the game comes back only to realize you’ve died 10 times. Then right when everything seems better, I’d enter combat and the lag returns. Because the internet works fine when we 2 box any other game, and party chat isn’t effected by this lag, it has to be something on your side. I even had my ISP ping our connection while the lag was happening. Everything was green. It doesn’t happen when only one of us plays at a time. Only when there are 2 boxes running. Ive opened ports, tried wireless and wired, even got a brand new router. PLEASE HELP IVE TROUBLESHOOTED HONEST TO GOD EVERYTHING. thanks.