Matchmaking

I seem to get into many lobbies where we are waiting for 1, 2, or 3 players for way too long. Sometimes the players never show up and I end up having to leave the lobby by way of the xbox one party app. Will this be fixed? I feel like the priority for filling players should be in lobbies that only need a few more instead of creating an entire new lobby.