Reduce memory usage when joining very large rooms over federation.