Minor speed ups when joining large rooms over federation.