Fix performance of device lists in `/key/changes` and sliding sync.