Fix a long-standing bug where invites from ignored users were included in incremental syncs.