SimplifyInputUnion was dropping element types that were not inputs. These changes fix that bug. Fixes #7454.