Set and Map don't infer own types (regression)

#20606 should be reverted for now.

**TypeScript Version:**  master

**Code**

```ts
new Set([]);
```

**Expected behavior:**

`Set<never>`

**Actual behavior:**

`Set<never | ReadonlyArray<never> | undefined>`

**Related:**
