Always fully compute variance structurally

[ahejlsberg]

Fixes#44572.

[ahejlsberg]

@typescript-bot test this
@typescript-bot user test this inline
@typescript-bot run dt
@typescript-bot perf test faster

[typescript-bot]

Heya @ahejlsberg, I've started to run the extended test suite on this PR at de4a166. You can monitor the build here.

[typescript-bot]

Heya @ahejlsberg, I've started to run the abridged perf test suite on this PR at de4a166. You can monitor the build here.

Update: The results are in!

[typescript-bot]

Heya @ahejlsberg, I've started to run the diff-based community code test suite on this PR at de4a166. You can monitor the build here.

Update: The results are in!

[typescript-bot]

Heya @ahejlsberg, I've started to run the parallelized Definitely Typed test suite on this PR at de4a166. You can monitor the build here.

[typescript-bot]

@ahejlsberg
Great news! no new errors were found between main..refs/pull/48080/merge

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..48080

Metric	main	48080	Delta	Best	Worst
Angular - node (v14.15.1, x64)
Memory used	333,049k (± 0.01%)	333,044k (± 0.01%)	-5k (- 0.00%)	332,915k	333,125k
Parse Time	2.02s (± 0.47%)	2.02s (± 0.55%)	+0.01s (+ 0.30%)	2.00s	2.05s
Bind Time	0.88s (± 0.25%)	0.88s (± 0.96%)	-0.00s (- 0.23%)	0.87s	0.91s
Check Time	5.51s (± 0.37%)	5.49s (± 0.48%)	-0.02s (- 0.40%)	5.45s	5.58s
Emit Time	6.27s (± 0.56%)	6.26s (± 0.57%)	-0.01s (- 0.19%)	6.17s	6.35s
Total Time	14.68s (± 0.31%)	14.65s (± 0.41%)	-0.03s (- 0.23%)	14.52s	14.78s
Compiler-Unions - node (v14.15.1, x64)
Memory used	193,823k (± 0.61%)	193,177k (± 0.60%)	-646k (- 0.33%)	191,818k	195,128k
Parse Time	0.86s (± 0.75%)	0.85s (± 0.43%)	-0.00s (- 0.35%)	0.85s	0.86s
Bind Time	0.56s (± 0.93%)	0.56s (± 0.53%)	+0.00s (+ 0.18%)	0.56s	0.57s
Check Time	7.43s (± 0.63%)	7.37s (± 0.52%)	-0.06s (- 0.75%)	7.31s	7.45s
Emit Time	2.49s (± 1.12%)	2.47s (± 0.57%)	-0.02s (- 0.72%)	2.44s	2.50s
Total Time	11.34s (± 0.59%)	11.26s (± 0.43%)	-0.08s (- 0.68%)	11.16s	11.36s
Monaco - node (v14.15.1, x64)
Memory used	325,126k (± 0.01%)	325,131k (± 0.01%)	+5k (+ 0.00%)	325,086k	325,168k
Parse Time	1.56s (± 0.38%)	1.56s (± 0.58%)	-0.00s (- 0.19%)	1.55s	1.59s
Bind Time	0.77s (± 0.44%)	0.77s (± 0.47%)	-0.01s (- 0.91%)	0.76s	0.77s
Check Time	5.40s (± 0.39%)	5.41s (± 0.52%)	+0.01s (+ 0.24%)	5.35s	5.46s
Emit Time	3.31s (± 0.83%)	3.29s (± 0.35%)	-0.02s (- 0.51%)	3.26s	3.31s
Total Time	11.04s (± 0.43%)	11.03s (± 0.34%)	-0.01s (- 0.08%)	10.94s	11.10s
TFS - node (v14.15.1, x64)
Memory used	288,669k (± 0.01%)	288,656k (± 0.01%)	-13k (- 0.00%)	288,607k	288,734k
Parse Time	1.33s (± 1.98%)	1.33s (± 1.76%)	+0.00s (+ 0.15%)	1.30s	1.41s
Bind Time	0.74s (± 1.37%)	0.73s (± 0.64%)	-0.01s (- 0.82%)	0.72s	0.74s
Check Time	5.06s (± 0.57%)	5.06s (± 0.59%)	+0.00s (+ 0.02%)	5.01s	5.14s
Emit Time	3.53s (± 1.82%)	3.47s (± 1.98%)	-0.06s (- 1.73%)	3.37s	3.62s
Total Time	10.66s (± 0.73%)	10.60s (± 0.84%)	-0.06s (- 0.56%)	10.45s	10.81s
material-ui - node (v14.15.1, x64)
Memory used	446,469k (± 0.00%)	446,288k (± 0.06%)	-181k (- 0.04%)	445,162k	446,458k
Parse Time	1.85s (± 0.31%)	1.85s (± 0.46%)	+0.00s (+ 0.05%)	1.84s	1.87s
Bind Time	0.70s (± 1.15%)	0.70s (± 1.17%)	-0.00s (- 0.14%)	0.68s	0.72s
Check Time	12.86s (± 0.57%)	12.80s (± 0.54%)	-0.06s (- 0.44%)	12.60s	12.95s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	15.40s (± 0.49%)	15.35s (± 0.49%)	-0.06s (- 0.38%)	15.14s	15.51s
xstate - node (v14.15.1, x64)
Memory used	535,257k (± 0.01%)	536,585k (± 0.01%)	+1,328k (+ 0.25%)	536,532k	536,656k
Parse Time	2.58s (± 0.35%)	2.57s (± 0.49%)	-0.02s (- 0.62%)	2.55s	2.60s
Bind Time	1.16s (± 1.05%)	1.15s (± 0.72%)	-0.01s (- 0.60%)	1.14s	1.18s
Check Time	1.50s (± 0.55%)	1.55s (± 0.26%)	+0.06s (+ 3.68%)	1.54s	1.56s
Emit Time	0.07s (± 3.14%)	0.07s (± 4.13%)	+0.00s (+ 1.41%)	0.07s	0.08s
Total Time	5.32s (± 0.32%)	5.35s (± 0.20%)	+0.04s (+ 0.66%)	5.33s	5.37s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-210-generic
Architecture	x64
Available Memory	16 GB
Available Memory	5 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v14.15.1, x64)

Scenarios

• Angular - node (v14.15.1, x64)
• Compiler-Unions - node (v14.15.1, x64)
• Monaco - node (v14.15.1, x64)
• TFS - node (v14.15.1, x64)
• material-ui - node (v14.15.1, x64)
• xstate - node (v14.15.1, x64)

Benchmark	Name	Iterations
Current	48080	10
Baseline	main	10

Developer Information:

Download Benchmark

[RyanCavanaugh]

@typescript-bot pack this

[typescript-bot]

Heya @RyanCavanaugh, I've started to run the tarball bundle task on this PR at de4a166. You can monitor the build here.

[ahejlsberg]

@typescript-bot perf test faster

[typescript-bot]

Heya @ahejlsberg, I've started to run the abridged perf test suite on this PR at d059791. You can monitor the build here.

Update: The results are in!

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..48080

Metric	main	48080	Delta	Best	Worst
Angular - node (v14.15.1, x64)
Memory used	333,049k (± 0.01%)	333,025k (± 0.01%)	-23k (- 0.01%)	332,980k	333,088k
Parse Time	2.02s (± 0.47%)	2.02s (± 0.70%)	+0.01s (+ 0.35%)	2.00s	2.06s
Bind Time	0.88s (± 0.25%)	0.88s (± 0.38%)	-0.00s (- 0.11%)	0.87s	0.89s
Check Time	5.51s (± 0.37%)	5.49s (± 0.50%)	-0.02s (- 0.33%)	5.43s	5.57s
Emit Time	6.27s (± 0.56%)	6.26s (± 0.39%)	-0.01s (- 0.22%)	6.21s	6.32s
Total Time	14.68s (± 0.31%)	14.66s (± 0.26%)	-0.03s (- 0.18%)	14.55s	14.75s
Compiler-Unions - node (v14.15.1, x64)
Memory used	193,823k (± 0.61%)	193,115k (± 0.58%)	-709k (- 0.37%)	191,870k	195,167k
Parse Time	0.86s (± 0.75%)	0.85s (± 0.52%)	-0.01s (- 0.70%)	0.84s	0.86s
Bind Time	0.56s (± 0.93%)	0.56s (± 0.53%)	+0.00s (+ 0.18%)	0.56s	0.57s
Check Time	7.43s (± 0.63%)	7.37s (± 0.53%)	-0.06s (- 0.75%)	7.32s	7.48s
Emit Time	2.49s (± 1.12%)	2.46s (± 0.87%)	-0.02s (- 0.88%)	2.42s	2.51s
Total Time	11.34s (± 0.59%)	11.25s (± 0.48%)	-0.08s (- 0.75%)	11.16s	11.37s
Monaco - node (v14.15.1, x64)
Memory used	325,126k (± 0.01%)	325,133k (± 0.00%)	+7k (+ 0.00%)	325,090k	325,152k
Parse Time	1.56s (± 0.38%)	1.56s (± 0.51%)	-0.00s (- 0.13%)	1.54s	1.58s
Bind Time	0.77s (± 0.44%)	0.77s (± 0.44%)	-0.01s (- 0.78%)	0.76s	0.77s
Check Time	5.40s (± 0.39%)	5.37s (± 0.35%)	-0.02s (- 0.39%)	5.34s	5.42s
Emit Time	3.31s (± 0.83%)	3.28s (± 0.39%)	-0.02s (- 0.73%)	3.25s	3.30s
Total Time	11.04s (± 0.43%)	10.99s (± 0.24%)	-0.05s (- 0.43%)	10.94s	11.05s
TFS - node (v14.15.1, x64)
Memory used	288,669k (± 0.01%)	288,667k (± 0.01%)	-2k (- 0.00%)	288,619k	288,697k
Parse Time	1.33s (± 1.98%)	1.32s (± 2.40%)	-0.01s (- 0.60%)	1.28s	1.42s
Bind Time	0.74s (± 1.37%)	0.74s (± 1.58%)	+0.00s (+ 0.00%)	0.72s	0.77s
Check Time	5.06s (± 0.57%)	5.05s (± 0.41%)	-0.01s (- 0.22%)	5.03s	5.13s
Emit Time	3.53s (± 1.82%)	3.59s (± 1.55%)	+0.06s (+ 1.61%)	3.39s	3.69s
Total Time	10.66s (± 0.73%)	10.70s (± 0.71%)	+0.04s (+ 0.38%)	10.46s	10.86s
material-ui - node (v14.15.1, x64)
Memory used	446,469k (± 0.00%)	446,314k (± 0.06%)	-155k (- 0.03%)	445,182k	446,488k
Parse Time	1.85s (± 0.31%)	1.85s (± 0.55%)	-0.00s (- 0.05%)	1.83s	1.87s
Bind Time	0.70s (± 1.15%)	0.70s (± 0.74%)	+0.00s (+ 0.14%)	0.69s	0.71s
Check Time	12.86s (± 0.57%)	12.76s (± 0.56%)	-0.10s (- 0.74%)	12.65s	12.90s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	15.40s (± 0.49%)	15.31s (± 0.48%)	-0.10s (- 0.62%)	15.20s	15.46s
xstate - node (v14.15.1, x64)
Memory used	535,257k (± 0.01%)	536,574k (± 0.00%)	+1,317k (+ 0.25%)	536,539k	536,633k
Parse Time	2.58s (± 0.35%)	2.58s (± 0.32%)	-0.01s (- 0.39%)	2.56s	2.60s
Bind Time	1.16s (± 1.05%)	1.15s (± 0.56%)	-0.01s (- 0.78%)	1.14s	1.17s
Check Time	1.50s (± 0.55%)	1.55s (± 0.53%)	+0.06s (+ 3.88%)	1.54s	1.58s
Emit Time	0.07s (± 3.14%)	0.07s (± 3.14%)	0.00s ( 0.00%)	0.07s	0.08s
Total Time	5.32s (± 0.32%)	5.36s (± 0.27%)	+0.04s (+ 0.81%)	5.34s	5.39s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-210-generic
Architecture	x64
Available Memory	16 GB
Available Memory	5 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v14.15.1, x64)

Scenarios

• Angular - node (v14.15.1, x64)
• Compiler-Unions - node (v14.15.1, x64)
• Monaco - node (v14.15.1, x64)
• TFS - node (v14.15.1, x64)
• material-ui - node (v14.15.1, x64)
• xstate - node (v14.15.1, x64)

Benchmark	Name	Iterations
Current	48080	10
Baseline	main	10

Developer Information:

Download Benchmark

[typescript-bot]

Hey @RyanCavanaugh, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{"devDependencies":{"typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/121047/artifacts?artifactName=tgz&fileId=A941446E3AD1E9DD2C00F39A3CBA6B60B8F87AE895A0C53B3DB474331FB6762002&fileName=/typescript-4.7.0-insiders.20220302.tgz" } } 

and then running npm install.

[ahejlsberg]

Tests are clean. Performance is slightly impacted in xstate but otherwise improved. I think this is an acceptable fix.

[weswigham]

This looks good, but I wonder if the inconpleteVariancesObserved flag needs to be saved and restored from the cached relation result similarly to variance markers themselves.

[ahejlsberg]

This looks good, but I wonder if the incompleteVariancesObserved flag needs to be saved and restored from the cached relation result similarly to variance markers themselves.

That shouldn't be necessary because we don't cache relations that observed incomplete variances (they result in Ternary.Unknown, and those never get cached).

[Andarist]

This PR doesn't fix the problem reported here while the previous PR was fixing it.

EDIT:// I've also rechecked my other report that was being fixed by the previous PR and it isn't handled by this PR here, the issue in question: #45859

[ahejlsberg]

Sadly there are still issues with the fix in this PR as well as the fix in (the now closed) #45628. The core issue looks to be mutually recursive types in which variance is flipped by one of the types. For example:

typeFoo<T>={x: T;f: FooFn<T>;}typeFooFn<T>=(foo: Foo<T>)=>void;declareletfoo1: Foo<string>;declareletfoo2: Foo<unknown>;declareletfn1: FooFn<string>;declareletfn2: FooFn<unknown>;foo1=foo2;// Errorfoo2=foo1;// Errorfn1=fn2;// Errorfn2=fn1;// Error

Above, the x property in Foo<T> witnesses T co-variantly and the f property witnesses T recursively and contra-variantly. The combination should cause T to be considered invariant, and it is in the example above. But with a small change to FooFn<T> things fall apart:

typeFooFn<T>=(foo: Foo<T[])=>void;

Now, because T isn't referenced as a "naked" type parameter, we don't examine the Foo<T[]> reference structurally, but rather attempt to rely on the computed variance--which is in the process of being computed. So, results come out wrong and are potentially sensitive to order of computation. For example, with this PR we get the following behavior:

foo1=foo2;// Errorfoo2=foo1;fn1=fn2;fn2=fn1;

meaning co-variance for Foo<T> and no witnessing in FooFn<T>. With #45628 we get

foo1=foo2;// Errorfoo2=foo1;fn1=fn2;fn2=fn1;// Error

i.e. contra-variance for FooFn<T> which is slightly more consistent, but still wrong.

The tricky aspect of this example (and others like it) is that invariance isn't revealed unless we examine multiple circular type references structurally and in-depth. Which of course is exactly what we're trying to avoid with the variance computation in the first place.

[Andarist]

Now, because T isn't referenced as a "naked" type parameter, we don't examine the Foo<T[]> reference structurally, but rather attempt to rely on the computed variance

Why is this making a difference? Is it purely an optimization for the most common case or something like that? I don't quite understand why naked and not-naked type parameters go through a different code path here 🤔

[typescript-bot]

Heya @ahejlsberg, I've started to run the extended test suite on this PR at a323037. You can monitor the build here.

[typescript-bot]

Heya @ahejlsberg, I've started to run the abridged perf test suite on this PR at a323037. You can monitor the build here.

Update: The results are in!

[typescript-bot]

Heya @ahejlsberg, I've started to run the diff-based community code test suite on this PR at a323037. You can monitor the build here.

Update: The results are in!

[typescript-bot]

Heya @ahejlsberg, I've started to run the parallelized Definitely Typed test suite on this PR at a323037. You can monitor the build here.

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..48080

Metric	main	48080	Delta	Best	Worst
Angular - node (v14.15.1, x64)
Memory used	332,984k (± 0.01%)	331,371k (± 0.01%)	-1,613k (- 0.48%)	331,325k	331,434k
Parse Time	2.03s (± 0.86%)	2.02s (± 0.60%)	-0.00s (- 0.15%)	2.01s	2.07s
Bind Time	0.88s (± 0.56%)	0.88s (± 0.56%)	0.00s ( 0.00%)	0.87s	0.89s
Check Time	5.57s (± 0.34%)	5.57s (± 0.48%)	-0.00s (- 0.04%)	5.52s	5.63s
Emit Time	6.26s (± 0.91%)	6.25s (± 0.40%)	-0.01s (- 0.13%)	6.21s	6.32s
Total Time	14.73s (± 0.51%)	14.72s (± 0.38%)	-0.01s (- 0.08%)	14.62s	14.88s
Compiler-Unions - node (v14.15.1, x64)
Memory used	194,104k (± 0.55%)	193,142k (± 0.62%)	-962k (- 0.50%)	191,104k	194,498k
Parse Time	0.85s (± 0.80%)	0.86s (± 0.55%)	+0.00s (+ 0.35%)	0.85s	0.87s
Bind Time	0.56s (± 1.15%)	0.56s (± 1.04%)	-0.01s (- 0.89%)	0.55s	0.57s
Check Time	7.44s (± 0.59%)	7.43s (± 0.48%)	-0.01s (- 0.15%)	7.35s	7.49s
Emit Time	2.50s (± 1.21%)	2.47s (± 0.72%)	-0.03s (- 1.28%)	2.43s	2.51s
Total Time	11.36s (± 0.63%)	11.31s (± 0.33%)	-0.04s (- 0.39%)	11.21s	11.37s
Monaco - node (v14.15.1, x64)
Memory used	325,203k (± 0.00%)	324,930k (± 0.00%)	-273k (- 0.08%)	324,903k	324,970k
Parse Time	1.56s (± 0.71%)	1.56s (± 0.71%)	0.00s ( 0.00%)	1.54s	1.59s
Bind Time	0.78s (± 1.07%)	0.77s (± 1.14%)	-0.00s (- 0.26%)	0.76s	0.80s
Check Time	5.48s (± 0.42%)	5.48s (± 0.42%)	+0.01s (+ 0.13%)	5.43s	5.54s
Emit Time	3.31s (± 0.74%)	3.32s (± 0.70%)	+0.01s (+ 0.33%)	3.26s	3.38s
Total Time	11.13s (± 0.40%)	11.14s (± 0.48%)	+0.02s (+ 0.16%)	11.02s	11.26s
TFS - node (v14.15.1, x64)
Memory used	288,770k (± 0.01%)	288,666k (± 0.01%)	-104k (- 0.04%)	288,606k	288,725k
Parse Time	1.33s (± 1.26%)	1.31s (± 0.81%)	-0.02s (- 1.43%)	1.29s	1.33s
Bind Time	0.73s (± 0.67%)	0.74s (± 0.93%)	+0.00s (+ 0.14%)	0.72s	0.75s
Check Time	5.14s (± 0.31%)	5.13s (± 0.44%)	-0.01s (- 0.21%)	5.07s	5.18s
Emit Time	3.53s (± 1.92%)	3.55s (± 2.00%)	+0.02s (+ 0.57%)	3.38s	3.68s
Total Time	10.72s (± 0.85%)	10.72s (± 0.68%)	-0.00s (- 0.01%)	10.54s	10.86s
material-ui - node (v14.15.1, x64)
Memory used	453,151k (± 0.00%)	450,137k (± 0.07%)	-3,013k (- 0.66%)	449,011k	450,364k
Parse Time	1.86s (± 0.55%)	1.85s (± 0.63%)	-0.00s (- 0.22%)	1.84s	1.89s
Bind Time	0.71s (± 1.05%)	0.71s (± 1.07%)	-0.00s (- 0.56%)	0.69s	0.72s
Check Time	20.58s (± 0.85%)	20.33s (± 0.75%)	-0.25s (- 1.20%)	20.06s	20.71s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	23.15s (± 0.76%)	22.90s (± 0.71%)	-0.25s (- 1.08%)	22.62s	23.32s
xstate - node (v14.15.1, x64)
Memory used	537,105k (± 0.00%)	667,197k (± 0.00%)	+130,092k (+24.22%)	667,131k	667,227k
Parse Time	2.59s (± 0.52%)	2.59s (± 0.54%)	+0.00s (+ 0.08%)	2.55s	2.62s
Bind Time	1.15s (± 0.59%)	1.15s (± 0.65%)	+0.00s (+ 0.09%)	1.14s	1.17s
Check Time	1.51s (± 0.57%)	2.96s (± 0.78%)	🔻+1.45s (+96.15%)	2.92s	3.03s
Emit Time	0.07s (± 4.13%)	0.07s (± 0.00%)	-0.00s (- 2.78%)	0.07s	0.07s
Total Time	5.32s (± 0.36%)	6.77s (± 0.45%)	+1.45s (+27.27%)	6.71s	6.84s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-210-generic
Architecture	x64
Available Memory	16 GB
Available Memory	5 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v14.15.1, x64)

Scenarios

• Angular - node (v14.15.1, x64)
• Compiler-Unions - node (v14.15.1, x64)
• Monaco - node (v14.15.1, x64)
• TFS - node (v14.15.1, x64)
• material-ui - node (v14.15.1, x64)
• xstate - node (v14.15.1, x64)

Benchmark	Name	Iterations
Current	48080	10
Baseline	main	10

Developer Information:

Download Benchmark

[typescript-bot]

@ahejlsberg
Great news! no new errors were found between main..refs/pull/48080/merge

[Andarist]

As mentioned - I'm very open to revisiting XState types to see if we can improve the situation anyhow there. I know that this is just a single library whereas this PR potentially impacts a lot more than that though. However, as a dev, I would always take a slower compilation over random errors - so this PR is a big improvement from my point of view.

Is there any rough description for the overall algorithm of how the variance is computed? If I understand correctly then:

• each type parameter has its own variance type
• to compute it we may need to traverse the whole type structurally, because if T is passed down through multiple layers then any layer can impact the variance type of the associated type parameter at the root level (and on each level in between)
• potentially when assessing that a type param is invariant you can return early from the algorithm since nothing else can change that
• this information can mostly be cached per uninstantiated type unless conditional types/mapped types do some weird-ish transformations under the hood (not saying that you are caching it, but that such a heuristic could potentially be used, I have no idea if it would be worth it - this is just me trying to understand the problem space better)

I wonder if there is any type construct that makes it impossible for you to take the associated "slot" into consideration? Like - I see how "makes" type params are taken into consideration, even when a type is "boxed" using some other type I can see how the relation gets transitive etc But when I think about crazy conditional & mapped types... I'm getting lost.

In somewhat other words - I wonder if there is anything that we possibly could do to make TS not descend into a particular branch of the structure?

[ahejlsberg]

@typescript-bot test this
@typescript-bot user test this inline
@typescript-bot run dt
@typescript-bot perf test faster

[typescript-bot]

Heya @ahejlsberg, I've started to run the diff-based community code test suite on this PR at 08c0fa9. You can monitor the build here.

Update: The results are in!

[typescript-bot]

Heya @ahejlsberg, I've started to run the parallelized Definitely Typed test suite on this PR at 08c0fa9. You can monitor the build here.

[typescript-bot]

Heya @ahejlsberg, I've started to run the abridged perf test suite on this PR at 08c0fa9. You can monitor the build here.

Update: The results are in!

[typescript-bot]

Heya @ahejlsberg, I've started to run the extended test suite on this PR at 08c0fa9. You can monitor the build here.

[typescript-bot]

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..48080

Metric	main	48080	Delta	Best	Worst
Angular - node (v14.15.1, x64)
Memory used	332,984k (± 0.01%)	331,341k (± 0.01%)	-1,644k (- 0.49%)	331,287k	331,384k
Parse Time	2.03s (± 0.86%)	2.02s (± 0.55%)	-0.00s (- 0.15%)	2.00s	2.05s
Bind Time	0.88s (± 0.56%)	0.87s (± 0.42%)	-0.00s (- 0.23%)	0.87s	0.88s
Check Time	5.57s (± 0.34%)	5.56s (± 0.61%)	-0.01s (- 0.14%)	5.48s	5.65s
Emit Time	6.26s (± 0.91%)	6.27s (± 0.42%)	+0.00s (+ 0.08%)	6.21s	6.32s
Total Time	14.73s (± 0.51%)	14.72s (± 0.39%)	-0.01s (- 0.05%)	14.61s	14.86s
Compiler-Unions - node (v14.15.1, x64)
Memory used	194,104k (± 0.55%)	193,782k (± 0.50%)	-322k (- 0.17%)	191,195k	194,498k
Parse Time	0.85s (± 0.80%)	0.85s (± 0.58%)	+0.00s (+ 0.12%)	0.85s	0.87s
Bind Time	0.56s (± 1.15%)	0.56s (± 0.65%)	+0.00s (+ 0.36%)	0.56s	0.57s
Check Time	7.44s (± 0.59%)	7.43s (± 0.72%)	-0.01s (- 0.09%)	7.31s	7.54s
Emit Time	2.50s (± 1.21%)	2.46s (± 0.84%)	-0.04s (- 1.48%)	2.43s	2.53s
Total Time	11.36s (± 0.63%)	11.32s (± 0.63%)	-0.04s (- 0.34%)	11.16s	11.49s
Monaco - node (v14.15.1, x64)
Memory used	325,203k (± 0.00%)	324,923k (± 0.00%)	-280k (- 0.09%)	324,892k	324,952k
Parse Time	1.56s (± 0.71%)	1.57s (± 0.56%)	+0.01s (+ 0.45%)	1.55s	1.59s
Bind Time	0.78s (± 1.07%)	0.77s (± 0.44%)	-0.00s (- 0.26%)	0.77s	0.78s
Check Time	5.48s (± 0.42%)	5.48s (± 0.54%)	+0.00s (+ 0.05%)	5.42s	5.55s
Emit Time	3.31s (± 0.74%)	3.30s (± 0.91%)	-0.01s (- 0.27%)	3.25s	3.39s
Total Time	11.13s (± 0.40%)	11.13s (± 0.52%)	-0.00s (- 0.00%)	11.02s	11.27s
TFS - node (v14.15.1, x64)
Memory used	288,770k (± 0.01%)	288,672k (± 0.01%)	-98k (- 0.03%)	288,630k	288,718k
Parse Time	1.33s (± 1.26%)	1.33s (± 2.43%)	+0.00s (+ 0.23%)	1.29s	1.44s
Bind Time	0.73s (± 0.67%)	0.74s (± 2.08%)	+0.00s (+ 0.68%)	0.73s	0.80s
Check Time	5.14s (± 0.31%)	5.14s (± 0.84%)	0.00s ( 0.00%)	5.06s	5.29s
Emit Time	3.53s (± 1.92%)	3.51s (± 2.26%)	-0.01s (- 0.43%)	3.37s	3.65s
Total Time	10.72s (± 0.85%)	10.72s (± 1.31%)	-0.00s (- 0.02%)	10.50s	11.07s
material-ui - node (v14.15.1, x64)
Memory used	453,151k (± 0.00%)	450,276k (± 0.03%)	-2,874k (- 0.63%)	449,715k	450,378k
Parse Time	1.86s (± 0.55%)	1.86s (± 0.53%)	+0.00s (+ 0.16%)	1.84s	1.88s
Bind Time	0.71s (± 1.05%)	0.71s (± 0.63%)	-0.00s (- 0.28%)	0.70s	0.72s
Check Time	20.58s (± 0.85%)	20.34s (± 1.00%)	-0.24s (- 1.16%)	20.00s	20.78s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	23.15s (± 0.76%)	22.91s (± 0.89%)	-0.24s (- 1.05%)	22.58s	23.35s
xstate - node (v14.15.1, x64)
Memory used	537,105k (± 0.00%)	666,424k (± 0.00%)	+129,319k (+24.08%)	666,374k	666,467k
Parse Time	2.59s (± 0.52%)	2.59s (± 0.76%)	+0.00s (+ 0.15%)	2.56s	2.64s
Bind Time	1.15s (± 0.59%)	1.15s (± 1.02%)	+0.00s (+ 0.26%)	1.12s	1.17s
Check Time	1.51s (± 0.57%)	2.95s (± 0.52%)	🔻+1.44s (+95.56%)	2.92s	2.99s
Emit Time	0.07s (± 4.13%)	0.07s (± 0.00%)	-0.00s (- 2.78%)	0.07s	0.07s
Total Time	5.32s (± 0.36%)	6.76s (± 0.30%)	+1.45s (+27.21%)	6.73s	6.82s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-210-generic
Architecture	x64
Available Memory	16 GB
Available Memory	5 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v14.15.1, x64)

Scenarios

• Angular - node (v14.15.1, x64)
• Compiler-Unions - node (v14.15.1, x64)
• Monaco - node (v14.15.1, x64)
• TFS - node (v14.15.1, x64)
• material-ui - node (v14.15.1, x64)
• xstate - node (v14.15.1, x64)

Benchmark	Name	Iterations
Current	48080	10
Baseline	main	10

Developer Information:

Download Benchmark

[typescript-bot]

@ahejlsberg
Great news! no new errors were found between main..refs/pull/48080/merge

[ahejlsberg]

@Andarist Here's a quick overview of the variance computation logic.

For a generic type T<A, B>, we determine the variance of A by structurally relating instantiations T<Sub, B> and T<Super, B>, where Sub and Super are internal marker types that have a known type relationship (i.e. Sub is a subtype of Super). Then

• if T<Sub, B> is related to T<Super, B>, but not vice versa, we know A to be covariant,
• if T<Super, B> is related to T<Sub, B>, but not vice versa, we know A to be contravariant,
• if neither is related to the other, we know A to be invariant, and
• if both are related to the other, we know A to be bivariant.

To determine the variance of B, we similarly structurally relate instantiations T<A, Sub> and T<A, Super>.

As we're relating these marker type instantiations, we may come upon other type instantiations for which we need to know the variance. If so, we recursively start a variance computation and use the results of that to relate the type arguments. We may also come upon a circular reference to an instantiation of a type for which we are already computing variance information. If so, we again structurally relate the instantiations, but only if some of the type arguments contain marker types. Ultimately, we stop relating after three levels of recursion.

In other to understand where time is spent in the variance computation logic you can use the --generateTrace compiler option. See instructions here. The xstate types appear to have a very deep dependency graph (some 35 levels), which generates quite a lot of work. Anything you can do to simplify that will likely help greatly.

This is what the recursive invocation tree looks like for getVariancesWorker when compiling xstate:

StateSchema[Partial[]]StatesConfig[StateNodeConfig[InvokeConfig[InvokeCreator[InvokeCallback[]PromiseLike[]StateMachine[StateNodesConfig[StateNode[ActionObject[ActionFunction[ActionMeta[State[Mapper[]PropertyMapper[]MachineOptions[Record[]ConditionPredicate[GuardMeta[GuardPredicate[ActivityDefinition[InvokeDefinition[ActionFunctionMap[ActivityConfig[DelayFunctionMap[DelayExpr[SCXMLEventMeta[Event[]]]]MachineSchema[]Map[IterableIterator[IteratorYieldResult[]]]TransitionDefinition[IteratorResult[IteratorReturnResult[]]Omit[Exclude[]]TransitionDefinitionMap[TransitionConfig[DelayedTransitionDefinition[SendAction[ExprWithMeta[]SendExpr[]AssignAction[Assigner[AssignMeta[PropertyAssigner[PartialAssigner[LogAction[LogExpr[]StopAction[Expr[]ChooseAction[ChooseConditon[Readonly[]StateNodeDefinition[StatesDefinition[StateTransition[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]SimpleActionsFrom[]]]Model[MachineConfig[]ExtractEvent[]Prop[]]Typestate[]RaiseAction[]Guard[]RaiseActionObject[]Action[]Interpreter[Set[]StateListener[]ContextListener[]Observer[]SendActionObject[]Subscribable[]]BaseActorRef[]ActorRef[Sender[]]Extract[]Event[]EventListener[]Promise[]Behavior[ActorContext[]]Iterable[Iterator[]]AdjList[]StateConfig[]ServiceConfig[]DelayConfig[]InvokeActionObject[]SingleOrArray[]AtomicStateNodeConfig[]

[ahejlsberg]

@typescript-bot pack this

[typescript-bot]

Heya @ahejlsberg, I've started to run the tarball bundle task on this PR at 367ef6d. You can monitor the build here.

[typescript-bot]

Hey @ahejlsberg, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{"devDependencies":{"typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/121427/artifacts?artifactName=tgz&fileId=3564D9FE4E925DEB072B7F2EA049AAB996BB172F4FA500BD881022D7F059E67E02&fileName=/typescript-4.7.0-insiders.20220307.tgz" } } 

and then running npm install.

---

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

[Andarist]

I can confirm that both of my issues related to this are being fixed with the latest build of this PR. However, I don't see an explicit test case for this being introduced here. The minimal repro created by me can be found here. From what I understand this has been fixed within this diff but as we can see there - no new tests have been added as part of this. Perhaps this is actually tested by the changed baselines - but assessing that is over my head.

[ahejlsberg]

Sadly we won't be able to merge this PR. While it improves accuracy of variance measurement, the adverse effects on type checking performance are simply too great. For example, the check time for the fairly popular immutable.js framework degrades by 50x with this PR in its current form. For now, we'll have to accept that variance measurement for circular types is sometimes inaccurate and/or dependent on declaration order. However, with #48240 it becomes possible to add variance annotations to establish the correct variance in such cases.