https://github.com/golang/go/issues/57644
This is a speculative issue based on the way that type parameter constraints are implemented. This is a discussion of a possible future language change, not one that will be adopted in the near future. This is a version of #41716 updated for the final implementation of generics in Go.
We currently permit type parameter constraints to embed a union of types (see https://go.dev/ref/spec#Interface_types). We propose that we permit an ordinary interface type to embed a union of terms, where each term is itself a type. (This proposal does not permit the underlying type syntax ~T to be used in an ordinary interface type, though of course that syntax is still valid for a type parameter constraint.)
That's really the entire proposal.
Embedding a union in an interface affects the interface's type set. As always, a variable of interface type may store a value of any type that is in its type set, or, equivalently, a value of any type in its type set implements the interface type. Inversely, a variable of interface type may not store a value of any type that is not in its type set. Embedding a union means that the interface is something akin to a sum type that permits values of any type listed in the union.
For example:
type MyInt int
type MyOtherInt int
type MyFloat float64
type I1 interface {
MyInt | MyFloat
}
type I2 interface {
int | float64
}
The types MyInt and MyFloat implement I1. The type MyOtherInt does not implement I1. None of MyInt, MyFloat, or MyOtherInt implement I2.
In all other ways an interface type with an embedded union would act exactly like an interface type. There would be no support for using operators with values of the interface type, even though that is permitted for type parameters when using such a type as a type parameter constraint. This is because in a generic function we know that two values of some type parameter are the same type, and may therefore be used with a binary operator such as +. With two values of some interface type, all we know is that both types appear in the type set, but they need not be the same type, and so + may not be well defined. (One could imagine a further extension in which + is permitted but panics if the values are not the same type, but there is no obvious reason why that would be useful in practice.)
In particular, the zero value of an interface type with an embedded union would be nil, just as for any interface type. So this is a form of sum type in which there is always another possible option, namely nil. Sum types in most languages do not work this way, and this may be a reason to not add this functionality to Go.
As an implementation note, we could in some cases use a different implementation for interfaces with an embedded union type. We could use a small code, typically a single byte, to indicate the type stored in the interface, with a zero indicating nil. We could store the values directly, rather than boxed. For example, I1 above could be stored as the equivalent of struct { code byte; value [8]byte } with the value field holding either an int or a float64 depending on the value of code. The advantage of this would be reducing memory allocations. It would only be possible when all the values stored do not include any pointers, or at least when all the pointers are in the same location relative to the start of the value. None of this would affect anything at the language level, though it might have some consequences for the reflect package.
As I said above, this is a speculative issue, opened here because it is an obvious extension of the generics implementation. In discussion here, please focus on the benefits and costs of this specific proposal. Discussion of sum types in general, or different proposals for sum types, should remain on #19412 or newer variants such as #54685. Thanks.
This proposal does not permit the underlying type syntax
~Tto be used in an ordinary interface type, though of course that syntax is still valid for a type parameter constraint.
Could you comment on why this restriction occurs? Is this simply to err on the side of caution initially and potentially remove this restriction in the future? Or is there a technical reason not to do this?
The reason to not permit ~T is that the current language would provide no mechanism for extracting the type of such a value. Given interface { ~int }, if I store a value of type myInt in that interface, then code in some other package would be unable to use a type assertion or type switch to get the value out of the interface type. The best that it could do would be something like reflect.TypeOf(v).Kind(). That seems sufficiently awkward that it requires more thought and attention, beyond the ideas in this proposal.
Is there a technical reason that the language could not also evolve to support ~T in a type switch? Granted that this is outside the scope of this proposal, but I think there is a valid use case for it.
In a vacuum, I'd prefer pretty much any other option, but since it's what generics use, it's what we should go with here and we should embrace it fully. Specifically,
1. type I2 int | float64 should be legal
2. v, ok := i.(int | float64) follows from 1
3. in a type switch case int | float64: works like 2
4. string | fmt.Stringer should be legal even though that does not currently work with constraints
@dsnet I think comparable and ~T could be considered and discussed separately—if for no reason other than this thread will probably get quite long on its own. I'm :+1: on both.
With the direct storage mechanism detailed in the post as an alternative to boxing, would it be possible for the zero-value not to be nil after all? For example, if the code value is essentially an index into the list of types and the value stores the value of that type directly, then the zero value with all-zeroed memory would actually default to a zero value of the first type in the list. For example, given
type Example interface {
int16 | string
}
the zero value in memory would look like {code: 0, value: 0}.
~~Also, in that format, would the value side change sizes depending on the type? For example, would a value of Example(1) look like {code: 0, value: [...]byte{0, 1}) ignoring endianess, while a value of Example("example") would look like {code: 1, value: [...]byte{/* raw bytes of a string header */}}? If so, how would this affect embedding these interface types into other types, such as a []Example? Would the slice just assume the maximum possible necessary size for the given types?~~ Edit: Never mind, dumb question. The size changing could be a minor optimization when copying, but of course anywhere it's stored would have to assume the maximum possible size, even just local variables, unless the compiler could prove that it's only ever used with a smaller type, I guess.
It would only be possible when all the values stored do not include any pointers, or at least when all the pointers are in the same location relative to the start of the value.
I don't understand this comment, which may indicate that I'm missing something fundamental about the explanation. Why would pointers make any difference? If the above Example type had int16 | string | *int, why would it not just be {code: 2, value: /* the pointer value itself, ignoring whatever it points to */}?
The example in the proposal is rather contrived, so I tried to imagine some real situations I've encountered where this new capability could be useful to express something that was harder to express before.
Is the following also an example of something that this proposal would permit?
type Success[T] struct {
Value T
}
type Failure struct {
Err error
}
type Result[T] interface {
Success[T] | Failure
}
func Example() Result[string] {
return Success[string]{"hello"}
}
(NOTE WELL: I'm not meaning to imply that the above would be a good idea, but it's the example that came most readily to mind because I just happened to write something similar -- though somewhat more verbose -- to smuggle (result, error) tuples through a single generic type parameter yesterday. Outside of that limited situation I expect it would still be better to return (string, error).)
Another example I thought of is encoding/json's Token type, which is currently defined as type Token any and is therefore totally unconstrained.
Although I expect it would not be appropriate to change this retroactively for compatibility reasons, presumably a hypothetical green field version of that type could be defined like this instead:
type Token interface {
Delim | bool | float64 | Number | string
// (json.Token also allows nil, but since that isn't a type I assume
// it wouldn't be named here and instead it would just be
// a nil value of type Token.)
}
Given that the exact set of types here is finite, would we consider it to be a breaking change to add new types to this interface later? If not, that could presumably allow the following to compile by the compiler noticing that the case labels are exhaustive:
// TokenString is a rather useless function that's just here to illustrate an
// exhaustive type switch...
func TokenString(t Token) string {
switch t := t.(type) {
case Delim:
return string(t)
case bool:
return strconv.FormatBool(t)
case float64:
return strconv.FormatFloat(t, 'g', -1, 64)
case Number:
return string(t)
case string:
return string
}
}
I don't feel strongly either way about whether such sealed interfaces should have this special power, but it does seem like it needs to be decided either way before implementation because it would be hard to change that decision later without breaking some existing code.
Even if this doesn't include a special rule for exhaustiveness, this still feels better in that it describes the range of Decoder.Token() far better than any does.
EDIT: After posting this I realized that my type switch doesn't account for nil. That feels like it's a weird enough edge that it probably wouldn't be worth the special case of allowing exhaustive type-switch matching.
Finally, it seems like this would shrink the boilerplate required today to define what I might call a "sealed interface", by which I mean one which only accepts a fixed set of types defined in the same package as the interface.
One way I've used this in the past is to define struct types that act as unique identifiers for particular kinds of objects but then have some functions that can accept a variety of different identifier types for a particular situation:
type ResourceID struct {
Type string
Name string
}
type ModuleID struct {
Name string
}
type Targetable interface {
// Unexported method means that only types
// in this package can implement this interface.
targetable()
}
func (ResourceID) targetable() {}
func (ModuleID) targetable() {}
func Target(addr Targetable) {
// ...
}
I think this proposal could reduce that to the following, if I've understood it correctly:
type ResourceID struct {
Type string
Name string
}
type ModuleID struct {
Name string
}
type Targetable interface {
ResourceID | ModuleID
}
func Target(addr Targetable) {
// ...
}
If any of the examples I listed above don't actually fit what this proposal is proposing (aside from the question about exhaustive matching, which is just a question), please let me know!
If they do, then I must admit I'm not 100% convinced that the small reduction in boilerplate is worth this complexity, but I am leaning towards :+1: because I think the updated examples above would be easier to read for a future maintainer who is less experience with Go and so would benefit from a direct statement of my intent rather than having to infer the intent based on familiarity with idiom or with less common language features.
@dsnet Sure, we could permit case ~T in a type switch, but there are further issues. A type switch can have a short declaration, and in a type switch case with a single type we're then permitted to refer to that variable using the type in the case. What type would that be for case ~T? If it's T then we lost the methods, and fmt.Printf will behave unexpectedly if the original type had a String method. If it's ~T what can we do with a value of that type? It's quite possible that these questions can be answered, but it's not just outside the scope of this proposal, it's actually complicated.
@DeedleFake The alternative implementation is only an implementation issue, not a language issue. We shouldn't use that to change something about the language, like whether the value can be nil or some other zero value. In Go the zero value of interface types is nil. It would be odd to change that for the special case of interfaces that embed a union type element.
The reason pointer values matter is that given a value of the interface type, the current garbage collector implementation has to be able to very very quickly know which fields in that value are pointers. The current implementation does this by associating a bitmask of pointers with each type, such that a 1 in the bitmask means that the pointer-sized slot at that offset in the value always holds a pointer.
@apparentlymart I think that everything you wrote is correct according to this proposal. Thanks.
In Go the zero value of interface types is
nil. It would be odd to change that for the special case of interfaces that embed a union type element.
It would be, but I think it would be worth it. And I don't think it would be so strange as to completely preclude eliminating the extra oddness that would come from union types always being nilable. In fact, I'd go so far as to say that if this way of implementing unions has to have them be nilable, then a different way of implementing them should be found.
The reason pointer values matter is that given a value of the interface type, the current garbage collector implementation has to be able to very very quickly know which fields in that value are pointers.
I was worried it was going to be the garbage collector... Ah well.
A major problem is that type constraints work on static types while interfaces work on dynamic types of objects. This immediately prohibits this approach to do union types.
type Addable interface {
int | float32
}
func Add[T Addable](x, y T) T {
return x + y
}
This works because the static type of T can only be int or float, which means the addition operation is defined for all the type set of T. However, if we allow Addable to be a sum type, then the type set of T becomes {int, float, Addable} which does not satisfy the aforementioned properties!!!
@merykitty per my understanding of the proposal, I think for the dynamic form of what you wrote you'd be expected to write something this:
type Addable interface {
int | float32
}
func Add(x, y Addable) Addable {
switch x := x.(type) {
case int:
return x + y.(int)
case float32:
return x + y.(float32)
default:
panic("unsupported Addable types %T + %T", x, y)
}
}
Of course this would panic if used incorrectly, but I think that's a typical assumption for interface values since they inherently move the final type checking to runtime.
I would agree that the above seems pretty unfortunate, but I would also say that this feels like a better use-case for type parameters than for interface values and so the generic form you wrote is the better technique for this (admittedly contrived) goal.
@merykitty No, in your example, Addable itself should not be able to instantiate Add. Addable does not implement itself (only int and float32 do).
also, note that the type set never includes interfaces. So Addable is never in its own type set.
Is something like that going to be allowed?
type IntOrStr interface {
int | string
}
func DoSth[T IntOrStr](x T) {
var a IntOrStr = x
_ = a
}
Let's say I have these definitions.
type I1 interface {
int | any
}
type I2 interface {
string | any
}
type I interface {
I1 | I2
}
Would it be legal to have a variable of type I? Can I assign an I1 to it? What about string? any(int8)? int8?
@mateusz834 Can't see why not.
Would it be legal to have a variable of type
I? Can I assign anI1to it? What aboutstring?any(int8)?int8?
I think the answer to all of these is "yes". For the cases where you assign an interface value, the dynamic type/value of the I variable would then become the dynamic type/value of the assigned interface. In particular, the dynamic type would never be an interface.
FWIW my main issue with this proposal is that IMO union types should allow representing something like ~string | fmt.Stringer , but for well-known reasons this isn't possible right now and it's not clear it ever would be. One advantage of "real" sum types is that they have an easier time representing that kind of thing. Specifically, I don't think #54685 has that problem (though it's been a spell that I looked at that proposal in detail).
I think this approach is elegant given that type sets on constraints already exist, and so for any union discriminated only by types this seems almost perfect.
I think there are three short comings of the proposal that would prevent it from being usable in many of the cases where I currently construct union-like structures.
nilIn particular, the zero value of an interface type with an embedded union would be
nil, just as for any interface type. So this is a form of sum type in which there is always another possible option, namelynil. Sum types in most languages do not work this way, and this may be a reason to not add this functionality to Go.
This is mentioned in the proposal, and I think this constraint simplification is problematic. In Go default values are useful, but by making sum types nillable, we make their default value not useful. Of course maybe this is reasonable given that Go has no widely used "optional" value type beyond pointers.
To address this shortcoming, could we make interface types that contain type sets non-nullable by default, and require an explicit nil | in the type set list. For type sets that do not specify nil, the default value of the interface value would be the zero value of the first type listed.
The proposal defines a discriminated union where the discriminant is always the types of each case in the union. This prevents applications from creating unions where the same type appears across multiple cases but with different semantics. This happens a lot in code where I write union-like types today, and I don't think I could use this proposal for most of my union cases without it.
Here's an example of a union-like structure from some code I have.
```go
type ClaimPredicateType int32
const (
ClaimPredicateTypeClaimPredicateUnconditional ClaimPredicateType = 0
ClaimPredicateTypeClaimPredicateAnd ClaimPredicateType = 1
ClaimPredicateTypeClaimPredicateOr ClaimPredicateType = 2
ClaimPredicateTypeClaimPredicateNot ClaimPredicateType = 3
ClaimPredicateTypeClaimPredicateBeforeAbsoluteTime ClaimPredicateType = 4
ClaimPredicateTypeClaimPredicateBeforeRelativeTime ClaimPredicateType = 5
)
type ClaimPredicate struct {
Type ClaimPredicateType
AndPredicates []ClaimPredicate xdrmaxsize:"2"
OrPredicates []ClaimPredicate xdrmaxsize:"2"
NotPredicate ClaimPredicate
AbsBefore Int64
RelBefore Int64
}
```
Ref: https://github.com/stellar/go/blob/b4ba6f8e6/xdr/xdr_generated.go#L5815-L5822
The proposal would allow only for writing the following case, which would fail to represent the complete union type:
go
type ClaimPredicate interface {
[]ClaimPredicate | ClaimPredicate | Int64
}
I have the same type in a few languages, and here's the same type in Rust:
rust
pub enum ClaimPredicate {
Unconditional,
And(VecM<ClaimPredicate, 2>),
Or(VecM<ClaimPredicate, 2>),
Not(Option<Box<ClaimPredicate>>),
BeforeAbsoluteTime(i64),
BeforeRelativeTime(i64),
}
Ref: https://github.com/stellar/rs-stellar-xdr/blob/154e07ebb/src/curr/generated.rs#L6672-L6679
To address this shortcoming could the type set be a type list where each type in the list is also given a field name? This doesn't feel good, but it's the only way I see to address this inside the proposal in its current form. It's not clear to me how this would work in a switch statement as well. For example:
go
type ClaimPredicate interface {
and []ClaimPredicate |
or []ClaimPredicate |
not ClaimPredicate |
beforeAbsoluteTime Int64 |
beforeRelativeTime Int64
}
Sometimes discriminated unions have cases where no data is required. I don't think the proposal supports this. The example in point 2 above has one case like that, the Unconditional case. If such a thing was supported, it could be like:
go
type ClaimPredicate interface {
unconditional void |
and []ClaimPredicate |
or []ClaimPredicate |
not ClaimPredicate |
beforeAbsoluteTime Int64 |
beforeRelativeTime Int64
}
@ianlancetaylor Does the proposal as-is allow both type sets and functions in an interface? It would have a remarkable property not typically present in sum types where you could have a closed set of types along with the ability to have those types implement some common functions and be used as an interface.
To address this shortcoming, could we make interface types that contain type sets non-nullable by default, and require an explicit
nil |in the type set list. For type sets that do not specify nil, the default value of the interface value would be the zero value of the first type listed.
For reference, this has been suggested a few times in #19412 and #41716, starting with https://github.com/golang/go/issues/19412#issuecomment-288485048. Requiring nil variants versus allowing source code order to affect semantics is the classic tension of sum types proposals.
Sometimes discriminated unions have cases where no data is required. I don't think the proposal supports this.
The spelling of a type with no information beyond existence is usually struct{}, or more generally any type with exactly one value. void, i.e. the zero type, means something different: logically it would represent that your unconditional variant is impossible, not that it carries no additional information.
Does the proposal as-is allow both type sets and functions in an interface? It would have a remarkable property not typically present in sum types where you could have a closed set of types along with the ability to have those types implement some common functions and be used as an interface.
Yes, since the proposal is just to allow values of general interfaces less ~T elements, methods would be fine and would dynamically dispatch to the concrete type. I agree that's a neat behavior. Unfortunately it does imply that methods can't be defined on a sum type itself; you'd have to wrap it in a struct or some other type.
Thanks @zephyrtronium. Taking your feedback into account, and also realizing that it is easy to redefine types, then I think points (2) and (3) I raised are not issues. Type definitions can be used to give the same type different semantics for each case. For example:
type ClaimPredicateUnconditional struct{}
type ClaimPredicateAnd []ClaimPredicate
type ClaimPredicateOr []ClaimPredicate
type ClaimPredicateNot ClaimPredicate
type ClaimPredicateBeforeAbsoluteTime Int64
type ClaimPredicateBeforeRelativeTime Int64
type ClaimPredicate interface {
ClaimPredicateUnconditional |
ClaimPredicateAnd |
ClaimPredicateOr |
ClaimPredicateNot |
ClaimPredicateBeforeAbsoluteTime |
ClaimPredicateBeforeRelativeTime
}
In the main Go code base I work in we have 106 unions implemented as multi-field structs, which require a decent amount of care to use. I think this proposal would make using those unions easier to understand, probably on par in terms of effort to write. If tools like gopls went on to support features like pre-filling out the case statements of a switch based on the type sets, since it can know the full set, that would make writing code using them easier too.
The costs of this proposal feel minimal. Any code using the sum type would experience the type as an interface and have nothing new to learn over that of interfaces. This is I think the biggest benefit of this proposal.
To me, nil seems to be the big question here?
On the one hand, interface types are nilable and their zero value is nil.
On the other hand, union interface constraints made only of non-nilable types prevent a T from being nil, and that behaviour seems useful here as well. Is it that big a can of worms to say these can't be nil?
Exhaustiveness in type switches could potentially be left to tools.
Is it that big a can of worms to say these can't be
nil?
And instead, they are what? The reason to use nil is that it's precedented for "there is no dynamic type to this interface". If you don't want to use nil, you'd at least have to say what the dynamic type of a union is.
More general, there are essentially four choices around the zero value of a union type:
make([]T, …), map-indexing of non-existing keys, receiving from a closed channel…) and to a lesser degree, that it's represented by all zero bits.nil. The downside is, that any union value has an additional case.This proposal makes the last choice and it seems to me, that's a pretty foundational choice to any union/sum type proposal. So, from the proposal text:
In discussion here, please focus on the benefits and costs of this specific proposal. Discussion of sum types in general, or different proposals for sum types, should remain on #19412 or newer variants such as #54685. Thanks.
So we should, in this discussion, assume that the choice of zero value is fixed as nil and not try to come up with alternative designs. If we dislike a separate nil zero value, then that's simply a reason to reject this proposal:
In particular, the zero value of an interface type with an embedded union would be
nil, just as for any interface type. So this is a form of sum type in which there is always another possible option, namelynil. Sum types in most languages do not work this way, and this may be a reason to not add this functionality to Go.
OK. If we are to leave it at that, then yes, IMO, this is a reason to reject the proposal.
I still think it might be worth discussing here why that's the best choice. Detractors might be persuaded that this is in fact the best choice.
PS: I suppose I find your 3rd option best, and not counter intuitive; but it's a different proposal, and I won't discuss it here if it's considered off topic.
In principle, I'm in favor of this proposal. Especially since it seems nicely orthogonal.
Just a few concerns that might require further thoughts as was mentioned:
~T : a quick idea off top, would be for this to be a special anonymous interface analogous to basic interfaces where instead of a method set, an underlying type T is specified. In which case, to retrieve the initial type, one would have to switch over as usual. Might also allow conversion to T. nil as the zero value is fine, provided a union interface value where the runtime.type pointer is nil cannot be assigned. It has some complexities wrt slices (make?) , the recently added clear, and perhaps a few other things (or not, I don't know). But if it works, that will be quite a nice fit. :)Especially since the implementation of an interface is inscrutable, even when aliased, an interface value internal representation cannot be changed back to nil(?) , so for now I'm optimistic.
(just an example)
A library I wrote needed to limit a function parameter to any of the Go types that can be sent over to JavaScript world via wasm (bool, string, float64, []any, map[string]any).
It's manageable without union types but the API is not as nice as it could be as it requires plenty conversions.
(had to use the trick of defining a Value interface with an unexported method, to be implemented by type Bool bool, type Float float64, etc...)
Also related to marshalling.
I still think it might be worth discussing here why that's the best choice.
I don't think that's the claim. It's just what's proposed here. This is not the only proposal of its kind.
provided a union interface value where the runtime.type pointer is nil cannot be assigned.
That seems impossible, or close to.
type I interface { int | string }
func F(p *I) {
var v I
if someProgramHalts() {
*p = v
}
}
So, the only way to do that, AFAICT, would be to make such interfaces not assignable to their own type. Which means, they can't be passed around as arguments either. Or used in many other places.
I don't believe this is workable at all.
Well that's the point.
If a variable of this type is declared but not assigned a proper value, it shouldn't be usable.
The one issue would be channels for example, one would have to find a palatable way to deal with channel closures.
One way could be to make nilability opt-in explicitly in those cases:
chan(int | string) // disallowed
chan(int | string | nil) // allowed and actually a supertype of the above, nil being sent on close
That would keep people from assigning the nilable supertype to the actual union type.
They would not exactly be the same type and a regular type assertion check would have to happen.
Anyway, this is just an idea that someone can work with and ponder, I'll leave it at that.
I hadn't thought of it before writing https://github.com/golang/go/issues/57644#issuecomment-1374380626, but I think being unable to define methods on these sum types is a major downside. If I define a type like type Parameter interface { int | string } then I can't make a String or UnmarshalText method for it. (Most encoders can marshal it without issue since it would be like having an int or string in an any, but because there's no reflection interface in the proposal, there's no way to automatically unmarshal.) I could define it as this instead:
type Parameter interface {
IntParam | StringParam
String() string
UnmarshalText([]byte) error
}
Then I need to define a separate type for each variant as well as the methods those types need, and I have to use IntParam and StringParam instead of just int and string. It seems like this is the code I would write already, except that I write a union instead of an unexported method. The only thing we've gained is that the implementing types show up in godoc.
Instead, I could write this:
type Parameter struct {
F interface{ int | string }
}
func (p Parameter) String() string
func (p *Parameter) UnmarshalText(text []byte) error
Now we gain some of the advantages of typical sum types, but first the author has to know that this is a good approach, and then we have to use the struct box instead of the interface value, UnmarshalText needs two layers of indirection (unless we choose a less conservative algorithm for unboxed sum type representations), and we lose some of the nice properties of interfaces like implicit conversions for assignments. Maybe those penalties aren't that bad overall, but I can imagine situations where they would push me toward a different design. And again, this looks very similar to code I might write today.
All of this would be a non-issue if we could define methods on interfaces. That was already rejected in #39799.
To me, nil seems to be the big question here?
I think the interface value being nillable is fine. I think if we wanted a way to make interface values not nil by default, there could be a separate proposal for that, and it would interact well with this. We don't need to solve that problem as part of this proposal.
Exhaustiveness in type
switches could potentially be left to tools.
I think whether switch is exhaustive is an entirely independent proposal to sum types. It can be proposed independent of any type changes, and it doesn't need to be attached to this proposal.
It would be nice to replace something like this
func (*VM) PushInt(i int)
func (*VM) PushString(s string)
func (*VM) PushEtc(etc *Etc)
with something like this
func (*VM) Push(v int | string | *Etc)
go func (*VM) Push(v int | string | *Etc)
I assume an anonymous interface would work:
func (*VM) Push(v interface { int | string | *Etc})
A | B | C is shorthand for interface { A | B | C } in generics code and I'm an advocate for that the same rule applying here
A | B | Cis shorthand forinterface { A | B | C }in generics code and I'm an advocate for that the same rule applying here
That sounds good, but it leads to this oddity:
func F1(int) { ... }
func F2(string) { ... }
func F3(int | string) { ... }
F1(nil) // Error.
F2(nil) // Error.
F3(nil) // Valid.
A benefit of this proposal is that it simplifies code that consumes an interface that attempts to do the same thing with interfaces today, especially for the consumer using the types.
Without this proposal:
type RGB struct {
R byte
G byte
B byte
}
func (RGB) isSumType() {}
type CMYK struct {
C byte
M byte
Y byte
K byte
}
func (CMYK) isSumType() {}
type Color interface{ isSumType() }
func PrintColor(c Color) {
switch v := c.(type) {
case nil:
fmt.Println("nil")
case RGB:
fmt.Println(v.R, v.G, v.B)
case *RGB:
if v == nil {
fmt.Println("RGB(nil)")
} else {
fmt.Println(v.R, v.G, v.B)
}
case CMYK:
fmt.Println(v.C, v.M, v.Y, v.K)
case *CMYK:
if v == nil {
fmt.Println("CMYK(nil)")
} else {
fmt.Println(v.C, v.M, v.Y, v.K)
}
}
}
With this proposal there are less surprising cases, like the fact that the type and pointer-type have to be included in the switch above.
type RGB struct {
R byte
G byte
B byte
}
type CMYK struct {
C byte
M byte
Y byte
K byte
}
func PrintColor(c interface { RGB | CMYK }) {
switch v := c.(type) {
case nil:
fmt.Println("nil")
case RGB:
fmt.Println(v.R, v.G, v.B)
case CMYK:
fmt.Println(v.C, v.M, v.Y, v.K)
}
}
@leighmcculloch The point of methods and interfaces is to avoid all this type switchery within a function. In your example the types RGB and CMYK could have a Print method, which can be part of an interface if needed. The concept of an "interface" is that types have something in common. This proposal counteracts this concept, because it allows to fit types through the same hole that have nothing in common, that's why I don't like it very much. I want less type switches in code, not more.
Putting sum types under the umbrella of interfaces is almost comedic. You almost always have to treat them with a type switch. If recipients need to inspect the things they receive and have to treat them differently, then it's not an interface, it's the opposite, it's a nuisance.
Type union elements when used as a constraint for type parameters, on the other hand, are ok to be called "interfaces", because we want to access their common operators (like +, * etc.), which is in the spirit of the "interface" concept (to treat things uniformly). And here we're not encouraged to type switch on them, because it's not supported (unless we convert it to 'any' first).
@gophun I don't see why you cannot treat them as normal interfaces with methods? The only difference between a normal interface and a union interface is that the former is open to inheritance while the dynamic type set of the latter is closed.
FWIW, I find much of it hard to understand, and Go is supposed to be simple. I agree with @gophun that it would be a petty if a new language feature made ubiquitous type switches necessary – I consider them “last resort” and not a good general pattern.
But I must also admit that I still have not understood the actual use cases. I’ve never had large Go code bases to maintain. Maybe explaining the Target function in https://github.com/golang/go/issues/57644#issuecomment-1373008575 would make it clearer to me.
My own approach has been: If I need type flexibility, use interfaces if methods are shared and embedding if attributes are shared. And in the remaining edge cases, I use any as the argument type and, well, use a type switch. Of course some errors are then caught in the tests rather than the compilation step, but is this disadvantageous enough that we should make interfaces even more complicated?
I think it's a feature that will be more useful for library writers who need to expose a given interface.
So basically, it should allow for improvements in coder UX.
I expect that the consumer of a library will often be oblivious to the internal type switching.
Another example of such usefulness is when defining a tree datastructure where nodes can be a handful of very specific types.
Yes but I still want to understand the motivation better. (Besides, I also write or will write libraries.)
Let be ask something very specific: If you use a sum type instead of any in a function signature, you obviously gain the compile-time check that the given type is one of the types included into the union. I also read in this issue that with sum types, there is the possibility to enhance the Go compiler or linters to see whether your type switches are complete. What are further advantages of sum types over any?
@bronger Some advantages that I can take from the top of my mind:
1, Sum types are also interfaces and can have common behaviours expressed through methods
2, The type sets of sum types are known, allowing better layout and improved performance
3, Similar to normal interfaces, they express intents regarding the signature of the function, instead of relying on reading the implementation details and documentations
1, Sum types are also interfaces and can have common behaviours expressed through methods
You mean, additionally to type unions, an interface defines some common methods?
@bronger WDYM? From the proposal
In all other ways an interface type with an embedded union would act exactly like an interface type.
Which means we can define a union as
type Vehicle interface {
Car | Bicycle
Go()
}
But then, your point (1) is not an advantage because this is also possible with any.
type Vehicle interface { Car | Bicycle Go() }
This interface is unnecessarily specific. It should just be
type Vehicle interface {
Go()
}
The other version unnecessarily limits the types an outsider can use, and you would need a type switch to discriminate between the two allowed types. The point of interfaces is to treat different things uniformly and do give an outsider the possibility to provide their own types that implement the interface. In this example the outsider can no longer add a Boat to implement the Vehicle interface.
@gophun That's just an example to show that a sum type also has methods.
The point of interfaces is to treat different things uniformly
This does not in any way say how they are treated to show the uniformity to the outside world. A dynamic dispatch is as valid as an explicit type switch.
and do give an outsider the possibility to provide their own types that implement the interface
No you just made this up, there are interfaces out there that intentionally declare private methods so that other packages cannot implement them. An interface just declares a contract, and a contract can involve no unexpected implementation.
The most commonly mentioned use-case for sum/union types are AST packages. For example, go/ast.Node is currently an interface, with a bunch of methods. But that definition is obviously wrong. For example, the type struct { ast.Node } also satisfies the interface and has all the necessary methods, but it's certainly not intended to be usable, from the point of view of the ast package.
This becomes a problem when you then pass this into ast.Walk, for example. Walk is implemented as a big type switch over all dynamic types which are expected to be possible Nodes. But the set of possible Nodes is infinite, so the (static) type of ast.Walk is really incorrect - there has to be additional type-checking at runtime.
There are several different ways to fix this:
ast package to enumerate all the valid Node types and be done with it. Nothing else needs to happen.Walk(f func(Visitor, Node)) (or something) method. But then, what about go/format? Or go/types? Or third party Go tools? These also commonly accept an ast.Node (or similar) to do their thing and they are not at liberty to specify that a Node has to have additional methods. So they still have to do the type-switch and runtime type checking.Node a struct, instead of an interface. It could look like type Node struct { comment *Comment; commentGroup *CommentGroup; /* … */ } with a field per possible case. As long as the ast package only creates Nodes with a single of these fields set, it can treat it as a sum. And a struct type can't be "subtyped". However, this has performance problems (there are a lot of possible Nodes, so you need to carry around a lot of pointers, almost all are nil). It's also, technically, still a bit prone to programmer error, as the ast package itself could contain a bug, creating a Node with more than one set field.So union/sum types are not the only way to solve this. And it might still be questioned if this problem needs solving and if the cost is justified to solve it. But they are a relatively common solution to this kind of problem, in other languages.
So,
type Vehicle interface {
Car | Bicycle
Go()
}
is for the use case that e.g. a function wants to call methods, and additionally do type-switching stuff with the argument?
@merykitty No, in your example, Addable itself should not be able to instantiate Add. Addable does not implement itself (only int and float32 do).
@Merovius So what you mean is that an Addable does not satisfy itself during generics parameter resolution but it will do during runtime assignments? That seems a little confusing to me.
@merykitty Maybe. It's already a situation we will be in with Go 1.20 and comparable. So there is precedent for these two things to be different. I also don't think it's an entirely natural idea for these to mean different things in different contexts - that is, an Addable variable is "an opaque box that can hold any of these types" while an Addable constraint is "the type argument must be any of these types".
So, yes, I think there is a certain amount of possible confusion here. But I'm not sure how confusing it'll be, how often it will be a problem and I'm not sure it's avoidable. Surprisingly, there are things which are confusing if you think about them, but if you don't, you just never notice. For example, I doubt most Go programmers could really explain why they can't use a bytes.Buffer as an io.Reader, even though they can call r.Read on it - but in practice, they manage to use it just fine.
@merykitty
you can see it as Addable not satisfying itself in both cases (the constraint being that something should be either int and float32)
It should implement itself however (the Addable type implements the same constraint (i.e. enforce the same contract as itself)) .
Note that interface{int} also implements Addable as it merely enforces the contract more strictly: not only arguments are int or string, but we know for a fact that they have to be int.
Modulo nil, which I'm optimistic (or at least hopeful) can be solved.
This is similar to a ReaderWriter interface implementing the Reader interface. (subtyping)
@merykitty FWIW as an analogy: It also doesn't seem like many people are confused that an io.Reader variable can't contain an io.Reader - that is, the dynamic type of an interface is never an interface itself. It's essentially the same situation, it's the same confusion, yet in actual practice no one really wonders why that is.
If we had:
type Vehicle interface {
Car | Bicycle
}
type Mover interface {
Move()
}
would we say that Vehicle satisfies Mover if each element of Vehicle (Car.Move, and Bicycle.Move) do as well?
@AndrewHarrisSPU As I interpret the proposal, you'd need to have a Move() function on Vehicle so that Vehicle types implement Mover.
type Vehicle interface {
Car | Bicycle
Move()
}
type Mover interface {
Move()
}
Or you could do:
type Vehicle interface {
Car | Bicycle
Mover
}
type Mover interface {
Move()
}
If we had:
```
type Vehicle interface {
Car | Bicycle
}type Mover interface {
Move()
}
```would we say that
VehiclesatisfiesMoverif each element ofVehicle(Car.Move,andBicycle.Move) do as well?
Worth noting that that does not work with constraints currently, either: https://go.dev/play/p/TLkZkYzOcdO
I think it makes sense for it not to work. It would be quite confusing and annoying to have to go searching through every type listed and figuring out the intersection of their available methods to see what you could do with it. Instead, just rely on the principle of defining interfaces where they're used and add the expected methods to the interface manually, which should then work.
Worth noting that that does not work with constraints currently, either: https://go.dev/play/p/TLkZkYzOcdO
I think it makes sense for it not to work. It would be quite confusing and annoying to have to go searching through every type listed and figuring out the intersection of their available methods to see what you could do with it.
If we can define a truly disjoint, finite, non-nil-able (or at least nil is an explicit element) type set, we can't include interfaces, but do we need interfaces to reason about the behaviors that are defined on that type set? I'm thinking (maybe naively?) that a compiler can tractably compute various sets-of-method-sets from the type set here. In practice I think a compiler could emit some precise and useful information ("error: Vehicle union doesn't implement Brake(): jetpack doesn't implement Brake()").
Going off-track a bit, I think there's also cases where defining a method on an element in a union itself could be interesting - I could call SetAlpha() on the union of rgb and rgba and still maintain a valid union, but not on an rbg value in isolation. In this case the union could satisfy SetAlpha(), but not if we required SetAlpha() to be defined on rgb.
I believe the need to explicitly list methods in interfaces containing union elements is an implementation restriction by the current Go compiler and should be lifted sooner or later. I don't see a good reason why it can't (though sometimes these things are surprisingly subtle - there are other implementation restrictions which I don't think can be lifted, or am at least skeptical about). Though I don't understand what "nil-able" has to do with it, you can call methods on nil values just fine.
Also, I agree with the criticism that it's a downside not to be able to define methods on union types, if they are defined like this proposal. Though a lot of the boiler plate can probably be reduced by struct embedding. As for the SetAlpha example, an alternative would be to have the method be RGBA() rgba, which could be implemented on both types and the usage would then be x = x.RGBA(), instead of x.SetAlpha(), which doesn't seem that bad.
Though I don't understand what "nil-able" has to do with it, you can call methods on
nilvalues just fine.
With a closed, finite type set, do we have to recycle nil as a catch-all? I think we could disallow declaring an instance of a sum type without declaring a variant, and ask implementors to explicitly provide empty/zero/undef variants - at least, I really enjoy this about sum types when I've used them in other languages. If a sum type exhibits a field that is unsafely nil, maybe that could be regarded as programming error that justifies a resulting panic just like it would otherwise.
I'm not sure it'd be insurmountable to do things more like the proposal suggests, and recapitulate the nuances of nils and interfaces, but it makes me nervous ... it looks manageable in small type switches but I think it could get nastier in practice - hard to reason about disjointness.
Currently the language seems to rely on every type having some meaning for the value that is represented as all zero bytes in memory. For example:
type Example interface {
int | string
}
m := make(map[string]Example)
v, ok := m["foo"]
Under the current proposal I would expect v to be nil because that is the zero value of Example. If a nil Example were forbidden then it isn't clear what v ought to be here.
Personally, I feel okay with accepting nil interface values as an established part of the "texture" of Go and having these "sealed" interface types inherit that assumption, rather than introducing the one situation where there isn't a zero value and dealing with the effects on all other parts of the language that gave been defined on the assumption of zero values, although I do agree that it'll mean that patterns from other languages with different type systems won't translate over exactly.
Each time I revisit this I find myself thinking that this feature perhaps deserves a more specific name than "sum types" to help make it clearer that this is just an application of the theoretical idea of sum types to some specific situations, and not something that is intended to cover all possible use-cases for sum types. I still quite like "sealed interfaces" because it seems more clearly a special kind of interface and so inherits most of what we're already accustomed to with interfaces (including nils and method sets) and focuses only on constraining the full set of implementers at the declaration site.
Although I don't think it's a deal breaker, I think it is notable that declaring all of the implementers inside the interface block means that the package which defines the interface must import any packages which export types that will be included in the set.
This means that the package that exports a type set member would be unable to import the package containing the interface and so could not name the interface type to use it in its own code without creating an import cycle.
Most of the use-cases we discussed above aren't impacted by that problem so I think this proposal is still useful despite it, but I do think it's interesting to think about given that it seems to invert the usual way that interface implementation works, where it's the package that defines the implementer that is responsible for (implicitly) adding it to the type set of the interface.
Although I don't think it's a deal breaker, I think it is notable that declaring all of the implementers inside the interface block means that the package which defines the interface must import any packages which export types that will be included in the set.
That's actually one of the points of the proposal. The idea is to create something analogous to C's unions or Rust's enums. For example, consider the case of scanning a stream of tokens. It makes sense to have predefined types for the various token types, such as
type Token interface {
Number | String | Operator
}
type Number struct { /* ... */ }
type String struct { /* ... */ }
type Operator struct { /* ... */ }
// Not a great API, but it demonstrates the idea.
func Parse(r io.Reader) ([]Token, error) {
// ...
}
There are a surprising number of situations where a value can be limited to one of a handful of possibilities that are all known in advance. This proposal is designed to improve the ergonomics around those situations. The current way that something like the above is usually handled is to define the token type as type Token any, but this is error prone because the types are less discoverable and it loses potential features that are only possible if the compiler is told what all the possibilities are, such as the potential ability to remove boxing mentioned in the proposal itself, linter enforcement of exhaustive type switches, and so on.
I believe the need to explicitly list methods in interfaces containing union elements is an implementation restriction by the current Go compiler and should be lifted sooner or later. I don't see a good reason why it can't (though sometimes these things are surprisingly subtle - there are other implementation restrictions which I don't think can be lifted, or am at least skeptical about).
Regardless of whether it can be lifted, I think it should not be.
If you have A | B | C and those types happen to all have an M method, you can never add a type D without a method M to the union—even if M is irrelevant to the purpose and use of the union—because that would remove M from the union's method set.
To be more concrete, I imagine this would happen quite (most?) often when M = String() string.
I think whether
switchis exhaustive is an entirely independent proposal to sum types. It can be proposed independent of any type changes, and it doesn't need to be attached to this proposal.
It can't be added later, because it would break by then existing programs. A decision would have to be made together with this proposal.
@jimmyfrasche You already can not add types to an exported union, without breaking compatibility, regardless of what methods the types involved have. I don't think allowing to call methods would change anything.
FWIW "adding members to a union" is similar in effect on their type set to "removing a method from an interface" and "removing a member from a union" is similar to "adding a method to an interface". So, unless you are the only user of a union, you really can't do anything about it.
In fact, that's kind of why people want unions. They want a closed set of types. If you could change that set, it would no longer be closed.
In fact, that's kind of why people want unions. They want a closed set of types. If you could change that set, it would no longer be closed.
It's also why I don't like it. Interfaces are a tool to grant freedom, to empower users to provide their own types by implementing them, even if the original author of a function didn't think of them. They mean to open the world for extension, not to close it off.
Yupp. FWIW in #19412 I brought up the inability to ever modify a union as an argument against their inclusion into a language that is - at least in part - deliberately designed to allow for gradual evolution of APIs a bunch of times. I'm not sure I still totally buy it, as most type system features kind of work that way in one way or another.
But it is something that might be a bit easier with a first-class union/sum type, as you wouldn't run into this aspect of constraints already having variance. For example, if you required a type-switch over a union to always have a default case and made it impossible to assign them to other unions (even if they are subsets of each other), I think you could then add new cases to them backwards compatibly. So, at least in part, it's a point against this specific implementation (overloading union elements for constraints).
@Merovius the situation is kind of different in that even if you are doing a v(N+1) you can't add it unless you can add the method (not an option if you want to add a primitive type) or be sure no one relied on the existence of the accidental method (and it's not even obvious that you need to look for this since it sneaks in implicitly). Unduly brittle for little gain when it'd make much more sense to be explicit.
I don't think I understand. Why wouldn't you be able to do a v(N+1)? And why is "being sure no one relied on the existence of the accidental method" any harder than "being sure no one ever used your interface as a constraint and then called a differently constrained function with it"?
(In any case, this is probably off-topic; this proposal is not about allowing to call methods not explicitly mentioned in an interface with unions)
Returning to the go/ast.Node example for a second, an advantage of a union type there would have been to consolidate the definition to one location. type Node interface { *AssignStmt | *BadDecl | ... }. Currently what implements an ast.Node is spread out a bit over a pretty big file (and is kept readable via discipline in how things are written). Only needing to consult one place in the code would have helped me read/use this and similar libraries in the past. This can be addressed by tooling so it is an overwhelming advantage. But overall I think this proposal would help with the readability of some packages.
I suspect we will not be putting 56 cases (# of ast.Node impls in ast) separated by '|' on the same line. So I would anticipate there will be a lot of trailing '|' for larger cases.
type Node interface {
*AssignStmt |
// more *Stmts
*ArrayType |
// more *Types
...
}
Still readable enough IMO, but worth taking into account.
(Not suggesting Node change from its current meaning of range of token.Pos. The token range definition has other existing uses and is a good example of where not to use a closed type set.)
Why is this spread out over a big file and not listed in a comment?
@timothy-king I think it would rather be written as
type Stmt interface {
*AssignStmt | *BadStmt | *BlockStmt | … | *TypeSwtichStmt
}
type Expr interface {
*BadExpr | *BinaryExpr | … | *UnaryExpr
}
type Decl interface {
*BadDecl | *FuncDecl | … | *GenDecl
}
type Node interface {
Decl | Expr | … | Stmt
}
There's still relatively long unions there, but it gets more manageable (and it might even be possible to break them up further).
@Merovius I suspect Stmt and Decl would not be exported in this case. But your point that these could be broken up into a union of union types is well taken.
If a nil example were forbidden then it isn't clear what v ought to be here.
Definitely this would require something heavy-handed, I have strong opinions here based only on speculation, but for the sake of speculation - there are places in Go (like having to make maps and chans) where the builtins get special cases, and I’d be interested in going to these lengths to eliminate a ubiquitous ‘nil’. There might be better ideas, disabling the walrus operator for sum types would be brutal and special but seems like one option.
Even if it’s in the machinery that somehow, somewhere a truly invalid instance might panic, I really think the only reasonable response to a ‘nil’ variant of a sum type is panic. Otherwise it’s very tempting to treat such an instance as a zero value of some other included variant, or a predicate to produce a valid value. Then, as a reader of that code, or a writer of code employing an unfamiliar sum type, I simply do not have the ability to immediately observe that ‘nil’ is a properly disjoint case.
We've gone down the path of some types not having a zero value several times in the past, and it's never worked. Let's not go down that path again. Let's just assume that in Go types must have a zero value. Thanks.
And since the proposal here is for a particular kind of interface type, and since the zero value for all interface types is nil, that is what this proposal says also. We can certainly discuss a sum type that has a different zero value (there is a lot of discussion over at #19412). But it would be very strange to say that for some interface types the zero value is nil and for some other interface types the zero value is something else. That is a level of complexity that I don't think we are going to add to the language.
@timothy-king Note that both Stmt and Decl are already existing and exported interface types in the ast package.
@apparentlymart @AndrewHarrisSPU
One idea would be to simply disallow non-nilable unions as channel or map Types (and elsewhere)
type Example interface{
int | string
}
type NilableExample interface{
int | string | nil
}
m:=make(map[string]Example) //compilation error: nil(type?) is not in the type set of Example
m:=make(map[string]NilableExample) // OK
e, ok:= m["something"]
//...
v, ok:=e.(Example) // regular type assertion to check that it's a legit Example and not nil.
Of course, the zero value of Example would still be nil. That doesn't change.
But such a value would only be created by variable declaration and not assignable where a non-nilable Example is expected.
So would come down to having to be explicit about nil.
One idea would be to simply disallow non-nilable unions as channel or map Types (and elsewhere)
Or slice types. Or fields. Or interface values.
The language doesn't like types without zero values. That can't really be helped.
Not necessarily a big issue for slice types either, or fields. One could still define the field case with the explicitly nilable supertype.
~~Some operations have to be disabled or modified otherwise, for slices of non-zeroable union values, that's true:~~
- ~~make would require a 0 length~~
- ~~slices of union types which don't allow nil assignment would deal with clear differently or disallow it~~
(edit:
I don't even think that it's important to be able to do that.
If a variable of type T is not zeroable (doesn't mean T doesn't have a zero value btw, just that it cannot be assigned the zero value for that type although var v T is the zero) , one can simply define a slice of {T | nil} which is explicit.
Because the current semantics of slices demand that each indexed slot can be empty or emptied.
Essentially, there are a few things that require optionality/ability to assign zero (to denote the lack of value) but that is easily built)
It's merely switching from nilable by default to nilable by construction which should be safer.
I don't think it would be much of a problem a priori. That's workable afaict. It's more an issue of proper value initialization, i.e. assignability.
For interfaces what do you have in mind as an issue?
Edit: I was randomly browsing and came across a similar treatment in Dart https://dart.dev/null-safety/understanding-null-safety
So it is possible. I still believe this would be more workable for Go unions since it reuses traditional mechanisms such as type assertions.
If subtyping was made more prevalent one day, that could be even further improved but it's not a necessity.
FWIW the idea of non-nilable interfaces has exactly the sample problems as the periodically happening discussion of non-nilable pointers. It's the same problem. It's not going to happen.
I'm not sure of what you mean.
I'm striclty talking about unions.
Basic interfaces would remain the same.
And I don't know what a non nilable pointer is.
Edit: I was randomly browsing and came across a similar treatment in Dart https://dart.dev/null-safety/understanding-null-safety. So it is possible.
Not to point out the obvious, but Dart is not Go. This is about how other aspects of the design of Go assume that every type has a zero value. Obviously, a language that is not designed under this assumption doesn't have this problem and there are many languages without nil.
Well I'm well aware obviously... , if you browse through it, there are a few sections that might be of interest such as the fact that they had to deal with initialization of variables. (to implement null safety after the fact!)
So appears that some issues were still shared and they've found a way.
Just saying that it's possible, not that it will be done but before we shoot the idea down for unions, might be interesting to explore it.
I used a map element as an example earlier but note that even a type assertion -- an operation specifically for interfaces, and so would be weird to ban here -- relies on zero value for the non-matching case:
v, ok := interfaceVal.(Type)
Although the type in a type assertion can be a non-interface type, an interface type is also valid in that position and is a common pattern for detecting if the dynamic type in the interface value also implements a more specific interface.
In that situation if the test fails then v is the zero value of the given interface type, which is always nil in today's Go.
This is just one more of many places where Go assumes there is a zero value of every type. I don't think it's feasible to simply ban a particular type from any situation where a zero value is required, because that assumption is all over the language.
To be accurate, the zero value should still exist and would still be nil.
The issue is rather definite assignment analysis.
It is sensitive to branching.
As long as this kind of analysis can be made fast, in a modular fashion, and without false negatives/positives, it should probably be fine.
I think Go might be one language for which it might be possible. (there are others, historically). I don't know if the ssa backend might not be of help here.
That's something to study.
@atdiar As soon as it is possible to create a zero value, it is literally impossible to guarantee that it's not getting assigned. That is, if I can create a nil T, it is impossible for a compiler to prove that any given T is not nil. Moreover, you've been told multiple times, by multiple people now, that this is infeasible and not going to happen. At this point, saying that there is "something to study" is pretty frustrating. There just is not. Take a "no".
As long as this kind of analysis can be made fast, in a modular fashion, and without false negatives/positives, it should probably be fine.
Take this function
func AddNElems[T any, S ~[]T](slc S, n int) S {
return append(slc, make([]T, n)...)
}
Can we prove that this function take only slices of non-nilable elements?
@DmitriyMV
The slice type constructor implicitly requires that T is nilable/zeroable (because a slice can be cleared) . I don't see why such a check would be infeasible.
@Merovius
What makes you think that I have to be compelled to agree with you?
You don't even understand what I am saying.
I'm not even claiming that a variable shouldn't be assigned. I'm saying that a variable shouldn't be assigned the zero value in certain cases.
Before claiming that something is impossible, one might want to study. This is not the first time that you make such claims and exhibit an attitude that is slightly disrespectful. I hope it will be the last time.
The slice type constructor implicitly requires that T is nilable/zeroable (because a slice can be cleared) . I don't see why such a check would be infeasible.
So, you are saying AddNElems where T is any will not work with "non-nilable interfaces" - is that correct?
Where T is constrained by any? Yes it won't work. (in your example, because T also is declared as a slice type)
If you have a non-nilable union type U, U wouldn't be usable in a slice. However interface{U | nil} should be usable as a type argument.
Just a reminder to keep the discussion here respectful. You know who you are. Thanks.
So, to be clear, you essentially want a separate class of types (meta types?) similar to existing, but with additional and very specific restrictions to available operations on this "class" of types.
I mean, the whole point of any is that it can satisfy any type (and by extent, being type constraint, works for any type). With what you are proposing, Go type system will be divided not only by two "type hierarchies" but also two "type constraint hierarchies". Essentially, this means we will have two entirely separate type systems which are not interchangeable and have very different low level semantics. Which, in turn. essentially means you will have a two different languages in one - first one assumes that zero state is valid (starting from slices and ending with "reflect") and the second one demands initialization and disallows the set of operations like make([]T, n) and such.
The complexity of implementing this, in the end, equals to creating a new language, so the question becomes - why bother with adding this to Go? What I'm trying to say, is that no matter how we don't like "zero value" in specific situations, it is one of the fundamentals the language is built on. We can adjust sum types to work with it, but we cannot break it or make a parallel mechanism for sum types - this will either be a fundamentally breaking change (the road Dart 2.0 took BTW) or unsound type system. I don't think we want either.
This is off-topic now, please create another proposal if you have ideas on adding non-zero variables to the language. Thanks.
Understood. I'll create another prospective issue. To be clear one more time, this is still not "adding non-zero variables" , it is about definite (un)assignment.
Zero values are fine, but the runtime.Type of the zero value is not necessarily in the type set which is what I'm trying to address. Doesn't seem that off-topic to me but fair enough.
Cheers.
I think one of the first things some Go developers would try to do with this feature is implement a generic Option type. Here's a brief exploration of that idea.
My first thought was that it would look like this:
type None struct{}
type Option[T any] interface{None | T}
But the first problem here is that a type parameter can't be used in a type list: the definition above results in a MisplacedTypeParam error.
The second problem is that, since an interface value can be nil, a variable of type Option[T] could either nil, or None, or a value of type T. So it would be a sort of optional optional.
In fact to define an Optional int64 type this would be sufficient:
type OptionalInt64 interface{int64}
This would actually have many of the properties I would be looking for in an option type:
nil ("None")int64 value would be stored directly (avoiding memory allocations)I have sometimes used sql.Null* types such as sql.NullInt64 as generic optional types (even, sadly, in code that doesn't otherwise deal with sql) because they have the above advantages. In fact if this proposal was implemented with values stored directly, then OptionalInt64 would be stored as the equivalent of sql.NullInt64.
But OptionalInt64 would add some type safety: you couldn't treat an OptionalInt64 as an int64 without first guaranteeing it was not nil (right?).
A standard generic Option type was discussed in #48702. From what I can see the proposal was rejected because of unanswered questions, and not necessarily because a standard optional type was not desirable.
Difficult question, but I think it may contribute to the “costs” of this proposal: How serious do you estimate the danger that people start using sum types instead of error types as return types? Would this be even feasible?
@bronger I don't think this proposal is particularly useful for that - and insofar as it is, you could get basically the same effect without it.
One thing people want from a Result[T] type is that it can either be an error or a T. But with this proposal, there's always a third option: It could be nil. So, right off the bat this proposal gives weaker guarantees than what people really want.
Then you'd have to jump through a couple of hoops to make such a Result type actually safe. There are some obstacles:
T could implement errorT nor error can be union terms (the former because it's a type parameter, the latter because it has methods)Result[T] using type-assertions, so the cases must be disjoint, to be type safeSo you'd get something like
type Error struct { E error }
type Success[T any] struct { Val T }
type Result[T any] interface { Error | Success[T] }
There's a lot of overhead in using this, over returning (T, error) - not just because you need to type-assert, but you also need to wrap and unwrap the individual structs.
And the value you get is saying that a Result[T] is either an Error or Succes[T], fair enough. But what does that get you, except documentation? The compiler won't actually type-check that you correctly type-switch on it exhaustively. So, how is this really any better than just returning an any and documenting that it's either an Error or a Success (or construct a different non-union interface for wrapping)?
I think the value of this proposal can only really come in enforcing constraints on inputs, really. There is some value in knowing that you aren't being given anything but one of these N things. The information that a function only returns a finite set of types isn't super useful without further infrastructure (like match statements and exhaustiveness checks and the like).
One advantage, to me, is if it makes it easier to plug things that return Result[T] into things that accept Result[T] (or, possibly, ...Result[T]) than it is with (T, error). Same for Option[T] vs (T, bool).
That points out another problem, namely that defining methods is sometimes impossible because the type parameters don't propogate fully:
type Result[T any] interface {
Success[T] | Error
Or(T) Result[T]
Must() T
}
// So far so good...
func (r Success[T]) Or(v T) Result[T] { return r }
func (r Success[T]) Must() T { return r.Val }
// Uh oh...
func (r Error) Or(v ???) Result[???] { return ??? }
func (r Error) Must() ??? { panic(r.E) }
You could fix this by attaching an otherwise unused [T any] to Error so that you can propogate the parameter through, but that's going to make instantiation annoying:
func SomethingThatCanFail() Result[Value] {
if failure {
// Need to manually specify even with a constructor.
return NewError[Value](err)
}
}
The reason to not permit
~Tis that the current language would provide no mechanism for extracting the type of such a value. Giveninterface { ~int }, if I store a value of typemyIntin that interface, then code in some other package would be unable to use a type assertion or type switch to get the value out of the interface type.
If I understand correctly, it's the proposed implementation and not the the proposal itself that would prevent this. In the current language, non-nil interface values always include a reference to the dynamic type, so type assertions or switches can be used to extract the type as long as that type is in the type set. However if as proposed a small code was used instead of a reference to a type, then the type would always have to be one of the finite set of declared types. Is this correct?
@johnwarden No. The hindrance @ianlancetaylor is talking about is that you need to be able to lexically refer to a a type to type-assert on it. That is, to write x.(time.Duration), you first have to be able to write time.Duration and thus import time. That is, you have to statically know a type to type-assert on it. This has nothing to do with the implementation.
I think, on the contrary, the implementation of using "tags" is only possible as long as ~T is forbidden, FWIW. Because otherwise you'd have to carry the type as a pointer, so you can pass such an interface value to reflect and/or do a full type-assertion on it. That is, int64 and time.Duration must always have different representations when stored in an interface { ~int64 }.
@ncruces Why do you need this proposal for that? i.e. why do people not do it today?
I mean, I guess I see the argument - that if it's good to take something as a parameter, returning it is also a bit good, because it makes this composition easier. But I think for the specific case of Result[T], I don't see a lot of use cases for accepting a Result[T] where that kind of safety is very useful, or offsets the incredible awkwardness of using it. So I'm not worried about people moving away from (T, error), is all I'm saying. It just seems like the worse mechanism with this proposal.
@johnwarden No. The hindrance @ianlancetaylor is talking about is that you need to be able to lexically refer to a a type to type-assert on it. That is, to write
x.(time.Duration), you first have to be able to writetime.Durationand thus importtime. That is, you have to statically know a type to type-assert on it. This has nothing to do with the implementation.
Yes, I understand you can't actually extract the type of a value in Go except by using reflection. But I am trying to understand this comment from @ianlancetaylor where he refers to extracting the type of an interface value: "The reason to not permit ~T is that the current language would provide no mechanism for extracting the type of such a value". I assume he meant "find out" the type by using a switch or assertion.
I think, on the contrary, the implementation of using "tags" is only possible as long as
~Tis forbidden, FWIW. Because otherwise you'd have to carry the type as a pointer, so you can pass such an interface value toreflectand/or do a full type-assertion on it. That is,int64andtime.Durationmust always have different representations when stored in aninterface { ~int64 }.
By "tags", do you mean using the short (e.g. one byte) code to indicate the type?
If so then it sounds like it is the proposed implementation, not the the proposal itself that would prevent use of ~T in type lists, correct?
@johnwarden I don't understand. ISTM you are restating your questions, quoting my answers to them.
But I am trying to understand this comment from @ianlancetaylor where he refers to extracting the type of an interface value:
Yes. I'm pretty confident he refers to what I said: If you have func F(v ~int), then F has no way to "unpack" v by writing a type-assertion or type-switch, as the set of types v can have is infinite and unknown to F. So it makes little sense to have it as a value. It couldn't be used.
Or, to be more precise, one thing we could allow is make x.(~int64) on a time.Duration have type int64, instead of time.Duration. That wouldn't require the function to know the actual type. But that has other problems.
By "tags", do you mean using the short (e.g. one byte) code to indicate the type?
Yes.
If so then it sounds like it is the proposed implementation, not the the proposal itself that would prevent use of ~T in type lists, correct?
No. The issue is the semantic one I explained above, which has nothing to do with the implementation.
The proposed implementation is to use a plain interface, as it is represented now. And that wouldn't stand in the way at all. The proposal also says that in some cases we might choose a different implementation as an optimization and yes, that would stand in the way. But it doesn't matter that it would, because the actual semantic problem is far more important. Anything else would only be an optimization, it wouldn't prevent us from doing it.
@johnwarden I don't understand. ISTM you are restating your questions, quoting my answers to them.
Hmm, my apologies. I don't have a lot of experience contributing to this project, and I think there may be some fundamental concept I am missing. It may not be worth you trying to explain it to me and I would not be offended if you don't respond. But I will take one more shot.
But I am trying to understand this comment from @ianlancetaylor where he refers to extracting the type of an interface value:
Yes. I'm pretty confident he refers to what I said: If you have
func F(v ~int), thenFhas no way to "unpack"vby writing a type-assertion or type-switch, as the set of typesvcan have is infinite and unknown toF. So it makes little sense to have it as a value. It couldn't be used.
I am confused by func F(v ~int) here because that is not correct Go code. Are you suggesting Func F(v interface{~int}) would not make sense, because the set of types v can have is unknown to F?
But interface always have infinite type sets. So a far as I see interface{~int} would be no different. If you wanted to get at the dynamic type, you would need to do a type switch or assertion on a known type.
So for example you could do this:
type IntLike interface {~int64}
func F(x IntLike) {
switch x.(type) {
case int64:
fmt.Println("int64")
case time.Duration:
fmt.Println("time.Duration")
}
}
F(time.Seconds)
// Output: time.Duration
Or, to be more precise, one thing we could allow is make
x.(~int64)on atime.Durationhave typeint64, instead oftime.Duration. That wouldn't require the function to know the actual type. But that has other problems.
Yeah allowing ~T in type switches seems problematic because ~T defines an infinite set of types. But I don't see why it would be necessary to allow ~T in type switches.
Are you suggesting
Func F(v interface{~int})would not make sense, because the set of typesvcan have is unknown toF?
Correct.
But interface always have infinite type sets. So a far as I see
interface{~int}would be no different.
The difference is that you can do something useful with an interface without knowing its dynamic type: call its methods. You can'd do anything useful with an interface{ ~int } (except pass it on). interface{ ~int } is semantically equivalent to any (well, not completely, but for the most part. You can also do binary operations with integer literals. But even that goes away once you add more terms to the union).
The current, idiomatic, way of returning an optional T is (T, bool). The way of accepting it is, probably, if any, *T. If you want to plug the two, there's work to do. If the idiom were Option[T] plugging would be easier.
With errors, the idiom is to return (T, error), with no real idiom to accept. Result[T] would change this.
As for use cases, no pressing need. But I imagine stream processing, reactive programming, some functional idioms would probably benefit. Overall, I don't expect the Go idioms to change much if at all. But in those niches/corners they might?
@ncruces Maybe. If I thought they might, I'd count that as a reason against this proposal. It would make for bad code and I don't like making it too tempting to write bad code.
[edit] Actually:
With errors, the idiom is to return
(T, error), with no real idiom to accept.Result[T]would change this.
ISTM if you need an idiom, accepting (T, error) would work far better (as it's already established in existing corpora). And I think that's what current attempts in the fields you mention are doing.[/edit]
The difference is that you can do something useful with an interface without knowing its dynamic type: call its methods. You can'd do anything useful with an
interface{ ~int }(except pass it on).interface{ ~int }is semantically equivalent toany(well, not completely, but for the most part. You can also do binary operations with integer literals. But even that goes away once you add more terms to the union).
Yes, I see what you are saying. interface{ ~int } might not be very useful. One thing you can do with it, however, is ~coerce~ convert it to an int (because one thing the compiler knows about everything in its typeset is that it is ~coercable~ convertible to int). So this should be possible.
type IntLike interface { ~int }
var x IntLike = int(2)
var y = int(x)
I had forgotten that ~T can be shorthand for interface{~T}. So in that case, since interfaces are allowed in type switches, it seems reasonable that ~T should be too. ~T would match anything that implemented the interface (e.g. anything with an underlying type of T), and a value of that type could be ~coerced~ converted into T.
I think it would follow that the meaning of ~T in type switches would be pretty well defined. For example:
type Num = interface { ~int64 | ~float64 }
func F(x Num) {
switch y := x.(type) {
case ~int:
// here y has type interface{ ~int }
fmt.Println("x is in an ~int64:", int64(y))
case ~float64:
// here y has type interface{ ~float64 }
fmt.Println("x is in a ~float64:", float64(y))
}
}
F(2 * time.Seconds)
// Output: x is an ~int64: 2000000000
One thing you can do with it, however, is coerce it to an int (because one thing the compiler knows about everything in its typeset is that it is coercable to int).
But that falls apart as soon as you have interface{ ~int | ~string }. So it would only really help if you'd have exactly one type which seems an oddly restrictive special case when talking about unions.
~Twould match anything that implemented the interface (e.g. anything with an underlying type ofT), and a value of that type could be coerced intoT.
That's what I mentioned here, where I also linked to this comment explaining that this has problems as well.
@johnwarden I tend to agree if we substitute coercion with conversion which might have been what you meant.
Overall, I don't expect the Go idioms to change much if at all. But in those niches/corners they might?
I don't think Option[T], Result[T] are likely to do much, the arity of the cases really are covered by multiple returns.
I might speculate that there's more utility when considering sum types with larger arity of variants, when business logic has more than a few variants that are nearly synonyms of zero (e.g. how missing, none, n/a, undefined can be meaningfully different than zero or nil), or more than a few variants that are synonyms of "a specific error occurred and should be dealt with specifically".
I tend to agree if we substitute coercion with conversion which might have been what you meant.
@atdiar oh yes "conversion" is what I should have said. Thank you.
@Merovius Sorry for the confusion on my part, I think I do now understand what you are trying to explain, and that it has nothing to do with implementation.
But that falls apart as soon as you have
interface{ ~int | ~string }. So it would only really help if you'd have exactly one type which seems an oddly restrictive special case when talking about unions.
Yes, and I see that the issues with this restrictive case in type switches were discussed by Ian in this comment, as you pointed out.
But I think the answers to some of his questions follow naturally from the rules for interfaces in type switches: no special rules would need to be defined. In fact ~T syntax would not even need to be supported in type switches:
func hash64Bit(x interface { ~int64 | ~float64 | ~complex64 } ) int64 {
v := switch x.(type) {
case interface { ~int64 }:
return hashInt64(int64(v))
case interface { ~float64 }:
return hashFloat64(float64(v))
case interface { ~complex64 }:
return hashComplex64(complex64(v))
}
}
This may have limited usefulness -- this is definitely a contrived example. But is this limited usefulness reason to maintain different rules for interfaces used as type constraints and those used as regular interface types? The exception is often as costly as the rule; having consistency here would have benefits such as make the language easier to learn.
But as Ian also mentioned in this comment:
It's quite possible that these questions can be answered, but it's not just outside the scope of this proposal, it's actually complicated.
So I see that perhaps this discussion is off-topic, since this proposal explicitly excludes the possibility of interface { ~T }.
Overall, I don't expect the Go idioms to change much if at all. But in those niches/corners they might?
I don't think
Option[T],Result[T]are likely to do much, the arity of the cases really are covered by multiple returns.
Optional values are often included in data structures as well, especially when unmarshaling data from external data sources such as json, protobufs, and databases. While *T is common here, another common idiom is something like struct{Valid int, Value T} (for example sql.NullInt64), which avoids the memory allocation ~and safety issues of pointers~.
If this proposal were implemented, I suspect an Option[T] defined as follows would often be preferred to either of these:
type Option[T] interface{ T }
This would avoid memory allocations if the T value is stored directly ~, and would be safer than a pointer. It would also be safer than a struct{Valid int, Value T} because the value could not be accessed without first checking that it was valid~. The possible advantage over struct{Valid int, Value T} would be that if the value was accessed without first checking that it was valid, it would panic, instead of silently returning a zero value.
For example:
func f(optionalV Option[float64]) float64 {
// return f*2
// not allowed: * operator not defined on Option[float64]
if v, ok := optionalV.(float64); ok {
return f*2
}
return 0
}
EDIT: as @Merovius points out there are actually no safety benefits here, since x := optionalV.(float64) can panic.
This would avoid memory allocations if the
Tvalue is stored directly, and would be safer than a pointer.
I don't believe the value T can commonly be stored directly, for the same reason we can't do it in a regular interface. I also don't believe there is anything "safer" about an Option[T] than a *T, unless you add significant extra machinery to the language. In this particular case (your type Option[T] interface{ T }, which to be clear isn't actually valid - you can't use a type parameter as an interface element directly) they are exactly equivalent in terms of safety. You have to spell the check the same - x == nil - and in both cases the code will panic if you forget it.
Under this proposal, Option[T] offers no safety guarantees whatsoever over a *T.
because the value could not be accessed without first checking that it was valid. For example:
Of course it could. You can write x := optionalV.(float64). There is no safety difference between that and spelling it *x. And there is no meaningful difference between remembering to spell it if x, ok := optionalV.(float64); ok and spelling it if x != nil.
Under this proposal,
Option[T]offers no safety guarantees whatsoever over a*T.
It does a little:
type Value struct {
S string
}
type ViaPointer struct {
Val *Value
}
type ViaOption struct {
Val Option[T]
}
func P(v ViaPointer) string {
// Will panic if Val is nil.
return fmt.Sprintf("Value: %q", v.Val.S)
}
func O(v ViaOption) string {
// Compile-time error without explicit check.
return fmt.Sprintf("Value: %q", v.Val.S)
}
Option[T] also makes the case of an optional pointer nicer. Double pointers feel weird to me.
This would avoid memory allocations if the
Tvalue is stored directly, and would be safer than a pointer.I don't believe the value
Tcan commonly be stored directly, for the same reason we can't do it in a regular interface.
In the proposal, Ian mentions that in some cases values could be stored directly, rather than boxed. This is impossible now because the exact type and size of an interface value's dynamic value can't be known at compile time. There may be other reasons, but I would guess interface { int64 } would be one case where the value could be stored directly.
Under this proposal,
Option[T]offers no safety guarantees whatsoever over a*T.because the value could not be accessed without first checking that it was valid. For example:
Of course it could. You can write
x := optionalV.(float64). There is no safety difference between that and spelling it*x. And there is no meaningful difference between remembering to spell itif x, ok := optionalV.(float64); okand spelling itif x != nil.
Oh yes you are absolutely right, I hadn't thought that through.
The proposal above illustrates how to add sum type (aka: union), but tagged union is more powerful and useful (which allows multiple occurrences of a type, use tags to distinguish different variants).
Real world designs:
https://doc.rust-lang.org/book/ch06-01-defining-an-enum.html
https://docs.swift.org/swift-book/LanguageGuide/Enumerations.html
@leaxoy sigma types issue is probably what you are looking for.
Unlike most proposals, there are downsides to not accepting this one:
1. union elements in interfaces can only ever be used in constraints
2. if, hypothetically, there were a separate mechanism for sum or union types, it would be confusing that there is also this very similar mechanism that's unrelated
I think the second situation is unlikely. The bar would be much higher than any other language change, which is already pretty high.
I dislike the first situation. There are many uses for union types even if they have downsides compared to other more theoretically pure alternatives.
How about introduce new keyword enum or union and make it cannot be nil, just like struct. Nil in sum type is a big challenge.
@leaxoy Introducing a new keyword is not backward-compatible, because any code today using enum or union as variable names will cease to compile. We would also need to decide on what "it cannot be nil" actually means, because every type in Go must have a zero value. #19412 contains a great deal of discussion on this.
Perhaps this is a trade-off, although introducing new keywords breaks some compatibility, but handling nil is also tricky. Nil takes on too many features in go.
But after #56986 and #57001 and #60078, is there a mature way to introduce new features.
FWIW this issue is specifically about using the existing syntax, because it seems dubious to have two different syntactical constructs to mean very similar things. Being able to reuse that syntax was, in fact, one of the (minor) arguments for introducing it to begin with.
The proposal above illustrates how to add sum type (aka: union), but tagged union is more powerful and useful (which allows multiple occurrences of a type, use tags to distinguish different variants).
Real world designs: https://doc.rust-lang.org/book/ch06-01-defining-an-enum.html https://docs.swift.org/swift-book/LanguageGuide/Enumerations.html
This proposal can support tagged unions via specialized types:
type All struct{}
type None struct {}
type Some []string
type Filter interface {
All | None | Some
}
func Select(f Filter) ([]string, error) {
// ...
}
Empty struct values could be optimized away:
As an implementation note, we could in some cases use a different implementation for interfaces with an embedded union type. We could use a small code, typically a single byte, to indicate the type stored in the interface, with a zero indicating nil. We could store the values directly, rather than boxed.
Edit: "multiple values of a type:"
type Width uint32
type Height uint32
type Weight uint32
type Dimension interface {
Width | Height | Weight
}
func f(d Dimension) ...
_ = f(Width(10))
_ = f(Height(20))
In particular, the zero value of an interface type with an embedded union would be
nil, just as for any interface type. So this is a form of sum type in which there is always another possible option, namelynil. Sum types in most languages do not work this way, and this may be a reason to not add this functionality to Go.
@ianlancetaylor would you consider a form that would disallow nil?
Some hypothetical syntax:
type I1 interface! {
int | float64
}
I1?@ydnar The question of "what would the zero value be" is exactly the one that needs to be answered for that kind of proposal. Syntax aside, the concept of non-nillable interfaces or sum types has been suggested many times between here, #19412, and other proposals. None of them have answered the zero value question in a way that satisfies even a majority of people (including those that have tried the answer "that of the first type in the union").
Sum types in most languages do not work this way
That's true, but I think most languages also don't have a concept of zero values in the way that Go does. Rust, for example, requires all values of any type to always be explicitly set to something, even if it's a default value.
Random thought that might be terrible: What if Go did allow nil sum interfaces but with an unusual caveat: A variable of a sum interface type can't be set to nil manually. In other words, a declared but unassigned sum interface variable would be nil, but once changed it would be impossible to get it back to being nil, with a runtime check on assignment to make sure that an assignment of a dynamic value doesn't do that, either. Then it'll have a useful zero value, but one with minimal impact.
A variable of a sum interface type can't be set to
nilmanually.
You could still set values to nil with:
type Bits64 interface { int64 | uint64 | float64 }
type Dummy struct { Field64 Bits64 }
func (d *Dummy) SetField64(value Bits64) {
d.Field64 = value
}
func main() {
var d1, d2 Dummy
var value uint64
d1.SetField64(value) // d1.Field64 stops being nil
d1.SetField64(d2.Field64) // d1.Field64 is nil again...
}
To prevent this you would have allow only the concrete types listed in the sum type be assignable to sum type variables. Which is way, way much more restrictive. But yeah, it may be the only real alternative to fully nilable sum types if we want to make them be based on interfaces.
@DeedleFake I don't believe that would be practically feasible. x := make([]T, 42) is an "assignment of zero values", so would have to panic - and, less obviously, x = append(x, someNonNilThing) - so we couldn't have slices. _, ok := m[x] as well, so you couldn't check if an element is in a map. x, ok := <-ch would be an assignment of a zero value, if the channel is closed, so you couldn't use that as a select-case. Etc. We would have to exhaustively list exceptions to that rule of "assigning a zero value is a runtime check and causes a panic" to make this work. It's not practical.
And without that runtime check, there is no real benefit, because you still have to code against the possibility of it being nil.
@tinne26 Under @DeedleFake's suggestion, that code would panic, because passing d2.Field64 as an argument is an assignment (from the language POV) so it would panic, as it is nil.
including those that have tried the answer "that of the first type in the union"
I don't remember, what was the problem with it? It's a straightforward rule that can be easily explained. It also appears to be intuitive, as it's the default response for most proposers when posed with the question.
@gophun Currently, | in an interface is commutative - interface{ a | b } and interface{ b | a } mean the same thing. That would no longer be the case. There's also the question of what would happen with something like interface{ a | b ; b | a }. "first" is sometimes not super straight forward.
@Merovius
Yes, it would be awkward if interface is reused for union types. But if we made them separate (adding a keyword is not completely ruled out) with the "zero value based on first type" rule:
type foo union { a | b }
Then these
interface { a | b }
interface { a | b ; b | a }
could be short form for:
interface { union{ a | b } }
interface { union{ a | b } ; union{ b | a } }
Here the order wouldn't matter.
@gophun This issue is about using general interfaces for unions. #19412 is about other options - which each have their own set of problems, but that discussion doesn't belong here. And FWIW, adding a union keyword like you suggest has been discussed over there at length.
@Merovius Thank you for the pointer; I'll take it over there. The keyword option was criticized solely on the grounds of being "not backwards compatible," a stance that has been clearly contradicted by the Go project lead.
The keyword option was criticized solely on the grounds of being "not backwards compatible,"
That is not true. But again, that discussion doesn't belong here.
Would this proposal also allow for something like this?
type MyType[T constraints.Integer | constraints.Float | string | []byte] struct {
Value T
}
func (m *MyType[T]) UnmarshalText(data []byte) (err error) {
switch v := (&m.Value).(type) {
case *constraints.Signed:
err = parseInt(data, v)
case *constraints.Unsigned:
err = parseUint(data, v)
case *constraints.Float:
err = parseFloat(data, v)
case *string | *[]byte:
// ideally we could get the concrete type of v
// in the context of this case as some type name *U,
// so we could even do: *v = U(data)
parseString(data, v)
}
return err
}
func parseInt[T constraints.Signed](data []byte, v *T) error {
i, err := strconv.ParseInt(string(data), 10, 64)
*v = T(i)
return err
}
func parseUint[T constraints.Unsigned](data []byte, v *T) error {
u, err := strconv.ParseUint(string(data), 10, 64)
*v = T(u)
return err
}
func parseFloat[T constraints.Float](data []byte, v *T) error {
f, err := strconv.ParseFloat(string(data), 64)
*v = T(f)
return err
}
func parseString[T string | []byte](data []byte, v *T) {
*v = T(data)
}
Because the below is a pattern I find myself using frequently at the moment, and it would be amazing to be able to replace it with something more compact like the above.
func (m *MyType[T]) UnmarshalText(data []byte) (err error) {
switch v := any(&m.Value).(type) { // note having to cast to any first...
case *int:
err = parseInt(data, v)
case *int8:
err = parseInt(data, v)
case *int16:
err = parseInt(data, v)
case *int32:
err = parseInt(data, v)
case *int64:
err = parseInt(data, v)
case *uint:
err = parseUint(data, v)
case *uint8:
err = parseUint(data, v)
case *uint16:
err = parseUint(data, v)
case *uint32:
err = parseUint(data, v)
case *uint64:
err = parseUint(data, v)
case *uintptr:
err = parseUint(data, v)
case *float32:
err = parseFloat(data, v)
case *float64:
err = parseFloat(data, v)
case *string:
*v = string(data)
case *[]byte:
*v = data
}
return err
}
(Note that case *int, *int8, *int16, *int32, *int64 isn't useful here, since this would coerce the type of v to any, so you couldn't pass it to any of the specialised parser functions.)
It would not, for a couple reasons:
1. The type parameter T is set at compile time. &m.Value is a *T where T is one of the elements of constraints.Integer, constraints.Float, string, or []byte, not a sum type. This proposal is about allowing run-time values of interface types that contain union elements; it is mostly orthogonal to type parameters.
2. constraints.Signed &c. use ~T elements. This proposal does not allow values of interface types when those interfaces contain ~T elements.
I think what you want is #45380.
Seeing this example, it occurs to me that https://github.com/golang/go/issues/57644#issuecomment-1373901347 actually seems to be wrong. Consider these definitions:
type bytestring interface {
string | []byte
}
func f[T bytestring]() {}
Type bytestring itself can instantiate f if bytestring satisfies bytestring, which it does if bytestring implements bytestring. Since bytestring is an interface, it implements any interface of which its type set is a subset, which trivially includes itself. Therefore f[bytestring] is a legal instantiation.
So, it seems that we need additional adjustments to the spec to make interfaces with union elements legal. Otherwise every type constraint which includes a union element and no ~T terms gains a non-empty set of members, all of interface type, which will be illegal in almost every case.
Triple posting aside, discussion on #48522 prompted me to think about what "additional adjustments to the spec" we would actually need for the proposed union types to not break existing code.
My initial thought was "interfaces containing union elements cannot satisfy the interfaces they implement." That would prevent bytestring above from instantiating functions with any constraints, which seems obviously a non-starter.
The minimal condition, in the sense of allowing unions to satisfy the most constraints, would be that they satisfy any constraint of which their operation set is a superset. Being interfaces, the operations they bring are comparison, type assertion, and their methods. Type assertion is mostly irrelevant due to the rules about interfaces in union elements. So, it seems like we could get away with a rule like, "an interface T containing union elements with no ~U terms satisfies constraints that are basic interfaces that T implements, as well as constraints that can be written in the form interface { comparable; E } where E is a basic interface that T implements."
The question then becomes where that rule leaves us with covariance. With this rule, and returning to the definitions above, can we write func g[T bytestring]() { f[T]() }? Since type parameters are interfaces underneath, I think the answer is no. We need more precision to handle type parameters. We end up with a definition of "satisfies" that looks, in total, something like:
A type T satisfies a constraint C if:
- T is not an interface containing union elements with no ~U terms, or T is a type parameter type, and T implements C;
- T is not an interface containing union elements with no ~U terms, C can be written in the form interface { comparable; E } where E is a basic interface, and T is comparable and implements E; or
- T is an interface containing union elements with no ~U terms, but not a type parameter type, and C is a basic interface that T implements or C can be written in the form interface { comparable; E } where E is a basic interface that T implements.
I find this definition hard to follow compared to the current one, but I think it does everything we need for this proposal.
I have been reading this thanks to @Merovius linking it from the [go-nuts] list.
Seems to me the biggest argument against interfaces-as-sum-types is over zero values, i.e. that Go fundamentally requires zero values and that can't change, there is no consensus on how to arrive at a zero value for these sum types, and with others wanting sum types to not have zero values as they see zero values conflicting with the benefits they see sum-types providing. IOW, a classic catch-22.
If I understanding this wrong, please let me know.
I think it would be great if this could become a feature of Go so I considered that catch-22 in hopes to resolve it and came up with something I think could work.
The first aspect would be to require that these sum types not be able to be instantiated without providing an explicit value. That would be mean some of the following would throw a compiler error:
type Identifier interface {
int | string
}
var widgetId Identifier // throws compile error
widgetId := Identifier(1) // compiles fine
widgetIds := make([]Identifier,3) // throws compile error
widgetIds := []Identifier{ // compiles fine
Identifier(123),
Identifier("happy"),
Identifier(456),
}
Unless I miss some way in which a property can get a zero value, the above limitation would be sufficient to ensure that a sum type never had an opportunity to have a zero value (I ignored in my example returning an uninitialized value from a func but let's assume that is disallowed to.)
If simply disallowing sum types from being created if not initialized is not sufficient — because someone might use CGo or some other edge case to create an uninitialized sum type — then we would need a real zero value. That is where IMO reconsidering the untyped builtin zero (#61372) could fit in. A sum type could have a zero value of just zero and could not be otherwise be represented.
So if it comes to pass that a variable or expression of type of a sum type has a zero value then using that variable or expression for anything other than assignment of a non-zero value or checking if it is equal to zero would be a runtime error generating a panic. Since in normal cases it should never have a zero value then having a zero value occuring would truly be an exceptional case indicating an error somewhere else and code, and thus deserving of a panic.
The zero value could be represented internally exactly as an interface containing nil is represented, but for a sum type st then st==nil would always be false and st==zero would only ever be true for an exceptional case that should never happen for normal use cases.
The only real downside I see to this approach would be that you could not pre-create a slice or map with any elements using make().
However, if for our 3rd aspect we allowed extending make() to recognize what I will call an "initializer" then make could initialize sum types to a default value. Consider setting the value of a slice of ten Identifiers to be 0:
ids := make([]Identifier{0},10)
And this sets a slice of 25 Identifiers to be initialized to an empty string:
ids := make([]Identifier{""},25)
So there it is. Please feel free to poke holes in this approach when and if you find any.
P.S. We don't really even need zero to make this work, the zero value could still be nil but would be just as constrained as I described for zero. But having zero would be a nice distinction because then sum types would be the only type in Go with a zero value but no other representationt vs. having a nil that behaves differently for sum types than for other nillable types.
@mikeschinkel Thanks. The idea of not permitting the type to be instantiated without a value has been suggested several times before in the various discussions of sum types. It has always been rejected. Zero values are built into the language too deeply. For one example--and just for one example, there are other problems--how do you handle a type assertion to the sum type if the type assertion fails? What value do you give to the first result of the type assertion?
@ianlancetaylor — Thank you for acknowledging.
I see your perspective in how my suggestion also results in a zero values concern.
how do you handle a type assertion to the sum type if the type assertion fails? What value do you give to the first result of the type assertion?
I expect you meant that as a rhetorical question, but since you posed the question I hope you do not mind me at least answering it.
If a type assertion fails in an assignment to a variable of a sum type then the variable would get the value of zero (now it seems zero is required, after all), as the same rules I outlined above would apply. Doing anything with that variable besides assigning it a non-zero value or testing it for equality with zero should panic.
That seems reasonable to me, at least, because a failed type assertion is a failure so accessing the value of that variable is almost certainly a logic error anyway. Right?
That scenario does bring up a question of whether or not a zero-valued sum-type could be passed to a function, and I could go either way on that. Seems less than reasonable to disallow it, but then the panic would occur elsewhere compared to where the the panic was caused.
I do respect that you and others may view those constraints as not what Go should be, and I will be accepting of that if it is the final ruling.
However, AFAICT, I still think the logic of my suggestion is valid, unless there is some other scenario that emerges that cannot be resolved in the same way as for failed type assertions. 🤷♂️
@mikeschinkel IMO your suggestion is now back to the point where every sum type has a zero value of nil. It doesn't really matter if it is spelled nil or zero, as far as the contentious questions are concerned, as long as it the semantics are the same. Which it seems they mostly are.
That doesn't mean the suggestion isn't viable, it's just that it doesn't differ significantly from what we have been talking about so far.
To me, that means FWIW that there is no need to disallow make etc. either, because if there is any way to create an invalid zero value, usage of a variant needs to be prepared to deal with it. If it needs to be prepared to deal with it, might as well keep more coherency in the language and not treat them specially at the point of creation.
That scenario does bring up a question of whether or not a
zero-valued sum-type could be passed to a function, and I could go either way on that.
This has been discussed above as well. It seems hard to impossible to me to disallow it, without drastical changes to Go's type system. Whether or not a variable is zero is no longer a static property but a runtime property and trying to make static assertions about those tends to pretty quickly devolve into solving the halting problem. See also the various suggestions over the year to disallow dereferencing nil-pointers statically.
"IMO your suggestion is now back to the point where every sum type has a zero value of nil. It doesn't really matter if it is spelled nil or zero, as far as the contentious questions are concerned, as long as it the semantics are the same."
Admittedly there is not much difference, but there is one tangible difference; consistency.
If we allowed that sum types could just be nil then there would be the contentious question of consistency; i.e. that some variables that can contain nil would be handled differently than others. Using zero instead of nil removes that one objection, which is why I suggested it.
But yes, that is the only difference, however it could be the difference between someone objecting to sum types vs. supporting them. What percentage of people who would do each remains to be seen.
"That doesn't mean the suggestion isn't viable, it's just that it doesn't differ significantly from what we have been talking about so far."
Yes, and the difference is that the combination of things — at least on this issue — have not been discussed prior AFAICT.
"To me, that means FWIW that there is no need to disallow make etc."
With the addition of initializers in my suggestion there is no reason to disallow make() either, but I get that is likely orthogonal to your point.
"because if there is any way to create an invalid zero value, usage of a variant needs to be prepared to deal with it. If it needs to be prepared to deal with"
Yes, and that is where we disagree. My suggestion proposes making zero an exceptional case such that the vast majority of code would safely not deal with it because existence of a zero value and subsequent use would in itself be an exceptional case worthy of an immediate panic.
"might as well keep more coherency in the language and not treat them specially at the point of creation."
From a purity standpoint you are probably correct. But my understanding of Go's nature is that they have historically placed emphasis on pragmatism over purity. Otherwise there would have been no append(), copy(), etc.
Respecting the existing nature of the Go language, I am arguing that since it is impossible to find a perfect solution, maybe instead we could be pragmatic and accept a really good one?
The reason I think this approach could work is because there is only one way thus far I have discovered thus far that a sum type variable could have a zero value during non-exceptional coding practices, that one way is itself a test for validity so the compiler could ensure that it is not misused if we limit to not allowing them to be passed to funcs when they have zero values (your questioning has now convinced me this is the right approach.)
"Whether or not a variable is zero is no longer a static property but a runtime property and trying to make static assertions about those tends to pretty quickly devolve into solving the halting problem."
Unless I am missing something, it would seem easy to determine if a variable is used without ok being checked, for our one known non-exceptional case. That is a highly constrained problem vs trying to determine if an arbitrary program will run forever. We would have to limit to checking ok and not allowing things like if ok || myStatus()=1 {...} but that feels like a reasonable limitation — since a simple restructure of that if statement would create a knowable construct — given the value offered by a workable sum type.
So I ask this: rather than discuss in abstract terms, can you or others identify places where the compiler could not easily identify when a sum type variable received a value of zero and thus not be able to disallow it?
For one example--and just for one example, there are other problems--how do you handle a type assertion to the sum type if the type assertion fails?
If we had sum types, we could return an Optional sum type. So something like
var typeAsserted optional[MySumType] = nothing
typeAsserted, _ = anything.(MySumType)
(In practice, you would usually not declare the destination separately, I just did it to highlight its type.)
I understand this would be somewhat inconsistent with non-sumtype-type-assertions, but it sidesteps the zero-issue.
But if we can generalize it so nothing=zero=nil, maybe we can treat every type assertion as returning an optional, and optionals are implicitly convertible to zero values where available?
because there is only one way thus far I have discovered thus far that a sum type variable could have a
zerovalue during non-exceptional coding practices
For one example--and just for one example, there are other problems--how do you handle a type assertion to the sum type if the type assertion fails? What value do you give to the first result of the type assertion?
I feel like that should have made clear that this was just one example. As far as I can tell, to name a few others, you have not yet talked about channel-receives, map-accesses, reflect.New, extra capacity allocated by append, the statement var x T when T is a type-parameter (and any other statically disallowed code for these specific types), named returns (in particular in the presence of panic) or clear on a slice. There might be others.
I'll also note that the suggestion to disallow uninitialized values came up in this discussion before and most of this list has been posted there as well.
And while I appreciate that it is frustrating to be told that something you see as an easy solution is unworkable, I'd also ask for a little bit of trust that when people like Ian or I say things like "Zero values are built into the language too deeply", it's not just an off-the-cuff remark. We wouldn't say that, if we saw a realistic way to make it work.
In particular, listing instances of where zero values are mentioned in the spec is not meant as a request to special case solutions to them, but as a demonstration of what we mean when we say "zero values are built into the language too deeply".
This might be a bit off-topic, but have zero value semantics like this been discussed?:
func main() {
var m map[string]string
assert(m == zero)
assert(m == nil)
m = nil
assert(m != zero)
assert(m == nil)
m = zero
assert(m == zero)
assert(m == nil)
}
func assert(b bool) {
if !b {
panic("assertion failed")
}
}
And FWIW
Respecting the existing nature of the Go language, I am arguing that since it is impossible to find a perfect solution, maybe instead we could be pragmatic and accept a really good one?
I assume that the pragmatic solution we will eventually adopt (if any) is to use union-element interfaces as variants and make their zero value nil. To be clear, that's not my favorite solution, just the one that seems most pragmatic, given where we are. But it requires accepting the bad that comes with it and I don't resent the fact that we don't do that lightly.
@mrwonko Thanks. In these kinds of discussions, it is always possible to find a solution for any given problem. But it is also necessary to step back and consider the overall picture. Go is intended to be a reasonably simple, reasonably orthogonal language. When we add special cases we weaken those properties.
This proposal is, I think, a somewhat simple, reasonably orthogonal, change that we could make. The question here is not how to complicate it to make it better. We're almost certainly not going to do that. Rather than make it more complicated, we will choose to make no change at all. The question here is whether to make this change at all--that is, whether the benefits of the change are worth adding more complexity to the language. Or perhaps we can find a way to make it more simple and more orthogonal.
Odin lang is in many ways inspired by golang.
Odin has enum type that expresses sum type idea.
To instantiate an enum variable one has to spell out the concrete type
Foo :: enum {
A,
B,
C,
D,
}
f := Foo.A
https://odin-lang.org/docs/overview/#partial-switch
It is true that if golang tries to implement sum types via extending/overloading interface complications are guaranteed.
But what about creating a separate keyword enum ? This helps to sidestep a lot of complications that come from interface
type Foo enum {
A string
B int
}
f := Foo.A // OK
b := Foo // Compile failure, unspecified concrete type
@ngortheone Note that this issue is specifically about using union-elements in interfaces as variants. There are other issues (my personal favorite is #54685) to discuss other ideas and #19412 as an umbrella issue for the general idea of variant types.
I'll note that the vague notion of adding a new syntactical construct and type kind has been suggested a lot of times so far, so your suggestion isn't really novel.
so your suggestion isn't really novel
That probably means that the solution space to the sum type problem is small and the search has already exhausted all good options. The main question now is:
Knowing all pros and cons of each solution will golang decide to go for any solution at all?
I feel like that should have made clear that this was just one example.
Logically-speaking, why? I addressed the one example, and was looking forward to considering others.
As far as I can tell, to name a few others, you have not yet talked about channel-receives, map-accesses,
reflect.New, extra capacity allocated byappend, the statementvar x TwhenTis a type-parameter (and any other statically disallowed code for these specific types), named returns (in particular in the presence ofpanic) orclearon a slice.
I would tackle each of these, but from the tone of your arguments I don't feel like continuing what is evidently a contentious debate with you here on this issue.
I'll also note that the suggestion to disallow uninitialized values came up in this discussion before and most of this list has been posted there as well.
I had searched for "disallow" and "initialized" on this page prior to my posting and they appeared nowhere.
I searched again just now, but this time I opened up all the posts marked "off-topic" and found it was you telling @atdiar it wasn't possible, that culminated in his frustrated (your word) post before Robert Griesemer called for respectful discussion.
However, nowhere in that dialog did anyone other than you — i.e. no one from the Go team — argue against the idea.
So my takeaway is that you are asserting that if you already expressed an opinion against something that no one else should be able to discuss it? Just wanting to make sure I understand correctly.
And while I appreciate that it is frustrating to be told that something you see as an easy solution is unworkable,
No, it is absolutely not frustrating to be told something that what I presented as a strawman proposal is unworkable when objective and specific arguments against it are given. That is entirely the point of such a proposal to flesh out its feasibility.
"I'd also ask for a little bit of trust... We wouldn't say that, if we saw a realistic way to make it work. "
What is frustrating instead is to be told, effectively "We have already considered ever conceivable option and so you should just trust me that you have no value to offer here."
when people like Ian or I say things like "Zero values are built into the language too deeply", it's not just an off-the-cuff remark.
As George Bernard Shaw said "The single biggest problem in communication is the illusion that it has taken place." You assume the statement "Zero values are built into the language too deeply" are interpreted as you understand the phrase in a binary form exactly as you understand it without recognizing that others don't interpret that statement the same.
From my perspective my proposal absolutely respected that statement; why else would I have included the concept of zero to be applied to sum types if I was disrespecting Ian's comment? I was explicitly trying to address how useful sum types and zero values could coexist.
BTW, I really like how Ian engages in discussions on this forum. He always replies in a respectful manner, makes a statement when he needs to, but evidently doesn't feel the need to debate everyone who has a proposal, even if it is not one they will pursue. The Go team then ultimately makes their decisions and we all move forward. His approach makes everyone feel as if they can contribute, but is tactful when a discussion gets out of control and reigns it in with a statement of intent. It would be a lot nicer in these forums if everyone were able to participate without any self-appointed gatekeepers.
@mikeschinkel FWIW there is also #19412, which is more general, so contains more discussions of broader proposals than this one. Searching that casually brings up more discussion about these specific problems, involving a lot more people than me, including people on the Go team.
So, apologies for writing "this discussion". It was imprecise. The general discourse about variants has been going on for a while and I don't always remember where all parts of it happened.
Just to briefly underscore one point @Merovius has made a few times, I thought it might be helpful to re-post this snippet of Ian's original proposal text (from top comment above):
In discussion here, please focus on the benefits and costs of this specific proposal. Discussion of sum types in general, or different proposals for sum types, should remain on #19412 or newer variants such as #54685. Thanks.
(And of course, given how bad GitHub issues are for long conversations, it's worth keeping in mind the benefits of scaling with many conversations in places outside of GitHub issues, such as the #generics channel of Gopher Slack, which has friendly & thoughtful discussions, or elsewhere like golang-nuts, r/golang on Reddit, or by sharing a Gist you wrote in one of those places, etc.).
@thepudds — Given your comment it is worth noting that while some people may see discussion as being a different proposal, others making suggestions see it as addressing ways to make the original proposal viable.
Also, given the concept of scaling with many conversations, it would be respectful of and incumbent on those who have the time to seek out and follow many different discussions in many different places that not everyone is fully aware of to not seek to tamp down comments by others without at least first linking to their specific points from those other discussions, and especially before admonishing people for discussing things "that have already be addressed and resolved," but elsewhere. #fwiw
I think it would be very helpful if the resolved concrete types were also possible to list using the reflect package. If they are, it would be possible to add functionality to json.Unmarshal to write to these interfaces.
That is, this example would work, if json.Unmarshal would get the types [int, string] from the passed pointer and try to decode each in turn.
func main() {
var v interface{ string | int }
err := json.Unmarshal([]byte(`42`), &v)
fmt.Println(v, err)
}
would print 42, int. For var v any the underlying type would be float64.
I'm don't think the json package functionality would have to be part of this exact proposal. There would have to be decisions made about error handling, for example. But I would expect some support in the reflect package. Presumably reflect.Type.Implements will also need this list, regardless.
// SumTypes returns the concrete types implementing t.
// It panics if t is not an interface type.
// It returns nil if t does not have any type constraints set.
// The returned types are sorted in lexicographic order, including package path.
func SumTypes(t Type) []Type
With that documentation, I suppose methods would also be checked, otherwise Implements(t) might still return false on some of the returned types.
Also see https://github.com/golang/go/issues/68710#issuecomment-2265850918
In particular, the zero value of an interface type with an embedded union would be nil, just as for any interface type. So this is a form of sum type in which there is always another possible option, namely nil. Sum types in most languages do not work this way, and this may be a reason to not add this functionality to Go.
Is there any active proposal about nil safety? If so, how might it interact with the concern raised above?
@gonzojive not yet to my knowledge.
That would require to think about the usual type constructors in terms of accepting zeroable arguments or not. nil being the zero of interface types it would become a compile time error to create slices of such union/sum types for instance.
Easily resolved with a notation such as A | nil.
In fact, everything can be done. The question is whether that would be a good use of the complexity budget.
Even, type assertions could be handled properly.
In
w, ok:= v.(A)
where A would not have untyped nil in its typeset meaning nil wouldn't be assignable, we could still decide that such interface zero value is nil. Just assignment of nil would not be possible.
So w, ok = v.(A) might be forbidden unless w is a newly unassigned variable declared as var w A.
Then it's about what to do in the branch if !ok{...}
w is zeroed here and thus cannot be passed as argument where a A is expected until after it has been assigned with a value(which we know has to be non-nil).
This require some form of typestate analysis.
Overall nil is not an issue. The issue is nil in interface values. Because interfaces don't differentiate between value and pointer types when we call a method and because checking an interface value for the nilness of its content is not very practical yet.
Hindsight is always 20/20 unfortunately, I wish == applied to interface content including when it's a nil comparison and that we had another way to check for an interface being empty (whether ok:= a.(nil) or the shorter a! as in if err! {...}.
That would get rid of the faq entry on nil comparison, perhaps shorten error handling a tiny bit.
But anyway, this is all related. That's why I still like this proposal because it seems that it's just a piece of the overall puzzle.
Is this still alive? Recently I was writing a program, I use Value to simulate limited data types in json, like: integer->int64 / number -> float64.
In the absence of sum type, I had to use any to express it, but encountered a very disgusting API when using it.
I have to assert it, it is type-unsafe and very difficult to use.
This is still alive, but as the proposal says, this is a speculative issue. We're not working actively on it at the moment.
After thinking about this for a while, reading all of this issue and #19412, and coming up with my own design, I now believe that this proposal is the only practical way forward. All other designs add to the language, in some cases quite significantly. This one removes a restriction, making the language more orthogonal.
If you think your pet union design doesn’t add much to the language, consider that the spec would need to explain how it works with: assignment (including the case where the right-hand side is an untyped constant); type identity; equality; conversions; type assertions (as both expression and type); type switches; operators; and method sets. This proposal needs only a few tweaks to the spec, because unions are just interfaces.
Union interfaces used as types will have to be different in small ways from those used as constraints. “~” forms should be left out until we understand how they could be useful, as Ian wrote in the top post. Type parameters must be allowed. We should also allow interfaces with methods: we should be able to write unions like
type IOer interface { io.Reader | io.Writer }
That last problem needs to be solved for union interfaces as constraints anyway.
Having nil as the zero value of a union is unfortunate, but there is no other reasonable choice. It’s also not so terrible. For many unions, like the sure-to-be-popular Optional[T], nil is the right zero value. Those where a nil is unwelcome are no worse off than pointers that should never be nil. We can classify those cases together under the problem of nil safety, and treat them as Go programmers have been doing all along: avoid sprinkling nil checks everywhere, trusting that other parts of the code are correct. There are also decent static analysis tools for unwanted nils, notably Uber’s NilAway. The situation is not ideal, but it is better than any alternative.
Last but not least, there is the question of exclusive type switches. Should a type switch over a union require all cases (let’s call that union closed), or not (an open union)? Adding a type to a closed union is a breaking change, because type switches would need updating. Adding a type to an open union is a compatible change, at least according to the definition of “compatible” we typically employ, which only cares if programs still compile (and modulo the subtle considerations discussed here).
There are valid uses for both open and closed unions. A type like Optional[T] will never change, so it can be closed to provide a useful compile-time check. The same is true for unions representing mathematical concepts: a binary tree consists of leaves and interior nodes and nothing else ever. If a union isn’t exported, or lives in an internal package, then it can be closed without worrying about breaking other users, because all uses of the union are under the developer’s control. But many exported unions should be open. For example, the current version of the MCP specification says that a “progress token” must be a string or integer. The natural way to represent that is with a union. But the MCP authors could add a new type in the next version, and we wouldn’t want that to force a new major version on the Go SDK. If we use a union to represent a progress token, it must be open.
If we adopt this proposal as is, all unions would be open. Adding closed unions later would require some syntactic marker. I actually think the default should be closed, and open unions should be marked. I also think we can make type switches for open unions slightly safer.
The last term in a union may be the token .... A union without that term is closed: its type switches must cover every type in its type set, or must have a default case. A union with the ellipsis is open: its type switches must have a default case. The ellipsis is ignored except for its effect on type switches.
The required default case for open unions is intended to nudge users into panicking or returning an error if there is an uncovered type, so they can learn about it and decide how to handle it.
@jba what about https://github.com/golang/go/issues/57644#issuecomment-2265853797 i.e. #68710? I think that makes this proposal infeasible?
Since (consider the below snipped) foo is the same thing as any, not any or int.
type Foo interface {
any | int
}
var foo Foo = int(0)
if v, ok := foo.(int); ok {
// branch - hit
}
if v, ok := foo.(any); ok {
// branch - hit (Users would expect not to be hit)
}
Or is that expected?
@mateusz834 I don't have anything to add about that strange case. Why do you think it makes the proposal infeasible? Because the behavior is surprising?
@mateusz834 that's a very interesting example.
Am I mistaken, or Foo has the exact same semantics as the open union
type Foo interface {
int | ...
}
from @jba 's proposal?
If that's the case then open unions are already supported by the language, and one could just add the requirement for a default case in switches (i.e. no additional syntax required)!
UPDATE: unfortunately not, I just realized that requiring a default case for | any unions would be a breaking change :(
I think instead of | ... we should just embrace that these are inherently open. It would be easy to add an annotation to "closed" unions and have a linter that checks exhaustiveness.
Regardless, I'm not sure how you would write Optional[T] with these. If you have type Optional[T any] interface { T } what happens when it's instantiated with another interface type?
Because the behavior is surprising?
Yes, since that would be inconsistent with how ifaces with methods would work.
type Bar interface { Bar() }
type BarImpl struct{}
func (BarImpl) Bar() {}
type Foo interface {
Bar | *BarImpl
}
var foo Foo = &BarImpl{}
if v, ok := foo.(*BarImpl); ok {
// branch - hit
}
if v, ok := foo.(Bar); ok {
// branch - not hit
}
I think a different way to say what @mateusz834 is pointing out is that since the sum types in this proposal are interface types, they can't consistently and coherently represent sums of interface types.
Personally I don't think that makes the proposal infeasible. It's akin to the way that assigning a value of an interface type to a variable of some different interface type loses the right hand side's interface type. It means that we should not describe this extension as fully general sum types. We should describe them as enhanced interface types.
Of course we could decide that this means that the proposal is not worth doing.
We could also consider banning these general purpose interface from having an interface type as a sum element. (Except perhaps for | any as an indicator that any type is permitted--that the sum is "open".)
When I was thinking about it, I had come to the conclusion that any union case written should be considered discriminated and for full support, the dev should explicitly specify the intersection as one of the case.
That is consistent with how ~T would be handled as well.
Otherwise it gets too confusing.
even any | int should not actually be equivalent to any: We would require {any | int | interface{any;int}} the later element being empty and only then ending up equivalent to any | int.
the presence of int should induce higher specificity. (I know there is an issue that discussed this, I don't think we had reached the right conclusion on this one).
The disjunction remains part of the definition. (especially relevant for constraint usage and ensuring the commutatibility of the union operation, only then int | any <=> any | int)
I'm sympathetic with @jba analysis but I believe we should try and tackle the issue in a more principled manner.
I understand that it is harder to implement and touches many elements of the language.
Regardless, I'm not sure how you would write Optional[T] with these. If you have type Optional[T any] interface { T } what happens when it's instantiated with another interface type?
Seems like a new kind of syntax might be needed, like:
type Optional[T] interface {
T |
}
It's akin to the way that assigning a value of an interface type to a variable of some different interface type loses the right hand side's interface type. It means that we should not describe this extension as fully general sum types. We should describe them as enhanced interface types.
As I understand it, it is definitely incorrect to call them "sum types", but it is fully correct to call them "union types". In that way, I think the title of this issue is just wrong; the proposal is to add unions, not sums.
The difference has been mentioned in this issue a couple of times. I also recently explained it at the Zurich Go meetup (unfortunately the recording setup was pretty bad).
enum MySum {
Int(i64),
Duration(i64), // note: same values as the Int term
}
fn main() {
let mut x: MySum;
x = MySum::Int(42); // MySum::Int is the tag
x = MySum::Duration(23);
}
var (
x io.Reader+*bytes.Buffer
y *bytes.Buffer)
)
x = io.Reader(y)
switch x.(type) {
case *bytes.Buffer:
panic("not taken, as x is tagged as a Reader")
case io.Reader:
fmt.Println("taken")
}
type MyUnion = number | string;
let x: MyUnion;
x = 42;
x = "Hello, world";
@jba In general, I agree with
I now believe that this proposal is the only practical way forward. All other designs add to the language, in some cases quite significantly. This one removes a restriction, making the language more orthogonal.
Personally, this makes me sad, though, because I really think this is not a good design and that it will make no one actually happy.
We should also allow interfaces with methods: we should be able to write unions like
go type IOer interface { io.Reader | io.Writer }That last problem needs to be solved for union interfaces as constraints anyway.
[…]
The last term in a union may be the token .... A union without that term is closed: its type switches must cover every type in its type set, or must have a default case.
I want to advise extreme caution here.
Exhaustiveness checking is trivial for sums, but potentially NP-complete for unions. With a sum, you can simply check that a match statement lists all the terms ("colors") of the sum. Every value falls into exactly one of those branches. With a union, branches can have overlaps, because a value can be contained in more than one term.
For Go, that would be especially the case, if we allowed methods in unions. We'd need to be really careful that exhaustiveness checking does not become NP-complete (it would definitely be when using them as constraints). Or we need to be clear that we are okay that type-checking Go requires solving an NP-complete problem.
I also bring up another problem in my talk:
type Signed interface{ int8 | int16 | … | int64 }
type Unsigned interface{ uint8 | uint16 |…| int64 }
type Integer1 = interface{ Signed | Unsigned }
type Integer2 = interface{ int8 | … | int64 | uint8 | … | uint64 }
Currently - and generally, when considered as unions intuitively - Integer1 and Integer2 are identical. So, would we
switch to mention all the int8,int16,… cases as separate branchesSigned | Unsigned covers all cases (in which case, are we sure we want to allow methods?), orOh and as I overlooked it:
We could also consider banning these general purpose interface from having an interface type as a sum element. (Except perhaps for
| anyas an indicator that any type is permitted--that the sum is "open".)
This would address most of the issues I mention with unions, as it removes the possibility of a case being in multiple terms - it effectively makes the union disjoint. As long as we also do something about chan T/chan<- T/<-chan T, which is the other case I can think of where a value can be in multiple types. But that seems easy to address with a special case somehow (waves hands).
That also means it disallows a lot of currently existing constraints as types. So it would either maintain a class of interfaces that can only be used as constraints, or it would be breaking as we had to disallow existing nested unions. It also just generally makes this less useful.
(Usual disclaimer applies: doesn't mean we can't do it, just like everything I said above. Just that we should be clear on the consequences beforehand)
Because the behavior is surprising?
Yes, since that would be inconsistent with how ifaces with methods would work.
```
type Bar interface { Bar() }
type BarImpl struct{}
func (BarImpl) Bar() {}type Foo interface {
Bar | *BarImpl
}var foo Foo = &BarImpl{}
if v, ok := foo.(*BarImpl); ok {
// branch - hit
}if v, ok := foo.(Bar); ok {
// branch - not hit
}
```
@mateusz834, I don't understand why the branch is not hit in the second case. A *BarImpl is a Bar. Why can't A | B mean "A or B or both"?
You've also written that type Foo interface { any | int } "is the same thing as any, not any or int." I think of these things as ways to describe type sets, so I don't find that surprising. It's like saying (using set notation, not Go): {animals} ∪ {dogs} is the same set as {animals}. That is what you'd expect.
since the sum types in this proposal are interface types, they can't consistently and coherently represent sums of interface types.
@ianlancetaylor if you mean disjoint sums, I think you could represent those with a level of indirection:
type Any struct { any }
type Foo interface { Any | int }
I think instead of
| ...we should just embrace that these are inherently open. It would be easy to add an annotation to "closed" unions and have a linter that checks exhaustiveness.
I guess, but it seems like a missed opportunity. I really like the ... syntax.
Regardless, I'm not sure how you would write
Optional[T]with these. If you havetype Optional[T any] interface { T }what happens when it's instantiated with another interface type?
interface { I } is just I, so you'd just get the interface back, which already has a nil zero value. I think that's fine, just not useful. The real pitfall is Optional[*T], because that can hold (untyped) nil, a nil pointer, or a non-nil pointer. Maybe there should be a linter for that.
it will make no one actually happy
I guess I'm a counterexample. I would happily write
type Optional[T any] interface { T }
// nil means absent
----
package mcp
type ProgressToken interface { int | string | ... }
// nil is needed to express that progress tokens are optional.
----
package jsonschema
type Schema struct {
Type interface { string | []string }
...
}
// The "type" keyword can be a string or a JSON array of strings, or it can be omitted.
// Currently represented as two fields, Type and Types: https://pkg.go.dev/github.com/google/jsonschema-go/jsonschema#Schema
----
package some_tokenizer
type Token interface { int | string | bool | rune }
// Maybe we just ignore nil, or maybe it means some particular token like EOF.
@mateusz834, I don't understand why the branch is not hit in the second case. A *BarImpl is a Bar. Why can't A | B mean "A or B or both"?
That is my understanding of union types. That v.(T) (and in the same way type switch) would not
follow normal interface rules, but only check whether it is one of the types listed in the union.
type Foo interface {
Bar | *BarImpl
}
only v.(Bar) and v.(*BarImpl) would be allowed.
But I see what you are suggesting, I am just not yet convinced this is the right approach.
I wonder about other corner cases that might arise with that approach, but at least now it makes sense to me (at high level) how this would work with ifaces.
I guess I'm a counterexample. I would happily write
I can't help but feel that this ignores 90% of the relevant arguments I made. Like, you expressed a bunch of opinions in your comment about what we should do and I made the argument that we probably can't do most of them.
But fair enough, it would make you happy. But the people who make fun of Go for not having variants will have a field day and the people who like Go to be simple and understandable will generally be angry at struct{ Stringer fmt.Stringer } | string.
For Go, that would be especially the case, if we allowed methods in unions. We'd need to be really careful that exhaustiveness checking does not become NP-complete (it would definitely be when using them as constraints). Or we need to be clear that we are okay that type-checking Go requires solving an NP-complete problem.
Agreed, we must be wary of anything that is computationally hard, or even super-linear. I don't think we want to go the route of C++ and allow NP-hardness into the compiler.
What if a union can only have one disjunction in addition to a list of methods? That's what people generally mean by unions anyway. Do you think that's still NP-complete? I don't think your current proof goes through with that change.
I actually think that gives a quadratic algorithm for typeset subset. Not great, but maybe do-able with clever indexing and an implementation restriction on the size of a disjunction.
We can use typeset subset to compute switch exhaustiveness. A switch over a union type U is exhaustive if the type set of U is a subset of the type set formed by taking the disjunction of the switch cases. Which means we could pick choice 2 in https://github.com/golang/go/issues/57644#issuecomment-3669221805: force the compiler to prove exhaustiveness.
I can't help but feel that this ignores 90% of the relevant arguments I made. Like, you expressed a bunch of opinions in your comment about what we should do and I made the argument that we probably can't do most of them.
Sorry, for some reason I was focused on the nil zero value, even though you never said that.
I started to address the methods-on-interfaces issue above. Was there something else?
people who like Go to be simple and understandable will generally be angry at
struct{ Stringer fmt.Stringer } | string.
Unless I misunderstand what you mean by that — and it is certainly possible that I do — I would be quite the opposite of angry if that were possible. A project I am working on right now could really benefit from that capability.
To elaborate on my current understanding, my project could benefit if it could define a type that can accept either a string or a value of a type that implements fmt.Stringer. Even better if it could also accept ~string.
What if a union can only have one disjunction in addition to a list of methods?
I am not entirely sure what you are suggesting. My intuitive understanding of what something like
type X interface{
string | int
Foo()
}
means, is what it means today (and should mean in the future): either a string, or an int, and it has to have a Foo() method. That would work, but it isn't really removing the restriction of interfaces with methods in unions; the restriction would remain.
It also would rule out many things for which variants could be useful. I think fmt.Stringer | string is an obvious candidate. But also type Result[T any] interface { T | error } wouldn't work. I think most people who want to use interfaces with methods in unions want the methods to provide fallback behavior: "here's a list of concrete types I can handle, if you want something else, here are some methods I would need".
One way to get that is to say "we allow interfaces with methods in a union, but those interfaces can't have any unions themselves and you can only have one union". I think this might have come up in a conversation with @griesemer and @ianlancetaylor a while back. At first glance at least, that would also be fine.
In any case, it kind of depends on the details and I've arguably been rambling for too long, given that I just didn't really understand what you where suggesting in the first place.
Sorry, for some reason I was focused on the nil zero value, even though you never said that.
I think the nil zero is kind of sad, but honestly, fairly low on my list of worries. I'd be fine with that. And there really is no way to avoid it, if we want to use the union syntax.
I started to address the methods-on-interfaces issue above. Was there something else?
No, really, my main concern is the question of how to do exhaustiveness checking and/or allow methods in unions. That's where I just haven't come up with a feasible solution myself. It's what makes me think "this doesn't work" (or rather "this would introduce a bunch of really strange new restrictions to the language"), not just "this isn't aesthetically pleasing to a theorist" (which, well, it's Go. It's not a theorist's language).
my project could benefit if it could define a type that can accept either a string or a value of a type that implements fmt.Stringer.
I agree, it would be very useful. And it would be a kind of sad design, if you couldn't do that without hacks like wrapping your interfaces in a struct. It would add a lot of syntactic noise and would just look strange to most people, I would argue.
Note that it is possible already.
@Merovius I meant one fully general disjunction, including interfaces with methods.
I was under the impression that your reduction requires an interface with multiple disjunctions:
type AtLeastOne interface {
X | NotX
Y | NotY
Z | NotZ
}
I gather from your statement
One way to get that is to say "we allow interfaces with methods in a union, but those interfaces can't have any unions themselves and you can only have one union"
that allowing nested unions enables the reduction, but I don't quite see that.
I agree, it would be very useful. And it would be a kind of sad design, if you couldn't do that without hacks like wrapping your interfaces in a struct. It would add a lot of syntactic noise and would just look strange to most people, I would argue.
It would definitely be a shame to have to wrap the interface in a struct.
That said, if it were the difference between having the capability with syntactic noise vs. not having the capability at all, I would vote for the latter, all day every day as I am a pragmatist. I am however aware that not every constituent feels the same.
As an aside, I do miss what felt like a more pragmatic phase of Go’s evolution, which originally drew me to the language. Early design choices like introducing built-ins such as append showed a willingness to use special-case language support when it materially improved developer ergonomics, rather than insisting everything be expressible as ordinary library code.
Note that it is possible already.
Possible yes, for package functions. But not for methods, where I need them. 😢
I don't feel interfaces are the right foundation to build sum/enum/union/variant/whatever types upon. They seem similar, but the details are all wrong.
I just filed #76920 with an alternate approach.
In that issue, I suggest some motivating examples which any sum type design should support well:
Considering those in terms of sums-as-interfaces:
type North, South, East, West struct{}
type Direction interface { North | South | East | West }
var d Direction = North{}
if d == North{} {
// go north
}
switch d.(type) {
case North:
// go north
}
// or
switch d {
case North{}:
// go north
}
The Direction case works, but it is clunky. We need to define a type for each variant. We don't have nice constants describing variants--we write "North{}" rather than "North". We need to have nil as one of the possible values of a Direction, whether we want it or not. We can't have a String method on Direction.
For Maybe, I'm not sure how we define it under this proposal.
type Maybe[T any] interface {
T
}
This seems sort-of reasonable, since a Maybe[int] is either nil or an int. But unless I'm missing something, a Maybe[io.Reader] wouldn't be able to represent an io.Reader(nil) value.
So perhaps we do something like:
type Value[T any] struct { V T }
type Maybe[T any] interface {
Value Value[T]
}
var m Maybe[io.Reader] = Value[io.Reader](nil)
if v, ok := m.(Value[io.Reader]); ok { ... }
...but that looks terribly cumbersome.
For the IP address case, we again can't define a String method on the type and we need to handle a nil case whether we want it or not.
Perhaps I'm missing something, but I'm not convinced using interfaces as the basis of enums lets us support any of these use cases well.
For what it's worth, I'm already using a variant of that Maybe type in real code in today's Go.
In my case the "maybe" is representing known vs. unknown instead of present vs. absent so I have type Known[T any] struct instead of type Value[T any] struct, and my interface has an unexported method that Known implements because that's the only way to write a "sealed" interface today, but otherwise it's just like in the previous comment and mechanically it works... it looks a little ugly in use, but not so ugly that I don't want to use it.
I agree that the other two situations in the previous comment are far less satisfactory in today's Go, though. I have several examples of similar flavor to the "IPv4 or IPv6 address" use-case and again it can work in today's Go, but it gets increasingly cumbersome to use as the "pseudo-union" grows to include more variants.
If a proposal like the alternative in https://github.com/golang/go/issues/76920 can be made to work then I would enjoy that, though I must admit I'm concerned it's going to have a similar fate as the previous enum/union-shaped proposals for solving this problem, since it has similar characteristics as many of those. I liked this proposal because it seemed like a pragmatic compromise that would take some patterns that are already used in Go today and "make them official" as language features, but of course that means it does inherit many of the drawbacks of those current patterns.
Whether via an enum/sum type or a union as interface, the same problem will remain.
You have to handle the zero value for instance.
I will note that the union has to be discriminated and there is no NP completeness issue in that case.
Also interface{
string | int
f()
}
does not make much sense but that might be a forgivable mistake as basic types can't have methods.
It's important to think about discrimination especially for the constraint case usage (for type parameter switches if we get to have them)
I agree that for union interfaces in type usage it seems to be workable although if we don't discriminate union cases we lose commutativity and then it can get confusing when trying to find out what exactly is put in a value of such union. Possibly, it could require to check all cases.
That's what @mateusz834 was sensing.
I honestly don't think there is a better choice than biting the bullet but it is more complex than just using interfaces. But not undoable.
For the fun note, gemini predicted that it could take a couple months/years to implement when I asked a few weeks ago.
But with typestate tracking for nil values and everything.
I meant one fully general disjunction, including interfaces with methods.
So, would this be allowed, for example (it is today)? And what if we replaced string and int themselves with interfaces? Because if so, I don't see how it would make a material difference. If not, I'm not sure how the rules can be clearly phrased and conveyed.
type A interface {
string | int
Foo()
}
type B interface {
string | int
Baz()
}
type C interface {
A
B
}
In general, I'm not sure this format of discussion works. The details matter. Even with a pretty detailed design doc about making methods in unions work, including a description of the type checking algorithm, it took me a good day or so to find the hole in it and make an argument. Plus, I'm by no means even an expert in any of this. I just don't think this is something that can be discussed to (my) satisfaction on a github issue. All I'm saying is, that if we want to prevent needing a SAT solver to implement type checking, "exhaustiveness should be checked" isn't something one should just put in a design, without explaining how precisely. @griesemer and @ianlancetaylor have spent some time on trying to figure out how to allow methods in unions and so far, it hasn't happened.
~@neild~
~In that issue, I suggest some motivating examples which any sum type design should support well:~
Result[T] type (an error or a T) to that list, as well as a Stringish (fmt.Stringer or string (or ideally ~string)) to it.~[edit] actually, I'll do this on #76920 directly
Sums are based on direct sums of sets. You can have e.g. ℤ⊕ℚ and it contains a "1 from ℤ" and a "1 from ℚ" and they are different.
It has been a very, very long time since my abstract algebra class, but I do not believe this is how direct sums work. A direct sum is a set of tuples where each tuple contains one element from each of the summands. The direct sum of the x and y axes is the coordinates in a plane.
The reason why it's called either "sum" or "product" is because that's how you calculate the number possible instances of the combined type.
All possible tuples of (any X, any Y) has the number of items in the combined type as X*Y. This is multiplicative = product type ~= struct with two fields
All possible tuples of (any X, null) union (null, any Y) has the number of items in the combined type as X+Y. This is additive = sum type ~= haskell sum / rust enum
@neild Hm, I apparently misremembered. How embarrassing. I really had that stored pretty firmly in my mind.
Nevertheless, there is a clear distinction between unions and disjoint unions¹. And the types discussed in #76920 are disjoint unions and the ones discussed here are not.
[1] This is the definition I had in mind. In particular:
For i ≠ j , the sets
A*_iandA*_jare disjoint even if the setsA_iandA_jare not.
Using the same syntax and rules as type constraints is still beating all the counter arguments and variations in my mind. There are certainly ways something similar could be more useful but that means having two very similar but different things in the language.
But unless I'm missing something, a Maybe[io.Reader] wouldn't be able to represent an io.Reader(nil) value.
@neild, Is io.Reader(nil) different from the ordinary (untyped) nil that is the zero value for io.Reader? If so, I can't figure out how to construct one or how it would be useful.
I think of Maybe[T] as useful only for types that don't already have nil, like basic types, arrays, structs and so on. It adds no value for interfaces, and is actually harmful for pointer types because it gives rise to two forms of nil, typed and untyped.
Using the same syntax and rules as type constraints is still beating all the counter arguments and variations in my mind. There are certainly ways something similar could be more useful but that means having two very similar but different things in the language.
This. It looks like we are comparing two equal-sized concepts because each has one issue. But another way to look at it is that we are comparing a net 3-sentence addition to the spec (remove 1, add 4) that removes a restriction on an existing concept with a 20-sentence addition that introduces a new concept. (Numbers are approximate of course.)
I think of Maybe[T] as useful only for types that don't already have nil, like basic types, arrays, structs and so on. It adds no value for interfaces, and is actually harmful for pointer types because it gives rise to two forms of nil, typed and untyped.
I think of Maybe[T] as useful for cases where you want to distinguish between the zero value and no value.
For example, Maybe[error] could be used to distinguish between an operation which has not completed (no error yet) and one which has completed successfully (nil error).
Maybe[T] and Result[T] can be made with generic structs and accessors/factories to hide the invalid states today, if desired. One of the main reasons you wouldn't do that in functional languages is deconstruction in pattern matching which isn't relevant here
But another way to look at it is that we are comparing a net 3-sentence addition to the spec (remove 1, add 4) that removes a restriction on an existing concept with a 20-sentence addition that introduces a new concept.
I don't think the size of the spec addition is a good measure of whether we should do something. That optimizes for convenience to the language designers, at the potential expense of everything else.
I don't think sum types based on interfaces do a good job at implementing any of the examples I mentioned above (https://github.com/golang/go/issues/57644#issuecomment-3672503206): Direction, Maybe, IP.
Are those bad examples of things that we don't need to support? Rust considers Optional to be so fundamental that it's part of the core language. An IP address which can be IPv4 or IPv6 seems like one of most basic possible uses of a sum type. Are there better examples that we can use to evaluate a proposal?
can be made with generic structs and accessors/factories
That's true for any use of sum types. Nothing we're discussing is something that can't be done within the language today, except a requirement for exhaustive switches.
Using the same syntax and rules as type constraints is still beating all the counter arguments and variations in my mind. There are certainly ways something similar could be more useful but that means having two very similar but different things in the language
having the interface syntax being used also for constrains is confusing enough although the familiarity helps with the simple cases. overloading a 3rd concept will make things much harder. Again, on simple cases the familiarity brings you a false sense of confort. it's the edge cases that make you go around trying to understang why something that should be working by analogy is not working.
Maybe[T]andResult[T]can be made with generic structs and accessors/factories to hide the invalid states today, if desired. One of the main reasons you wouldn't do that in functional languages is deconstruction in pattern matching which isn't relevant here
I think it is more useful to think of those as stand-in. Maybe[T] can stand in for a slew of patterns where variants are used in data structures, to denote "other nodes, or nothing". Or used for optional nodes in an AST. Yes, you can model it as a struct, but you can model every union as a struct with accessors/factories to hide invalid states. So if that's the measuring stick, let's not do variants at all.
Result[T] is interesting here, because it uses T | error (disallowed today). But that's a more general problem, that applies to string | fmt.Stringer as well. And it shows that Result[error] - which is reasonable on its face - presents a problem, because it suddenly is just error, which applies just as well to other cases where you use terms which are interfaces and terms which are type parameters.
It's not about necessarily doing these, it's that they are very common use cases for variants, so if we can't do them well, that indicates a design flaw.
Nothing we're discussing is something that can't be done within the language today, except a requirement for exhaustive switches.
Maybe[T]can stand in for a slew of patterns where variants are used in data structures, to denote "other nodes, or nothing". Or used for optional nodes in an AST.
In languages with maybe/optional they generally only use that. If you have multiple things or none you Maybe[all the things].
If all the nodes are pointers to structs, you could encode that as *Node1 | *Node2 | ... | *NodeN where nil operates as None.
The issues with Maybe[T] don't come up because the issues with it are (a) it's confusing to have a single summand because of the ambiguity with interface embedding vs declaring a singleton union (for lack of a more rigorous term) and (b) interface types as summands don't work how you'd like (which is also the problem with Result[T]). Those aren't pros by any means.
It's not ideal but it removes a special case from the languages and provides enough to improve the status quo and enable things, even if you have to write a wrapper type sometimes to get the desired behavior or use a linter to check for exhaustiveness. If the situation were "look, either we add this or nothing ever," I'd much rather have this than nothing.
In languages with maybe/optional they generally only use that. If you have multiple things or none you
Maybe[all the things].
data Tree a
= Leaf
| Node a
(Tree a)
(Tree a)
If all the nodes are pointers to structs, you could encode that as
*Node1 | *Node2 | ... | *NodeNwhereniloperates as None.
Sure, you can always model everything as something else. What people find attractive about not modelling Maybe[T] as a pointer is that it enforces (or at least strongly encourages) a check before dereferencing.
It's not ideal but it removes a special case from the languages and provides enough to improve the status quo and enable things, even if you have to write a wrapper type sometimes to get the desired behavior or use a linter to check for exhaustiveness. If the situation were "look, either we add this or nothing ever," I'd much rather have this than nothing.
That's fair enough. But it doesn't mean we can't talk about the downsides, does it? Talking about things like Maybe[T] and Result[T] and their limitations with this design is important. If nothing else, then so that we know about them when writing documentation.
Are there better examples that we can use to evaluate a proposal?
Most of the development we do on our daily basis, is to model domain-driven facts, processes or events. Forgetting cases in the domain can get very expensive. Therefore, we try to be mostly as exhaustive and explicit as possible in relevant areas. We use all kinds of handwritten box types, marker interfaces, custom json marshalers (e.g. adjacent or externally tagged) and exhaustive matching functions. So yes, one can do everything today even in multiple ways, but it is way to much (handwritten) boilerplate and still more error prone than it ought to be. Using interfaces raises also efficiency concerns due to heap allocations, e.g. during event replays.
But another way to look at it is that we are comparing a net 3-sentence addition to the spec (remove 1, add 4) that removes a restriction on an existing concept with a 20-sentence addition that introduces a new concept.
I don't think the size of the spec addition is a good measure of whether we should do something. That optimizes for convenience to the language designers, at the potential expense of everything else.
It's about keeping the language small so it can fit in your head. Not literally minimizing effort for language designers. With this proposal, there is very little for a Go programmer to learn. With the others, there is a fair amount.
I don't think sum types based on interfaces do a good job at implementing any of the examples I mentioned above (#57644 (comment)): Direction, Maybe, IP.
Are those bad examples of things that we don't need to support? Rust considers Optional to be so fundamental that it's part of the core language. An IP address which can be IPv4 or IPv6 seems like one of most basic possible uses of a sum type. Are there better examples that we can use to evaluate a proposal?
Except for Optional, those are not motivating examples for me. I think enums like Direction are handled adequately by consts and iota. Some examples that are relevant to me:
type ProgressToken interface { int | string }
type Schema {
Type interface { string | []string }
}
In both of these cases (the MCP SDK and the jsonschema package), these types are used to express data that's going to be marshaled, so there's no need for them to have methods. But I would say that if you want behavior to apply to a union type, you can write a function with a type switch, or you can use named types with a method:
type IPV4 ...
type IPV6 ...
type IPAddr interface {
IPV4 | IPV6
String()
}
Your Optional[error] example is interesting. I've never felt the need for Optional[some interface type]. The Google Cloud SDKs have modeled Optional as any for some time, where it's only been needed for basic types and time.Duration. Here are examples from the storage, pubsub, and bigquery SDKs. In all other situations where absence needs to be indicated, the nil interface or pointer has worked.
@jba —
I think enums like Direction are handled adequately by consts and iota.
The issues with const and iota used as enums include:
iota results in fragile code, especially for long lists where developers have to be certain to never change the order, which can too easily happen accidentally, and also to always maintain placeholders (at least) for long deprecated values. (I always avoid using iota for fear I might accidentally break relied-upon values in the future.)Having a closed set that has no underlying type and that cannot be set to an invalid value set via a literal would be a substantial improvement to the language.
I think what this means is there would be a benefit to having enums that have no underlying types and whose tags(?) are the literals for the value.
Note that the concerns about iota are orthogonal to the argument for first-class no-value enums; the core issues are 1. and 2.
I think enums like Direction are handled adequately by consts and iota.
There's also cases where both are mixed. For example, a while back I wrote a parser for Portable Game Notation, used to represent chess games. The Moves could be represented by something like
enum Move {
Basic(Piece, Square)
WithFile(Piece, Square, File) // disambiguate two pieces that can move to a square by file
WithRank(Piece, Square, Rank) // disambiguate by rank
Promotion(File, Piece)
EnPassant(File, Square)
Castle
LongCastle
WhiteWin
BlackWin
Draw
}
And sure, this can be done by adding a concrete type for everything. And then taking the union of it. But it's quite a bit of boilerplate. And if you wanted a method on it - e.g. I think it should really implement fmt.Stringer - you have to declare ten versions of it.
It can be done. But I hope no one really believes this design does it well.
Except for Optional, those are not motivating examples for me. I think enums like Direction are handled adequately by consts and iota.
This is too dismissive of something that many have requested since the beggining.
No, consts with iota are not an adeguate replacement for enums like direction. it's missing the most basic property of enums, that of being closed.
with iota, nothing forbids someone to pass any integer into a function expecting a Direction.
No, consts with iota are not an ade[q]uate replacement for enums like direction. it's missing the most basic property of enums, that of being closed. (@doggedOwl )
[Not being closed] means all code that interacts with these enum's values must be validated, or be known to have been validated. (@mikeschinkel)
Very few enums should be truly closed, in the sense that adding a value is a breaking change. The reasoning is the same as for unions (see https://github.com/golang/go/issues/57644#issuecomment-3667286818, paragraph beginning "There are valid..."). Therefore, code that uses an enum has to be written defensively anyway, in case the package exporting the enum adds a value. For example, a switch over the values should have a default case.
@jba That has not been my experience with Rust's enums. The ability to exhaustively match on enums is one of their greatest benefits, and it's common for Rust crates to expose closed enums.
Having to litter Go code for switching on sum type values with unreachable default cases would be a real annoyance in my opinion. It's also a safety and correctness hazard: then when adding a new variant (which is common especially for internal types) the compiler will be unable to detect which select statements must be updated to handle the new variant.
code that uses an enum has to be written defensively anyway, in case the package exporting the enum adds a value.
There should also be a way to mark a sum type as non-exhaustive (as with Rust's non_exhaustive attribute), but for closed types, the options for module authors are: (1) don't add new variants or (2) only add new variants with a major module version bump.
@jba —
"Very few enums should be truly closed, in the sense that adding a value is a breaking change."
I think this is conflating two separate concerns:
const/iota mostly fails at (2): even if the intended set is {Red, Green, Blue}, the type admits other values and the compiler can’t help you.
Example: showColor(4) compiles and only fails at runtime:
When you say “very few enums should be truly closed,” that’s really about (1) (forward compatibility / non-exhaustiveness), not about (2) addressing invalid values today.
A useful design point (used in other languages) is:
This ensures consumers must write a fallback.
I’m also increasingly sympathetic to #76920’s direction, but sticking to #57644-style syntax here: the proposed ... term looks like a clean way to express “extendable” without conflating it with “invalid values exist today.”:
package main
import "fmt"
// Variant types, very biolerplatey
// Also, sadly, must be package-unique cs. scope to the enum
type North struct{}
type South struct{}
type East struct{}
type West struct{}
// Direction is a CLOSED union: "valid values are exactly these types"
type Direction interface {
North | South | East | West
}
// ExtendableDirection is NON-EXHAUSTIVE / extendable: "valid values may grow over time"
type ExtendableDirection interface {
North | South | East | West | ...
}
func DirectionString(d ExtendableDirection) string {
switch d.(type) {
case nil:
return "<nil>" // invalid today (and always)
case North:
return "North"
case South:
return "South"
case East:
return "East"
case West:
return "West"
default:
// Forward-compatible: after future extension you’ll get e.g. "mypkg.NorthEast"
// rather than silently mapping it to an existing value.
return fmt.Sprintf("Direction(%T)", d)
}
}
Side note on ergonomics: if “interfaces-as-enums” can’t support attaching methods directly to the enum type (or something equally ergonomic), that’s a significant gap for me and more reason to favor #76920. Ideally you would be able to write something like this:
// Hypothetical ergonomic goal (not valid Go today):
func (d ExtendableDirection) String() string { /* ... */ }
If the answer is “wrap it,” that works, but it’s extra indirection/boilerplate just to attach behavior, which makes interfaces feel like a strained vehicle for enums/sum types.
BTW, I scanned ~75 real-world enum-ish types in the code of my currently active projects; ~20–25% were conceptually closed today (non-declared values should always be errors, even in the future), and not handling 20–25% of the use cases does not feel fully adequate.
I’m also increasingly sympathetic to #76920’s direction, but sticking to #57644-style syntax here
The problem is that the syntax of general interfaces prescribes certain semantics, unless you want to introduce massive inconsistencies. Two simple examples are:
| is commutative, so we can not rely on order to determine the zero value, so we need nil as a dedicated sentinel zero value.Result[T] That's the entire reason for filing #76920, to break free of the semantic implications of staying within the existing union syntax. I'm really not even sure what the difference would be, between this proposal and "#76920 with union syntax".
I interpreted...
I’m also increasingly sympathetic to #76920’s direction, but sticking to https://github.com/golang/go/issues/57644-style syntax here
...as "I would rather adopt #76920, but the following example uses the syntax of the current proposal." I don't think @mikeschinkel meant to say we should combine the two proposals, but correct me if I'm mistaken. 🙂
Oh yes, I probably misparsed that, apologies.
I don't think @mikeschinkel meant to say we should combine the two proposals, but correct me if I'm mistaken. 🙂
Thank you. You are indeed correct.
I was instead being respectful to @jba to use examples relevant to his proposal when commenting on his proposal.
Very few enums should be truly closed
Starting form the example you opposed, a Direction type is pretty closed given a domain geometry.
Base colours are closed for given colous scheme. (RGB, CMYK etc). WeekDays and Months are closed. these are just some the basic everyday closed enums.
In a given domain there can be tens of other set of options that are closed.
a Direction type is pretty closed given a domain geometry
...until the system can handle 45-degree directions like Northwest, etc.
In a given domain there can be tens of other set of options that are closed.
Definitely true. There are many real-world concepts that won't ever change, like the ones you mentioned.
But most computer systems, even ones that use such concepts, end up defining a lot of their own. Consider cloud APIs. Most of the enums there are made up for the particular service and could easily grow.
As an example, consider a blob storage service like Amazon S3 or Google Cloud Storage. There are few or no real-world concepts in that API. Here are some enums from the Go storage SDK, all of them open:
https://pkg.go.dev/cloud.google.com/go/storage#RetryPolicy
https://pkg.go.dev/cloud.google.com/go/storage#SigningScheme
https://pkg.go.dev/cloud.google.com/go/storage#Liveness
Or we could look at the standard library. Yes, it has Month and Weekday, but here are some open ones:
https://pkg.go.dev/log/syslog#Priority
https://pkg.go.dev/database/sql#IsolationLevel
https://pkg.go.dev/log/slog#Level
https://pkg.go.dev/crypto/dsa#ParameterSizes
I don't know what the ratio here is, but in my experience, there are far more open enums than closed ones. If you are working closer to a particular domain, like writing a physics simulator or a banking system, then you may see things differently.
it's common for Rust crates to expose closed enums...for closed types, the options for module authors are: (1) don't add new variants or (2) only add new variants with a major module version bump.
@jonathansharman I'm glad to hear that the Rust community takes backward compatibility seriously. But given that, I find it hard to understand why there are so many closed enums. Maybe Rustaceans are more comfortable with major version bumps than Gophers are?
Can you share some examples (links to crates.io if possible) of closed enums?
@jba —
I (and I assume most others would) agree with your broader point: most enums in many APIs (cloud SDKs, service-specific config, etc.) are open to future extension.
My concern is that “open to future extension” is a different axis than “values outside the declared set are valid today.” With const/iota, you can’t prevent invalid values today, which means you lose the ability to model the subset of enum-ish types where “non-declared values should always be treated as bugs.”
The ... idea you first proposed seems like it could let us model both flavors cleanly:
closed/exhaustive union (no ...): the declared set is the valid set, and (if we add exhaustiveness checking) switches can be exhaustive without a default.
extendable/non-exhaustive union (with ...): the declared set is valid today, but the compiler would force a default in switches so future additions don’t introduce backward compatibility breakage.
That feels like it directly addresses the cloud-enum case you’re highlighting and the closed-enum case some of us care about, without conflating the two?
Re ratio: in my currently active projects I previously mentioned I scanned ~75 enum-ish types and ~20–25% felt conceptually closed. That’s only one data point, but it suggests “closed” isn’t vanishingly rare.
Is your hesitation mainly about how often this comes up, or are there deeper design/complexity concerns? If it’s mostly the former, what level would make “closed” worth addressing?
If it would be useful, I could write a small AST scanner to extract “enum-ish” patterns (typed const blocks) to help quantify this across larger codebases. The output would still need human classification as closed vs extendable, which is obviously subjective, but anyone doing it could publish their results for others to review/critique.
If you're dealing with anything external to the process you have to handle unknown cases. That's not a ding against exhaustive checking. It's a ding against incorrectly modeling. Even with exhaustive sums you are totally allowed to add an UnknownMessage and then be forced to handle the case of not knowing how to handle the case!
I'm not even for exhaustive checking. I just think it's a flawed argument.
Even if most sums are open and lack of exhaustive checking is deemed a virtue for the compiler, it's still possible to add an exhaustiveness linter that can use comment directives to opt sums in (or out). It's easy to do that because the sum lists all its cases so it's very simple to analyze externally. That even lets you compile and run tests when you're still in the process of updating your switches.
Is your hesitation mainly about how often this comes up, or are there deeper design/complexity concerns? If it’s mostly the former, what level would make “closed” worth addressing?
@mikeschinkel How often it comes up. 20% is not enough to justify a new language feature, in my opinion as a Go programmer. (If we were talking about a different language, my answer might change.)
I can't give you a numerical breakpoint. It would depend on several factors, like how complex the new feature was and how serious the bugs from non-exclusivity are. To my knowledge, no such bugs have caused major outages like the Cloudflare DNS outage that resulted in adding monotonic time.
I'd love to make use of unions for typescript-go, porting the TS compiler to Go, and for our LSP implementation.
TypeScript unions are basically always closed. Anything that "opens" the union causes the type to reduce to something more general ("a" | "b" | string is just string). This applies to discriminated unions of objects as well.
Not having closed unions is a pretty severe limitation; a large number of porting bugs we have are forgetting to handle an AST node type that we would have caught had we been able to check for completeness.
For the LSP types, we generate structs that contain fields for every possible type, which is tedious but at least is better than any (and allows for unmarshal methods).
20% is not enough to justify a new language feature, in my opinion as a Go programmer.
By that argument, we should never have added int16 to the language, since int, int32, and int64 handle well over 80% of all signed-integer cases.
My personal feeling is that if we add sum types to Go (and I'm largely agnostic on whether we should), we should aim to support the same sorts of features provided by these types in other popular languages, or have a good reason why a feature isn't appropriate for Go. Failing to do so will result in all the costs of a new language feature while still leaving the people asking for, for example, Rust-style enums unsatisfied.
On open vs. closed enums: A point which I don't think has been emphasized enough is that not all enums are public.
Adding a new case to a closed enum is a backwards-incompatible change, but if the enum is not public the only code broken is code which can be fixed as part of the addition.
If I'm writing a fantasy game, I might have an enum of character classes: fighter, wizard, etc. I probably want this to be a closed enum, so that when I add new characters I get compiler errors pointing at the code that needs updating. Backwards-incompatibility is not a concern if this is all contained within a single program.
One alternative that avoids new language features entirely: make exhaustiveness an opt-in go vet check.
E.g. mark enum-ish types as closed via a directive/comment (e.g. //go:exhaustive, bikeshed as needed). A vet analyzer would then:
Unmarked types remain “open by default”; optionally a complementary marker could require a default for explicitly open enums.
This supports closed/internal enums (add a constant → vet points at all affected switches) without committing the language to open/closed sum types, and fits Go’s existing tool-first, opt-in model.
One alternative that avoids new language features entirely: make exhaustiveness an opt-in go vet check.
It's unfortunate that everytime there is a request for something that the compiler should do, an external tool is proposed as a solution, be it a vet check or code generation etc. these are tasks that are done by the compiler in every other language. Keeping the compiler/language "simple" by externalising it's work it's a fake simplicity.
I was thinking about Maybe[T] again this morning so I sketched it. This is roughly what I'd want from such a type in Go: https://go.dev/play/p/qky2qkFD3vB
This is certainly a case where exhaustiveness checking would be useful. The linked implementation may panic at runtime if misused, but that could itself be linted, I believe.
The real missing thing is that you'd want serialization libs to handle it similar to *T. For that to happen, they would need to know to handle it as a special case, which would require it to be in std regardless of implementation.
20% is not enough to justify a new language feature, in my opinion as a Go programmer.
By that argument, we should never have added
int16to the language, sinceint,int32, andint64handle well over 80% of all signed-integer cases.
int16 is a tiny addition to the language, and it would have been weird to leave it out while having int8 and int32.
You're comparing a whole new language feature with this proposal, which, like int16, is a de-weirding (orthogonalization) of the language.
@mwriter brought up an interesting example in [#65555](https://github.com/golang/go/issues/65555). I will try to reword the point for this issue.
One question that came up in this issue is how to handle methods on sum types. We can not declare methods on interfaces, so you can not do e.g.
type Circle struct { Radius float64 }
type Square struct { Side float64 }
type Shape interface{ Circle | Square }
func (s Shape) Area() float64 { // error: receiver is interface type
switch s := s.(type) {
case Circle:
return s.Radius*s.Radius*math.Pi
case Square:
return s.Side*s.Side
}
}
One way around that would be, to define the methods on the concrete types instead:
func (c Circle) Area() float64 { return c.Radius*c.Radius*math.Pi }
func (s Square) Area() float64 { return s.Radius*s.Radius }
That still doesn't allow us to call Area() on a Shape, though, because of #51183 (i.e. it is not possible to call a method on a type parameter, even if every type in the type set has it). We can work around that, by adding the method to Shape explicitly:
type Shape interface {
Circle | Square
Area() float64 // just to help the compiler
}
func SumOfAreas(seq iter.Seq[Shape]) float64 {
var total float64
for s := range seq {
total += s.Area()
}
return total
}
However - and that's the new argument - this doesn't work with generic methods even if #77273 is implemented. That's because we do not allow to add generic methods to interfaces.
There is no implementation reason we couldn't allow it in this specific case, because the set of types implementing Shape is definitionally closed. So the compiler could implicitly do
type Shape interface {
Circle | Square
GenericMethod[T any](v T)
}
func (Circle) GenericMethod[T any](v T) { /* … */ }
func (Square) GenericMethod[T any](v T) { /* … */ }
// implicitly generated by the compiler:
func _GenericFunction[T any](s Shape, v T) {
switch s := s.(type) {
case Circle:
s.GenericMethod[T](v)
case Square:
s.GenericMethod[T](v)
default:
// can never happen, because Shape is closed - that's why this doesn't
// work with generic methods in interfaces in general
}
}
and then dispatch s.GenericMethod(v) to _GenericFunction(s, v). But allowing it would require us to carve out further exceptions in the spec.
So, in practice, this design would probably require the user to manually define the _GenericFunction themselves and the caller to use the function- instead of the method-call syntax. So this can be worked around, but it is a bit awkward.
That's, to me, a further downside of this design, compared to e.g. #76920. Because that design makes the sum types their own kind of concrete type, which naturally allows to define methods on them - which naturally extends to those methods being generic.
One thing I'm not sure has been suggested (which I'm not going to verify because this issue has like 300 comments), is to provide the ability to explicitly define and rename the nil case. This would allow sums that have nullary/unitary variants to take advantage of the nil case, instead of treating them separately. For example
type Option[T any] interface {
Some[T] | None = nil
}
type Some[T any] T
x := Some(42)
switch x := x.(type) {
case Some[int]:
fmt.Println(x+1)
case None:
}
Here, you get exhaustive checking and the nil variant gets folded into the existing ones, which is spelled out inside the union syntax. And since nil is the zero value for interfaces, the zero value for Option[T] would be None == nil.
@smasher164 That doesn't seem to have any advantages over just using nil. That is (ignoring some other issues that make your code not compile right now), this seems fully equivalent:
type Option[T any] interface {
Some[T]
}
type Some[T any] T
x := Some(42)
switch x := x.(type) {
case Some[int]:
fmt.Println(x+1)
case nil:
}