From 62862a65a458d1e20647f51120418b9ab1469c78 Mon Sep 17 00:00:00 2001 From: Simon Ochsenreither Date: Tue, 15 Nov 2011 16:43:42 +0100 Subject: [PATCH] Added Ivan Towlson's C# articles with his kind permission. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - I left out some Scala-on-.NET specific parts, which should probably be reviewed and put into a diferent article. - I did some minor changes to the wording, I'm a bit reluctant to do more extensive changes because Ivan's personal writing style make it such a joy to read and English isn't my mother tongue. - I replaced the notion of “Pimp My Library” with the more neutral “Enriching ...” as discussed on the mailing list. I'm open to better proposals. - I replaced “operators” with “methods” if it revolved around the name and with “methods in infix notation” if it revolved around the calling syntax. Both as mentioned in the style guide. Some thougths: - The article was written before C# and Scala got optional/default parameters, we probably need to adjust this section - C# also got Tuples ... not sure if or how much we need to cover it now. - We could probably need some introduction to syntax differences at the beginning. --- tutorials/scala-for-csharp-programmers.md | 1037 +++++++++++++++++++++ 1 file changed, 1037 insertions(+) create mode 100644 tutorials/scala-for-csharp-programmers.md diff --git a/tutorials/scala-for-csharp-programmers.md b/tutorials/scala-for-csharp-programmers.md new file mode 100644 index 0000000000..1f316d400d --- /dev/null +++ b/tutorials/scala-for-csharp-programmers.md @@ -0,0 +1,1037 @@ +--- +layout: overview +title: A Scala Tutorial for C# Programmers + +disqus: true +--- + +By Ivan Towlson + +## Introduction + +Scala is a hybrid of functional and object-oriented languages. +Its functional aspects make it very expressive when writing algorithmic +code, and play nicely with the brave new world of concurrency; its +object-oriented aspects keep it familiar and convenient when creating +business objects or other stateful models. + +Scala's primary platform is the Java virtual machine, and some of the +interest in Scala comes from Java programmers' interest in features such +as type inference, comprehensions and lambdas, with which C# programmers +are already familiar. So what's left that might be of interest +specifically to C# programmers? + +## Mixins and Traits + +### Motivation + +Interfaces in C# and Java play a dual role. +First, they are a set of capabilities that an implementer has to, well, +implement. Second, they are a feature set that a client can use. + +These two roles can be conflicting: The first means that interfaces want +to be minimal, so that implementers don't have to implement a whole lot +of superfluous and redundant guff. The second means that interfaces want +to be maximal, so that clients don't have to clog themselves up with +boilerplate utility methods. + +Consider, for example, `IEnumerable` (and its sister interface `IEnumerator`). +This is a very minimal interface: implementers just need to be able to +produce values in sequence. But this minimalism means that clients of +`IEnumerable` need to write the same old boilerplate again and again and +again: foreach loops to filter, foreach loops to call a method on each +element of the sequence, foreach loops to aggregate, foreach loops to +check whether all elements meet a criterion, or to find the first member +that meets a criterion, or... + +This is frustrating because the implementations of "filter," "apply", +"aggregate," and so on are always the same. Of course, we could put +these methods into concrete types (`List` includes several), but then +those concrete types will contain duplicate code, and users who only have +an `IEnumerable` will still miss out. And yet we can't put these methods +into the interface because then every implementer of `IEnumerable` would +have to implement them -- and they'd end up writing the same boilerplate, +just now in all the zillions of `IEnumerable` classes instead of their clients. + +### The C# and Scala Solutions + +We could resolve this tension if we had a way for interfaces to contain +implementation: for example, if `IEnumerable` required the implementer +to provide the class-specific iteration functionality, but then provided +the standard implementations of "filter," "apply", "aggregate" and so on +automatically: + + public pseudo_interface IEnumerable + { + IEnumerator GetEnumerator(); // must be implemented + IEnumerable Filter(Predicate predicate) // comes for free + { + foreach (object o in this) + if (predicate(o)) + yield return o; + } + } + +C# 3 addresses this using extension methods: the methods mentioned above +are all in fact included as extension methods on `IEnumerable` as +part of LINQ. + +This has some advantages over the approach described above: specifically, +the "standard methods" aren't bound up in the interface, so you can add +your own methods instead of being limited to the ones that the interface +author has included. + +On the other hand, it means that method implementations have to be packaged +in a different class from the interface, which feels less than modular. + +Scala takes a different approach. A Scala trait can contain a mix of +abstract methods without implementation as well as concrete methods with +an implementation. (It can also be a pure interface, with only abstract +members.) + +Here's a Scala trait that represents objects that can be compared +and ordered: + + trait Ord { + def < (that: Any): Boolean + def <=(that: Any): Boolean = (this < that) || (this == that) + def > (that: Any): Boolean = !(this <= that) + def >=(that: Any): Boolean = !(this < that) + } + +Orderable objects need to extend `Ord`, but only need to implement the +method `<`. They then get the other operators for free, implemented +automatically by Ord in terms of `<`. + + class Date extends Ord { + def < (that: Any): Boolean = /* implementation */ + } + + // can now write: myDate >= yourDate + +### Scala Traits vs. C# Extension Methods + +Okay, so Scala has a different way of packaging standard implementations +from C#'s extension methods. It's different, but why is it interesting? +Well, there are a couple of things that you can do with Scala traits that +don't fall nicely out of the extension methods approach. + +First, you can override the default implementations of trait members, +to take advantage of additional information or capabilities available +in the implementing type. + +Let's look at another `IEnumerable` example, recast as a Scala trait: + + trait Enumerable { + def getEnumerator(): Enumerator + def count: Int = { + // Shockingly bad style; for illustrative purposes only + var c = 0 + val e = getEnumerator() + while (e.moveNext()) c = c + 1 + c + } + } + +This (ignoring style issues for now) is the only fully general +implementation we can provide for count: it will work with any `Enumerable`. +But for collections that know their sizes, such as `arrays` or `List`, +it's gruesomely inefficient. It iterates over the entire collection, +counting elements one by one, when it could just consult the `size` member +and return that. + +Let's fix that: + + class MyList extends Enumerable { + private var _size; + def getEnumerator(): Enumerator = /* ... */ + override def count: Int = _size + } + +The `count` member of the `Enumerable` trait works like a virtual method: +it can be overridden in classes which implement/derive from `Enumerable`. + +Compare this to the `Count()` extension method on `IEnumerable` in LINQ. +This achieves the same effect by trying to cast to `ICollection`, which is +fine as far as it goes but isn't extensible. + +Suppose you create an enumerable class that can count itself quickly but +isn't a collection -- for example a natural numbers range object. +With a Scala trait, the `NumberRange` type could provide an efficient +override of `count`, just like any other virtual method; with C# extension +methods, `Enumerable.Count()` would have to somehow know about the +`NumberRange` type in advance, or fall back on counting elements one by one. + +Second, with Scala you can choose a trait implementation when you +instantiate an object, rather than having it baked in at the class level +once and for all. This is called mixin-composition. + +Suppose you're creating a `MyList` instance, but you want it to puff itself +up to look bigger so as to frighten other `MyList` instances off its territory. +(This example would probably work better with fish, but we're stuck with +`Enumerable`s now. Work with me here.) In C#, you'd need to create a +`PuffedUpMyList` class and override the `Count` property. +In Scala, you can just mix in a `PuffedUp` version of the trait: + + trait PuffedUp extends Enumerable { + override def count: Int = super.count + 100 + } + + val normal = new MyList + Console.WriteLine(normal.count) // meh + val puffedUp = new MyList with PuffedUp + Console.WriteLine(puffedUp.count) // eek! + +As you can imagine this gives us much better granularity and composability +of traits and implementations than we get from the extension methods +approach, or indeed from single implementation inheritance type systems +in general. + +So Scala traits have some distinct advantages over extension methods. +The only downside appears to be the inability for clients to add their +own methods to a trait after the fact. + +Fortunately, you can work around this in Scala using so-called implicit +conversions. They enable Scala programmers to enrich existing types with new +functionality. + +## Singletons + +In C#, if you want to create a singleton object, you have to create a class, +then stop evildoers creating their own instances of that class, then create +and provide an instance of that class yourself. + +While this is hardly a Burma Railway of the programming craft, it does +feel like pushing against the grain of the language. Nor is it great for +maintainers, who have to be able to recognise a singleton by its pattern +(private constructor, public static readonly field, ...), or for clients, +who have to use a slightly clumsy multipart syntax to refer to the +singleton (e.g. `Universe.Instance`). + +What would be easier for all concerned would be if you could just declare +objects *as* singletons. That is, instead of writing class `Universe` and a +`public static readonly` instance of it, you could just write `object Universe`. + +And that's exactly what Scala allows you to do: + + object Universe { + def contains(obj: Any): Boolean = true + } + + val v = Universe.contains(42) + +What's going on behind the scenes here? It pretty much goes without saying +that the Scala compiler is creating a new type for the singleton object. +In fact it creates two types, one for the implementation and one for the +interface. The interface looks like a .NET static class (actually, the +.NET 1.x equivalent, a sealed class with only static members). +Thus, a C# program would call the example above as `Universe.contains(42)`. + +Singleton objects are first-class citizens in Scala, so they can for +example derive from classes. This is a nice way of creating special values +with custom behaviour: you don't need to create a whole new type, you just +define an instance and override methods in it: + + abstract class Cat { + def humiliateSelf() + } + + object Slats extends Cat { + def humiliateSelf() { savage(this.tail) } + } + +Obviously this is a frivolous example, but "special singletons" turn out to +be an important part of the functional idiom, for example for bottoming out +recursion. *Scala by Example (PDF)* describes an implementation of a Set class +which is implemented as a tree-like structure ("left subset - member - right +subset"), and methods such as `contains()` work by recursing down to the +child sets. For this to work requires an `EmptySet` whose implementation +(state) and behaviour are quite different from non-empty sets -- e.g. +`contains()` just returns `false` instead of trying to delegate to +non-existent child sets. Since `EmptySet` is logically unique it is both +simpler and more efficient to represent it as a singleton: i.e. to declare +`object EmptySet` instead of `class EmptySet`. + +In fact the whole thing can become alarmingly deep: *Scala by Example* +also includes a description of `Boolean` as an `abstract class`, and +`True` and `False` as singleton objects which extend `Boolean` and provide +appropriate implementations of the `ifThenElse` method. + +And fans of Giuseppe Peano should definitely check out the hypothetical +implementation of `Int`... + +## Pass by Name + +> You're only on chapter 3 and you're already reduced to writing about +> *calling conventions*? You suck! Do another post about chimney sweeps +> being hunted by jars of marmalade!" + +Silence, cur. Pass by name is not as other calling conventions are. +Pass by name, especially in conjunction with some other rather +theoretical-sounding Scala features, is your gateway to the wonderful +world of language extensibility. + +### What is Passing By Name? + +First, let's talk about what we mean by *calling convention*. A calling +convention describes how stuff gets passed to a method by its caller. +In the good old days, this used to mean exciting things like which +arguments got passed in registers and who was responsible for resetting +the stack pointer. Sadly, the days of being able to refer to "naked fun +calls" are consigned to history: In modern managed environments, the +runtime takes care of all this guff and the main distinction is pass +data by value or by reference. (The situation on the CLR is slightly +complicated by the need to differentiate passing values by value, values +by reference, references by value and references by reference, but I'm +not going to go into that because (a) it's irrelevant to the subject at +hand and (b) that's +[Jon Skeet](http://www.yoda.arachsys.com/csharp/parameters.html)'s turf +and I don't want him to shank me. Again.) + +In *pass by value*, the called method gets a copy of whatever the caller +passed in. Arguments passed by value therefore work like local variables +that are initialised before the method runs: when you do anything to them, +you're doing it to your own copy. + +In *pass by reference*, the called method gets a reference to the caller's +value. When you do anything to a pass-by-reference argument, you're doing +it to the caller's data. + +In *pass by name*, the called method gets... well, it's a bit messy to +explain what the called method gets. But when the called method does +anything to the argument, the argument gets evaluated and the "anything" +is done to that. Crucially, evaluation happens every time the argument +gets mentioned, and only when the argument gets mentioned. + +### Not Just Another Calling Convention + +Why does this matter? It matters because there are functions you can't +implement using pass by value or pass by reference, but you can implement +using pass by name. + +Suppose, for example, that C# didn't have the `while` keyword. +You'd probably want to write a method that did the job instead: + + public static void While(bool condition, Action body) + { + if (condition) + { + body(); + While(condition, body); + } + } + +What happens when we call this? + + long x = 0; + While(x < 10, () => x = x + 1); + +C# evaluates the arguments to `While` and invokes the `While` method with +the arguments `true` and `() => x = x + 1`. After watching the CPU sit +on 100% for a while you might check on the value of `x` and find it's +somewhere north of a trillion. *Why?* Because the condition argument was +*passed by value*, so whenever the `While` method tests the value of +condition, it's always `true`. The `While` method doesn't know that +condition originally came from the expression `x < 10`; all `While` knows +is that condition is `true`. + +For the `While` method to work, we need it to re-evaluate `x < 10` every +time it hits the condition argument. While needs not the value of the +argument at the call site, nor a reference to the argument at the call +site, but the actual expression that the caller wants it to use to generate +a value. + +Same goes for short-circuit evaluation. If you want short-circuit +evaluation in C#, your only hope if to get on the blower to Anders +Hejlsberg and persuade him to bake it into the language: + + bool result = (a > 0 && Math.Sqrt(a) < 10); + double result = (a < 0 ? Math.Sqrt(-a) : Math.Sqrt(a)); + +You can't write a function like `&&` or `?:` yourself, because C# will +always try to evaluate all the arguments before calling your function. + +Consider a VB exile who wants to reproduce his favourite keywords in C#: + + bool AndAlso(bool condition1, bool condition2) + { + return condition1 && condition2; + } + + T IIf(bool condition, T ifTrue, T ifFalse) + { + if (condition) + return ifTrue; + else + return ifFalse; + } + +But when C# hits one of these: + + bool result = AndAlso(a > 0, Math.Sqrt(a) < 10); + double result = IIf(a < 0, Math.Sqrt(-a), Math.Sqrt(a)); + +it would try to evaluate all the arguments at the call site, and pass the +results of those evaluations to `AndAlso` or `IIf`. There's no +short-circuiting. So the `AndAlso` call would crash if a were negative, +and the `IIf` call if a were anything other than 0. Again, what you want is +for the `condition1`, `condition2`, `ifTrue` and `ifFalse` arguments to be +evaluated by the callee if it needs them, not for the caller to evaluate +them before making the call. + +And that's what *pass by name* does. A parameter passed by name is not +evaluated when it is passed to a method. It is evaluated -- and +re-evaluated -- when the called method evaluates the parameter; +specifically when the called method requests the value of the parameter by +mentioning its name. This might sound weird and academic, but it's the key +to being able to define your own control constructs. + +### Using Pass By Name in Scala + +Let's see the custom while implementation again, this time with Scala +*pass by name* parameters: + + def myWhile(condition: => Boolean)(body: => Unit): Unit = + if (condition) { + body + myWhile(condition)(body) + } + +We can call this as follows: + + var i = 0 + myWhile (i < 10) { + println(i) + i += 1 + } + +Unlike the C# attempt, this prints out the numbers from 0 to 9 and then +terminates as you'd wish. + +Pass by name also works for short-circuiting: + + import math._ + + def andAlso(condition1: => Boolean, condition2: => Boolean): Boolean = + condition1 && condition2 + + val d = -1.234 + val result = andAlso(d > 0, sqrt(d) < 10) + +The `andAlso` call returns `false` rather than crashing, because +`sqrt(d) < 10` never gets evaluated. + +What's going on here? What's the weird colon-and-pointy-sticks syntax? +What is actually getting passed to `myWhile` and `andAlso` to make this work? + +The answer is a bit surprising. Nothing is going on here. This is the +normal Scala function parameter syntax. There is no *pass by name* in Scala. + +Here's a bog-standard *pass by value* Scala function declaration: + + def myFunc1(i: Int): Unit = ... + +Takes an integer, returns void: easy enough. Here's another: + + def myFunc2(f: Int => Boolean): Unit = ... + +Even if you've not seen this kind of expression before, it's probably not +too hard to guess what this means. This function takes a *function from +`Int` to `Boolean`* as its argument. In C# terms, +`void MyFunc2(Func f)`. We could call this as follows: + + myFunc2 { (i: Int) => i > 0 } + +So now you can guess what this means: + + def myFunc3(f: => Boolean) : Unit = ... + +Well, if `myFunc2` took an *Int-to-Boolean* function, `myFunc3` must be +taking a "blank-to-Boolean" function -- a function that takes no arguments +and returns a `Boolean`. In short, a conditional expression. So we can +call `myFunc3` as follows: + + val j = 123 + myFunc3 { j > 0 } + +The squirly brackets are what we'd expect from an anonymous function, and +because the function has no arguments Scala doesn't make us write +`{ () => j > 0 }`, even though that's what it means really. The anonymous +function has no arguments because `j` is a captured local variable, not an +argument to the function. But there's more. Scala also lets us call +`myFunc3` like this: + + val j = 123 + myFunc3(j > 0) + +This is normal function call syntax, but the Scala compiler realises that +`myFunc3` expects a nullary function (a function with no arguments) rather +than a `Boolean`, and therefore treats `myFunc3(j > 0)` as shorthand for +`myFunc3(() => j > 0)`. This is the same kind of logic that the C# compiler +uses when it decides whether to compile a lambda expression to a delegate +or an expression tree. + +You can probably figure out where it goes from here: + + def myFunc4(f1: => Boolean)(f2: => Unit): Unit = ... + +This takes two functions: a conditional expression, and a function that +takes no arguments and returns no value (in .NET terms, an `Action`). +Using our powers of anticipation, we can imagine how this might be called +using some unholy combination of the two syntaxes we saw for calling +`myFunc3`: + + val j = 123; + myFunc4(j > 0) { println(j); j -= 1; } + +We can mix and match the `()` and `{}` bracketing at whim, except that we +have to use `{}` bracketing if we want to batch up multiple expressions. +For example, you could legally equally well write the following: + + myFunc4 { j > 0 } { println(j); j -= 1; } + myFunc4 { println(j); j > 0 } (j -= 1) + myFunc4 { println(j); j > 0 } { j -= 1 } + +And we'll bow to the inevitable by supplying a body for this function: + + def myFunc5(f1: => Boolean)(f2: => Unit): Unit = + if (f1()) { + f2() + myFunc5(f1)(f2) + } + +Written like this, it's clear that `f1` is getting evaluated each time we +execute the if statement, but is getting passed (as a function) when +`myFunc5` recurses. But Scala allows us to leave the parentheses off +function calls with no arguments, so we can write the above as: + + def myFunc5(f1: => Boolean)(f2: => Unit): Unit = + if (f1) { + f2 + myFunc5(f1)(f2) + } + +Again, type inference allows Scala to distinguish the *evaluation of +`f1`* in the if statement from the *passing of `f1`* in the `myFunc5` +recursion. + +And with a bit of renaming, that's `myWhile`. There's no separate +*pass by name* convention: just the usual closure behaviour of capturing +local variables in an anonymous method or lambda, a bit of syntactic sugar +for nullary functions (functions with no arguments), just like C#'s +syntactic sugar for property getters, and the Scala compiler's ability to +recognise when a closure is required instead of a value. + +In fact, armed with this understanding of the Scala "syntax," we can +easily map it back to C#: + + void While(Func condition, Action body) + { + if (condition()) + { + body(); + While(condition, body); + } + } + + int i = 0; + While(() => i < 10, () => + { + Console.WriteLine(i); + ++i; + }); + +The implementation of the `While` method in C# is, to my eyes, a bit +clearer than the Scala version. However, the syntax for *calling* the +`While` method in C# is clearly way more complicated and less natural than +the syntax for calling `myWhile` in Scala. Calling `myWhile` in Scala was +like using a native language construct. Calling While in C# required a +great deal of clutter at the call site to prevent C# from trying to treat +`i < 10` as a once-and-for-all value, and to express the body at all. + +So that's so-called "pass by name" demystified: The Scala Web site, with +crushing mundanity, demotes it to "automatic type-dependent closure +construction," which is indeed exactly how it works. As we've seen, +however, this technical-sounding feature is actually essential to +creating nice syntax for your own control constructs. We'll shortly see +how this works together with other Scala features to give you even more +flexibility in defining your construct's syntax. + +## Multiple Return Values (Tuples) + +Everybody hates out parameters. We all know that code like this just +isn't nice: + + Widget widget; + if (widgetDictionary.TryGetValue(widgetKey, out widget)) + { + widget.ReticulateSplines(); + } + +And once you start composing higher-level functions into the mix, it gets +positively nasty. Suppose I want to make a HTTP request. Well, that's +going to produce two outputs in itself, the success code and the response +data. But now suppose I want to time it. Writing the timing code isn't a +problem, but now I have *three* outputs, and to paraphrase *Was Not Was*, +I feel worse than Jamie Zawinski. + +You can get around this in specific situations by creating custom types +like `DictionaryLookupResult` or `TimedHttpRequestResult`, but eventually +the terrible allure wears off, and you just want it to work. + +Enter tuples. A tuple is just a small number of values -- a single value, +a pair of values, a triplet of values, that sort of thing. You can tell +that tuples were named by mathematicians because the name only makes sense +when you look at the cases you hardly ever use (quadruple, quintuple, +sextuple, etc.). If C# had tuples, our timed HTTP request might look a bit +better: + + public Tuple Time(Func> action) + { + StartTimer(); + var result = action(); + TimeSpan howLong = StopTimer(); + return new Tuple(result.First, result.Second, howLong); + } + + var result = Time(() => MakeRequest(uri)); + var succeeded = result.First; + var response = result.Second; + var howLong = result.Third. + Console.WriteLine("it took " + howLong); + +What's that you say? Not better? + +The reason this keeps getting verbose on us is that C# doesn’t provide any +syntatical support for tuples. To C#, a `Tuple<>` is just another generic +type. To us, the readers, a `Tuple<>` is just another generic type with +particularly unhelpful member names. + +Really, what we're really trying to articulate by returning a `Tuple<>` is, +"this method has several outputs." So what do we want to do with those +outputs? We want to access them, for example to stash them in variables, +without having to construct and pick apart the tuple one value at a time. +That means the language has to provide some kind of syntax-level support +for tuples, instead of treating them like every other class the compiler +doesn’t know about. + +Many functional languages have exactly this kind of syntactical support, +and Scala is no exception. Here’s how the above pseudo-C# looks in Scala: + + def time(action: => (Boolean, Stream)): (Boolean, Stream, TimeSpan) = { + startTimer() + val (succeeded, response) = action + (succeeded, response, stopTimer()) + } + + val (succeeded, response, timeTaken) = time(makeRequest) + Console.WriteLine("it took " + timeTaken) + +Notice the multiple variables on the left hand side of the time call? +Notice the `(Boolean, Stream, TimeSpan)` return type of the time method? +That return value is effectively a tuple type, but instead of having to +capture the returned tuple in a `Tuple<>` variable and extract its various +bits by hand, we are getting the Scala compiler to (in the time function) +compose the tuple implicitly for us, without us having to write the +constructor call, and (in the calling code) unpick the tuple into +meaningful, named variables for us without us having to explicitly copy +the values one by one and by name. + +(By the way, a proper implementation of the `time()` method wouldn’t be +restricted to `(Boolean, Stream)` results: we’d be looking to write a +method that could time anything. I’ve skipped that because it would +distract from the point at hand.) + +How would this play with the dictionary example? + + val (found, widget) = widgetDictionary.GetValue(key) + if (found) { + widget.ReticulateSplines() + } + +We don’t actually save any lines of code, what with having to now capture +the “found” value into a variable and test it separately; and it’s not as +if the original C# version was horribly unreadable anyway. So maybe this is +a matter of taste, but I find this a lot easier to read and to write: all +the outputs are on the left of the equals sign where they belong, instead +of being spread between the assignment result and the parameter list, and +we don’t have that odd Widget declaration at the top. + +Encouragingly enough, .NET 4.0 will include a `Tuple` class, I believe +precisely in order to support functional and other languages that use the +multiple return value idiom. Unfortunately, C# 4.0 still isn’t getting +syntactical support for them, so we’re stuck with out parameters or +special result types for now. + +## Implicits + +Scala implicits offer some features which will be familiar to the C# +programmer, but are much more general in nature and go far beyond what can +be done in C#. + +### Enriching types in C# and Scala + +Scala, like C#, is statically typed: a class’ methods are compiled into the +class definition and are not open for renegotiation. You cannot, as you +might in Ruby or Python, just go ahead and declare additional methods on an +existing class. + +This is of course very inconvenient. You end up declaring a load of +`FooHelper` or `FooUtils` classes full of static methods, and having to +write verbose calling code such as `if (EnumerableUtils.IsEmpty(sequence))` +rather than the rather more readable `if (sequence.IsEmpty())`. + +C# 3 tries to address this problem by introducing extension methods. +Extension methods are static methods in a `FooHelper` or `FooUtils` kind +of class, except you’re allowed to write them using member syntax. +By defining `IsEmpty` as an extension method on `IEnumerable`, you can +write `if (sequence.IsEmpty())` after all. + +Scala disapproves of static classes and global methods, so it plumps for +an alternative approach. You’ll still write a `FooHelper` or `FooUtils` +kind of class, but instead of taking the `Foo` to be Helped or Utilised as +a method parameter, your class will wrap `Foo` and enrich it with +additional methods. Let’s see this in action as we try to add a method to +the `Double` type: + +class RicherDouble(d : Double) { + def toThe(exp: Double): Double = System.Math.Pow(d, exp) +} + +(We call the class `RicherDouble` because Scala already has a `RichDouble` +class defined which providing further methods to `Double`.) + +Notice that `toThe` is an instance method, and that `RicherDouble` takes a +`Double` as a constructor parameter. This seems pretty grim, because we’d +normally have to access the function like this: + + val result = new DoubleExtensions(2.0).toThe(7.0) + +Hardly readable. To make it look nice, Scala requires us to define an +*implicit conversion* from `Double` to `RicherDouble`: + + object Implicits { + implicit def richerDouble(d: Double) = new RicherDouble(d) + } + +and to bring that implicit conversion into scope: + + import Implicits._ + +Now we can write this: + + val twoToTheSeven = 2.0.toThe(7.0) + +and all will be well. The `Double` type has apparently been successfully +enriched with the `toThe` method. + +This is, of course, just as much an illusion as the C# equivalent. +C# extension methods don’t add methods to a type, and nor do Scala +implicit conversions. What has happened here is that the Scala compiler +has looked around for implicit methods that are applicable to the type of +`2.0` (namely `Double`), and return a type that has a `toThe` method. +Our `Implicits.richerDouble` method fits the bill, so the Scala compiler +silently inserts a call to that method. At runtime, therefore, Scala calls +`Implicits.richerDouble(2.0)` and calls the `toThe` of the resulting +`RicherDouble`. + +If setting this up seems a bit verbose, well, maybe. C# extension methods +are designed to be easily – one might even say implicitly – brought into +scope. That’s very important for operators like the LINQ standard query +operators, but it can result in unwanted extension methods being dragged +into scope and causing havoc. Scala requires the caller to be a bit more +explicit about implicits, which results in a slightly higher setup cost but +gives the caller finer control over which implicit methods are considered. + +But as it happens you can avoid the need for separate definitions of +`Implicits` and `RicherDouble`, and get back to a more concise +representation by using an anonymous class. (As you’d expect, Scala +anonymous classes are fully capable, like Java ones, rather than the +neutered C# version.) Here’s how it looks: + + object Implicits { + implicit def doubleToThe(d1 : Double) = new { + def toThe(d2 : Double) : Double = Math.Pow(d1, d2) + } + } + +Well, big deal. Scala can pimp existing types with new methods just like +C#, but using a different syntax. In related news, Lisp uses a different +kind of bracket: film at eleven. Why should we be interested in Scala +implicits if they’re just another take on extension methods? + +### Implicit Parameters + +What we saw above was an implicit method – a method which, like a C# +implicit conversion operator, the compiler is allowed to insert a call to +without the programmer writing that call. Scala also has the idea of +implicit parameters – that is, parameters which the compiler is allowed to +insert a value for without the programmer specifying that value. + +That’s just optional parameters with default values, right? Like C++ and +Visual Basic have had since “visual” meant ASCII art on a teletype, and +like C# is about to get? Well, no. + +C++, Visual Basic and C# optional parameters have fixed defaults specified +by the called function. For example, if you have a method like this: + + public void Fie(int a, int b = 123) { … } + +and you call `Fie(456)`, it’s always going to be equivalent to calling +`Fie(456, 123)`. + +A Scala implicit parameter, on the other hand, gets its value from the +calling context. That allows programmer calling the method to control the +implicit parameter value, creating an extensibility point that optional +parameters don’t provide. + +This probably all sounds a bit weird, so let’s look at an example. Consider +the following `Concatenate` method: + + public T Concatenate(IEnumerable sequence, T seed, Func concatenator); + +We pass this guy a sequence, a start value and a function that combines two +values into one, and it returns the result of calling that function across +the sequence. For example, you could pass a sequence of strings, a start +value of `String.Empty`, and `(s1, s2) => s1 + s2`, and it would return you +all the strings concatenated together: + + IEnumerable sequence = new string[] { “mog”, “bites”, “man” }; + string result = Concatenate(sequence, String.Empty, (s1, s2) => s1 + s2); + // result is “mogbitesman” + +But this is a unpleasantly verbose. We’re having to pass in `String.Empty` +and `(s1, s2) => s1 + s2` every time we want to concatenate a sequence of +strings. Not only is this tedious, it also creates the opportunity for +error when the boss decides to “help” and passes the literal +`"String.Empty"` as the seed value instead. (“Oh, and I upgraded all the +semi-colons to colons while I was in there. No, don’t thank me!”) We’d +like to just tell the Concatenate function, “Look, this is how you +concatenate strings,” once and for all. + +Let’s start out by redefining the `Concatenate` method in Scala. +I’m going to factor out the seed and the concatenator method into a trait +because we’ll typically be defining them together. + + trait Concatenator[T] { + def startValue: T + def concat(x: T, y: T): T + } + + object implicitParameters { + def concatenate[T](xs: List[T])(c: Concatenator[T]): T = + if (xs.isEmpty) c.startValue + else c.concat(xs.head, concatenate(xs.tail)(c)) + } + +We can call this as follows: + + object stringConcatenator extends Concatenator[String] { + def startValue: String = "" + def concat(x: String, y: String) = x.concat(y) + } + + object implicitParameters { + def main(args: Array[String]) = { + val result = concatenate(List("mog", "bites", "man"))(stringConcatenator) + println(result) + } + } + +So far, this looks like the C# version except for the factoring out of the +`Concatenator` trait. We’re still having to pass in the +`stringConcatenator` at the point of the call. Let’s fix that: + + def concatenate[T](xs: List[T])(implicit c: Concatenator[T]): T = + if (xs.isEmpty) c.startValue + else c.concat(xs.head, concatenate(xs.tail)) + +We’ve changed two things here. First, we’ve declared c to be an *implicit +parameter*, meaning the caller can leave it out. Second, we’ve left +it out ourselves, in the recursive call to `concatenate(xs.tail)`. + +Well, okay, it’s nice that `concatenate` now doesn’t have to pass the +`Concatenator` explicitly to the recursive call, but we’re still having to +pass in the `stringConcatenator` object to get things started. If only +there were some way to make the `stringConcatenator` object itself implicit… + + object Implicits { + implicit object stringConcatenator extends Concatenator[String] { + def startValue: String = "" + def concat(x: String, y: String) = x.concat(y) + } + } + +Again, we’ve done two things here. First, we’ve declared the +`stringConcatenator` object implicit. Consequently, we’ve had to move it +out of the top level, because Scala doesn’t allow implicits at the top +level (because they’d pollute the global namespace, being in scope even +without an explicit import statement). + +Now we can call `concatenate` like this: + + import Implicits._ + + object implicitParameters { + def main(args: Array[String]) = { + val result = concatenate(List("mog", "bites", "man")) + println(result) + } + } + +And we’ll still get “mogbitesman” as the output. + +Let’s review what’s going on here. The implicit parameter of concatenate +has been set to our `stringConcatenator`, a default value that the +`concatenate` method knew nothing about when it was compiled. This is +somewhere north of what classical optional parameters are capable of, +and we’re not finished yet. Let’s build a `listConcatenator`. + + object Implicits { + class ListConcatenator[T] extends Concatenator[List[T]] { + def startValue: List[T] = Nil + def concat(x: List[T], y: List[T]) = x ::: y + } + implicit object stringListConcatenator extends ListConcatenator[String] { } + } + +This is a bit vexing. `List` in Scala is a generic type, and has a generic +concatenation method called `:::`. But we can’t create a generic object, +because an object is an instance. And implicit parameters have to be objects. +So the best we can do is build a generic `ListConcatenator` class, and then +create trivial implicit objects for each generic parameter type we might +need. + +However, let’s not worry about the implementation, and see how this is used +at the calling end: + + val result = concatenate(List( + List("mog", "bites", "man"), + List("on", "beard") + )) + +This displays `List(mog, bites, man, on, beard)`; that is, it concatenates +the two `List[String]`s into one. Once again, we have not had to pass +`stringListConcatenator` explicitly: the Scala compiler has gone and found +it for us. We can use the exact same calling code to concatenate lists and +strings. + +### Why Should I Care? + +Isn’t this pointless? At the call site, I have access to +`stringConcatenator` and `listStringConcatenator`. I can easily pass them +in rather than relying on spooky compiler magic to do it for me. +Aren’t implicit parameters just job security for compiler writers? + +Yes, implicit parameters are technically unnecessary. But if we’re going +to play that game, C# is technically unnecessary. You could write all that +code in IL. Extension methods are unnecessary because you could write the +static method out longhand. Optional parameters are unnecessary because +you could read the documentation and pass them in explicitly. +Post-It notes are unnecessary because you could fire up Outlook and create +a Note instead. + +Implicit parameters are about convenience and expressiveness. Implicit +parameters give you a way of describing how a function should handle +different situations, without needing to bake those situations into the +function logic or to specify them every time you call the function. +You don’t want to have to tell the `concatenate` function whether to use +the `List` or `String` concatenator every time you call it: the compiler +knows what you’re concatenating; specifying how to concatenate it just +gives you a chance to get it wrong! + +Consequently, implicit parameters – like implicit conversions – contribute +to Scala’s ability to support internal DSLs. By setting up appropriate +implicits, you can write code that reads much more naturally than if you +had to pepper it with function objects or callbacks. + +### Conclusion + +Scala’s `implicit` keyword goes beyond C#’s equivalent. As in C#, it is +used for implicit conversions; unlike C#, this is the idiomatic way to add +operations to an existing type, removing the need for the separate +extension method syntax. Implicit parameters have no equivalent in C#. +They are like being able to add default values to a method: just as a C# +using statement bring implicit methods into scope, a Scala import statement +can bring default values into scope. If implicit conversions are a way of +extending classes, then implicit parameters are a way of extending methods, +creating simple, reliable shorthands for complex generic methods, and +making up another piece of the Scala DSL jigsaw. + +### Method Call Syntax + +C#, like most object-oriented programming languages, is pretty strict about +how you call methods: you use the dot notation, unless the method is a +special ‘operator’ method such as `operator+`, `operator==` or a conversion +operator. The special operator methods are predefined by the compiler: you +can write your own implementation, but you can’t create your own operator +names. You can teach the `+` operator how to handle your custom type, but +you can’t add an exponentiation operator: + + int a = b ** c; + +C# has three problems with this: first, it doesn’t like the method name +`**`; second, it doesn’t like that there’s no `.` before the name; and +third, it doesn’t like that there’s no brackets around the method argument. + +To get around the objection to the name, let’s compromise and call it +`ToThe` for now. So what C# insists on seeing is `a.ToThe(b)`. + +Scala, like many functional languages, isn’t so strict. Scala allows you +to use any method with a single argument in an infix position. Before we +can see this in the exponentiation example, we will enrich the `Double` +type with the `toThe` method as we learned earlier: + + import Implicits._ + import math._ + + class RicherDouble(d: Double) { + def toThe(exp: Double): Double = pow(d, exp) + } + + object Implicits { + implicit def richerDouble(d: Double) = new RicherDouble(d) + } + +Recall that this is just the Scala idiom for extension methods – it’s the +equivalent of writing +`public static ToThe(this double first, double second) { … }` in C#. +(If we were wanting to use infix notation with our own class, we wouldn’t +need all this malarkey.) So now we can write: + + val raised = 2.0.toThe(7.0) + +Okay, so what do we need to do to get this to work in infix position? +Nothing, it turns out. + + val raised = 2.0 toThe 8.0 // it just works + +This still doesn’t look much like a built-in operator, but it turns out +Scala is less fussy than C# about method names too. + + class DoubleExtensions(d : Double) { + def **(exp: Double): Double = Pow(d, exp) + } + + val raised = 2.0 ** 9.0 // it still just works + +Much nicer. + +This sorta-kinda works for postfix operators too: + + class RicherString(s: String) { + def twice: String = s + s + } + + val drivel = "bibble" twice + +Calling methods in infix and postfix nodadion is obviously fairly simple +syntactic sugar over normal dot notation. But this seemingly minor feature +is very important in constructing DSLs, allowing Scala to do in internal +DSLs what many languages can do only using external tools. For example, +where most languages do parsing via an external file format and a tool to +translate that file format into native code (a la `lex` and `yacc`), +Scala’s parser library makes extensive use of infix and postfix methods to +provide a “traditional” syntax for describing a parser, but manages it +entirely within the Scala language. +