Compiler technology comparison.

CS-Script allows hosting/executing the scripts being compiled by any of the three supported compilers: CodeDom, Mono and Roslyn. Thanks to generic interface CS-Script offers identical developer experience for all these three compilers. However the compilers themselves are not entirely identical. The choice of the underlying compiler technology my affect runtime behavior and maintainability of the hosting solution.

The initial CS-Script hosting model was based on CodeDom compilation and support for Mono and Roslyn was added much later when these technologies became available. Naturally the CodeDom based API is not 100% aligned with the Mono and Roslyn native API. In fact the native API of the all three compilers are extremely inconsistent. They are so inconsistent that it was quite a challenge to define a common interface that can reconcile all the differences.

This challenge has bee answered and today the main hosting interface can be accessed via CSScript.Evaluator property (e.g. CSScript.Evaluator.LoadCode(..)). It is the entry point for the all three compilers and this is the API that you should rely on in most of the cases. This common interface referred through all the documentation as CSScript.Evaluator.

The original native CodeDom-only interface is still available. And it can be accessed via direct members of the CSScript class (e.g. CSScript.LoadCode(..)). This CodeDom specific interface referred through all the documentation as CSScript.Native. This interface is provided for legacy reasons and to allow the access to the CodeDom specific functionality no available with CSScript.Evaluator.

CSScript.Native

CSScript.Native and CSScript.Evaluator share a great deal of similarities though you need to remember CSScript.Evaluator is the later arrival and as such it represents the result of a more thought through design. Meaning that by default you should use CSScript.Evaluator and CSScript.Native should be reserved for the cases when CSScript.Evaluator functionality is insufficient.

The following is an example of creating a delegate object from a method definition code with CSScript.Native:

var sayHello = CSScript.LoadDelegate<Action<string>>(
                                    @"void SayHello(string greeting)
                                      {
                                          Console.WriteLine(greeting);
                                      }");

sayHello("Hello World!");

And this is the similar task done with CSScript.Evaluator:

var product = CSScript.Evaluator
                      .LoadDelegate<Func<int, int, int>>(
                                  @"int Product(int a, int b)
                                    {
                                        return a * b;
                                    }");

int result = product(3, 2);

CSScript.Evaluator

CSScript.Evaluator exposes the scripting functionality, which is common for all supported compilers. The section below will explain certain runtime differences between the compilers.

This page is dedicated to the detailed analysis of the all aspects of hosting major compilers (via CS-Script). But a very concise summary of the comparative analysis of the compilers is presented in the table below.

Features CodeDom Mono Roslyn
Performance (remote) good very good poor1
Performance (local) good/excellent2 very good very good3
Debugging yes no no
C# 6 support yes4 no yes
CS-Script directives yes no5 no5
Unloading support full partial6 partial6

1 - may improve in future Roslyn releases
2 - if caching is triggered may the performance is 100% identical to a fully compiled (non scripted) code
3 - significant initial loading overhead once per AppDomain; may improve in future Roslyn releases
4 - supported by reconfiguring CS-Script to utilize Roslyn compilers
5 - supported only assembly referencing and probing directories but no any other directives (no multi file scripting)
6 - supported via IEvaluator Extension methods

CodeDom compiler

CSScript.EvaluatorConfig.Engine = EvaluatorEngine.CodeDom;

CodeDom is all about dynamically emitting the assemblies defined by collection of the runtime objects (code DOM objects) and only small part of CodeDom is dedicated for building assemblies defined by the text (code). Only this part is used by CS-Script. CodeDom is very well suitable for dealing with larger pieces of code representing a complete task/job (file or collection of files). Thus it naturally fits the role of a script compiler.
Advantages
  • It is mature. It is in fact the most utilized and tested over the years compiling platform of CS-Script.
  • The most important one: CodeDom emits file-based assemblies. Meaning that it also generates debug symbols. This in turn allows debugging the scripts.
  • By being native part of the CS-Script engine CodeDom supports all CS-Script code directives (e.g. //css_include).
  • The compiled scripts assemblies can be referenced by other scripts (thanks to the file-based assemblies).
  • Compiler loading is subject of CS-Script dependency-injection policy thus you can take advantage of the 'Alternative Compiler' feature. Meaning that it is possible to evaluate the script written in non-C# syntax (e.g. VB).
  • All scripts are automatically subject of CS-Script caching. This prevents script code being compiled more than once. Doesn't matter how many times you execute the script a given unique script code will always be evaluated only once. This in turn leads to an enormous performance bust. But of course it is the case only if the scripts are being executed multiple times are not changed between executions.
DisAdvantages
  • C# 6 syntax for scripts is not available out of box. It is hard to believe but Microsoft did not released C# 6 compiler with .NET 4.6. Though the work around is relatively simple. You just need to provide the path to teh Instead they delegated compiling to custom compiler for C# 6. If you have CS-Script present then it is only a single line of code:
    CSScript.GlobalSettings.UseAlternativeCompiler = 
                        "%CSSCRIPT_DIR%\Lib\CSSCodeProvider.v4.6.dll"; 
  • CodeDom provides a compiler-as-service experience but under the hood it does create some temporary files and these IO operations do not go unnoticed for the performance. This makes CodeDom the slowest of all three compilers. Automatic caching partially compensates the performance loses.

Mono compiler

CSScript.EvaluatorConfig.Engine = EvaluatorEngine.Mono; 

Mono is a true in-memory compiler. Fast, mature, reliable. The native interface of Mono Evaluator is a little unorthodox but because it is accessed via CSScript generic interface there is no any usability impact.
Advantages
  • It is mature.
  • It is fast. Much faster than CodeDom as all operations are in-memory tasks.
  • It is light. Just a single dll (included in CS-Script distro).
Disadvantages
  • The compiled scripts are in-memory assemblies. Meaning that they cannot be debugged. There is no any work around.
  • Mono Evaluator doesn't support C# 6 syntax (at least v4.0.0.0). There is no any work around.

Roslyn compiler

CSScript.EvaluatorConfig.Engine = EvaluatorEngine.Roslyn; 

Roslyn is another true in-memory compiler. It is fast (code evaluation only) and has a reasonably comfortable native interface, which is anyway normalized by the common CSScript.Evaluator. The biggest problem of Roslyn is that it is still not out of beta (v1.2.0-beta-20151211-01).
Advantages
  • It is fast. Practically as fast as Mono.
  • Support C# 6 syntax straight out of box.
Disadvantages
  • It is still in development. Its native interface can change any time and break CS-Script integration. To address this CS-Script is distributed with the full set of Roslyn assemblies.
  • It is heavy.
    • The very first call to can take up to 5 seconds to initialize the compiler infrastructure.
    • You need to reference 6.5M of 8 assemblies in total(included in CS-Script distro).
    • Compiling time is heavily affected by number of ref assemblies (Mono doesn't exhibit such a behaviour)
  • The compiled scripts are in-memory assemblies. Meaning that they cannot be debugged. There is no any work around.
  • Script cannot have namespaces. There is no any work around.
  • The compiled scripts cannot be referenced by other scripts. The only work around it to reuse the same evaluator instance.
  • All script types are compiled as nested classes.
  • Everything (e.g. class code) is compiled as a nested class with the parent class name "Submission#N" and the number-sign makes it extremely difficult to reference the type from other scripts


Last edited Feb 9, 2016 at 11:04 AM by oleg_s, version 3