Dynamic Expressions and Generics

Sep 2, 2007 at 12:48 PM
Hi there

I'm loving flee - it makes a lot of things very easy. Kudos to you! However, I'm having a problem with the dynamic ExpressionOwner class. This is part of your sample:

// Define variables "a" and "b" and specify their type
owner.DefineVariable("a", typeof(int));
owner.DefineVariable("b", typeof(double));

// Change the values of the variables
owner.SetVariableValue<int>("a", 4);
owner.SetVariableValue<double>("b", 3.4);


These two blocks have a fundamental difference: The first part (definition of the variable) is truly dynamic: The DefineVariable() method takes just a type parameter, which I can determine at runtime. However, the SetVariableValue method expects the type declaration at compile time, which renders this potentially powerful feature useless. This code here doesn't work although everything the ExpressionOwner class receives at runtime is of type int:

private void TestExpressionOwner()
{
CreateDynamicExpressionOwner(typeof(int), "123");
}

public void CreateDynamicExpressionOwner(Type t, string value)
{
DynamicExpressionOwner o = new DynamicExpressionOwner();
//this works fine, a variable of type int is being created
o.DefineVariable("x", t);

//the problem is here - the type depends on what's submitted at runtime,
//this does not work with generics
o.SetVariableValue("x", Convert.ChangeType(value, t));
}


This is because Convert.ChangeType returns an object of type "object". At runtime, this would be an int, but that's the problem with generics - they are compile-time dependent. However, you already have the type so I guess you wouldn't need the generic type parameter anyway - any chance to get a fix (or non-generic overload) for this?

Best
Philipp
Coordinator
Sep 2, 2007 at 11:00 PM
Hi Philipp,

Internally, I implement dynamic variables using generics to keep everything as fast as possible and I think I got carried away. It doesn't make sense to have dynamic variables whose type is static.

I've removed the generic get/set methods and replaced them with object-based equivalents. The changes are in release 0.9.6.0.

Thanks,

Eugene
Sep 3, 2007 at 8:03 AM
Hi Eugene

Wow, that was fast and everything works as expected now - thank you very much :-)

Keep up the great work!
Philipp

ps: You ever considered putting on a PayPal account or Amazon wish list? I sure would like to donate a little something...
Coordinator
Sep 3, 2007 at 7:07 PM
>ps: You ever considered putting on a PayPal account or Amazon wish list? I sure would like to donate a little something...

Hmm...never thought of that. I'll have to look into CodePlex's policy on it.
Coordinator
Sep 4, 2007 at 1:52 AM
Out of curiosity, what are you using Flee for? What problems does it help you solve? I'm trying to get a feel for the scenarios people are using it in.

Thanks.
Sep 4, 2007 at 7:56 AM
Edited Sep 4, 2007 at 10:01 AM
A somewhat short version:

- The application I'm developing is used to monitor and control a piece of hardware ("target"). However, for testing and demo purposes, we need to be able to run the application without a real target
- The application is built upon plug-ins, so all we had to do was create a simulation plug-in that declares itself as a target device controller.
- Interaction with a target is primarily based on parameters of different data types (ints, doubles, strings etc.). However, changing a target's parameters is not an isolated thing, but may cause side-effects: E.g. opening a valve (parameter A) causes pressure (parameter B) to increase.

For the demo plug-in, this behaviour is not hardcoded, but declaratively described using XML files. I've developed a generic rules engine (which I plan to release under the LGPL as well as soon as we're done) which allows me to declare actions ("increase value") and conditions ("only if gate is open"). I've developed a set of actions and conditions for this this demo plug-in, and there are some that use expressions. A sample declaration may look like this:


<!-- only allow changes if the context item (x) is within a given threshold -->
<rule id="threshold test" item-id="105">
<conditions>
<condition id="tc" type="ExpressionCondition" var-y="110" var-z="120"
expression="x &gt;= y AND x &lt;= z" />
</conditions>
<actions>
<action id="inc" type="IncrementDataPointAction" value="0.01" condition="tc" />
</actions>
</rule>


At runtime, I'm reading variable declarations ("x" is implicitely declared and represents item 105), and determine the data types of the evaluated parameters. This is were the DynamicExpressionOwner comes in VERY handy as both the number of variables and the underlying data types only depend on the markup :-)

Cheers!
Sep 4, 2007 at 12:02 PM
Hi again :-)

Another question: Wouldn't it be possible to make evaluation dynamic as well? Currently, I need to evaluate an expression like this:

ExpressionEvaluator<int> evaluator = (ExpressionEvaluator<int>)expression.Evaluator;
int result = evaluator();

Same as with variable declaration, this requires me to know at compile time what to expect, which is a loss of flexibility: I can declare my types dynamically thanks to the DynamicExpressionOwner, but the output must be of a given type. If possible, two variants would be great:

- a generic one we have right know (type-safe, convenient if the expected type is known at compile time)
- a dynamic one (might expect a type parameter or even determine the returned type on its own) which just returns object

Cheers,
Philipp
Coordinator
Sep 4, 2007 at 1:11 PM
Try the following:
ExpressionOptions options = new ExpressionOptions();
// Put an implicit convert to object at end of expression
options.ResultType = typeof(object);
 
Expression e = new Expression("1+1", owner, options);
// The evaluator will now always be of the same type as the ResultType
ExpressionEvaluator<object> evaluator = (ExpressionEvaluator<object>) e.Evaluator;
object result = evaluator();
Sep 4, 2007 at 9:58 PM
Edited Sep 4, 2007 at 9:59 PM
Hi Eugene

Thanks (again) for your help - works like a charm! However, wouldn't it come handy to be able to set the required type at runtime?

ExpressionOptions options = new ExpressionOptions();
options.ResultType = typeof(int); //usually determined dynamically
 
Expression e = new Expression("2 * 2", owner, options);
 
// Evaluate() returns object, but its the ResultType
object result = e.Evaluate();
Assert.IsInstanceOfType(typeof(int), result);


While everything is hardcoded above, ResultType as well as the expression and variable types may vary with dynamic code. Setting a desired ResultType to something more specific than object might allow built-in type conversions, which - afaik - are not currently supported but sure would be nice when it comes to dynamic code. However, I have no idea about the impact on the architecture and there might also be valid arguments against this request.
Example: The fragment below compiles fine, but throws an ExpressionCompileException rather than returning an int-value of 5:

ExpressionOptions options = new ExpressionOptions();
options.ResultType = typeof (int);
 
//double expression - throws exception (no type conversion)
Expression e = new Expression("2 * 2.5", new DynamicExpressionOwner(), options);
Coordinator
Sep 5, 2007 at 11:31 PM
>Setting a desired ResultType to something more specific than object might allow built-in type conversions, which - afaik - are not currently supported but >sure would be nice when it comes to dynamic code.
Type conversions are supported but they work differently than System.Convert. The conversions are more low-level and work just like C#: conversions between numeric types, no conversions to/from boolean, cannot convert "123" to a number.

The ResultType setting basically puts an implicit convert at the end of the expression. So if you set the Result type to double and have an expression like 2 * 4, then it will work since an integer can always be implicitly converted to a double. Your example threw an exception because there is no implicit conversion from double to int. You can force it using an explicit conversion: cast(2 * 2.5, int) which will truncate the double into an integer.

I'll have to think about the two types of conversions and how they would mix.
Sep 13, 2007 at 8:28 PM
Hell, great tool. I was dealing with some of the issues mentioned above, and ended up deciding that Generics was the way to go. So, I ended up with this wrapper class. Note that the variables collection does not support the 'Contains' method, hence my use of the Arraylist. 'Contains' would be a welcome addition, and simplify my code somewhat.

Any comments or suggestions on how I am using this would be welcome.

===================================
Imports System
Imports System.Collections

' for FLEE
Imports ciloci.Flee

Namespace MiddleTier.ItemLogic
Public Class CalcEngine(Of t)

Private _expressionOwner As New DynamicExpressionOwner()
Private _expression As Expression
Private _expressionOptions As ExpressionOptions
Private _calculation As String
Private _variables As New ArrayList()

Public Sub New()
_expressionOptions = New ExpressionOptions()
_expressionOptions.Imports.AddType(GetType(Math))
_expressionOptions.Imports.AllowGlobalImport = True
_expressionOptions.Imports.ImportBuiltinTypes = True
_expressionOptions.ResultType = GetType(t)
End Sub


Public Sub AddVariable(ByVal variableName As String, ByVal variableType As System.Type)
' make sure we do not already have it
If Not _variables.Contains(variableName) Then
_expressionOwner.DefineVariable(variableName, variableType)
_variables.Add(variableName)
End If

End Sub

Public Sub SetVariableValue(ByVal variableName As String, ByVal value As t)
If _variables.Contains(variableName) Then
_expressionOwner.SetVariableValue(variableName, value)
End If
End Sub

Public Function GetVariableValue(ByVal variableName As String) As Object
If _variables.Contains(variableName) Then
Return _expressionOwner.GetVariableValue(variableName)
Else
Return Nothing
End If
End Function

Public Sub ImportType(ByVal type As System.Type)
_expressionOptions.Imports.AddType(GetType(Type))
End Sub

Public Sub ImportNamespace(ByVal ns As String)
_expressionOptions.Imports.AddNamespace(ns)
End Sub

Public Property Calculation() As String
Get
Return _calculation
End Get
Set(ByVal calc As String)
_calculation = calc
expression = New Expression(calculation, _expressionOwner, _expressionOptions)
End Set
End Property

''' <summary>
''' evaluate the expression based on the values of
''' the current variables and return the result
''' </summary>
''' <remarks>
''' </remarks>
''' <returns></returns>
Public ReadOnly Property Result() As t
Get
Dim retval As t = Nothing
If _calculation.Length > 0 Then
Dim eval As ExpressionEvaluator(Of t) = CType(_expression.Evaluator, ExpressionEvaluator(Of t))
retval = eval()
End If
Return retval
End Get
End Property

End Class
End Namespace
===================================

In a nutshell, each time my object is accessed, I have to access other objects (by name) to get their values, and then apply the user-defined calculation based on the current value of other objects, so my class has an instance of this class that does the calculation work for me. If I am dealing with time series data, I loop through the dependent values, assign their value for a given row to the named variable, and calculate the new result, save that with the timestamp of the dependent values, then move to the next result. This works great for me. The use of Generics allows me to decide the result type based on runtime criteria, and the burden is on the consumer of the class to cast appropriately. The DynamicExpression class allows me to use the object name as the variable name. VERY nice!

Noel
Sep 13, 2007 at 8:29 PM
pardon me, the first word should have been 'Hey, all!'. Dagnabbit bluetooth keyboard.
Coordinator
Sep 13, 2007 at 11:05 PM
Hi Noel,

Glad to hear it's working for you. I looked over your code and I don't see any problems with the way you are using the library.

I will put in a Contains method in the next release.

Thanks!

ps: Codeplex lets you edit comments after posting them