Variable Lookup Callback?

Aug 21, 2007 at 2:43 AM
Hi!

I have an application that contains a dynamic list of plugins. Each plugin supports a list of properties, although the properties vary by plugin. I would like to use your library to perform evaluations based on the values of the properties of these plugins. Rather than using a DynamicExpressionOwner, is there any way to build a callback or delegate that will allow me to do my own lookup to get variable values? (Overriding EmitVariableLoad could get me a step in the right direction, I guess?)

Also, the names of my plugins and their properties are not ".net compatible" names. They may contain spaces and other characters, if that makes the user happy. For instance, I may have a plugin called "System Clock" with a property called "Current Hour". I'd like to write expression like:

{System Clock:Current Hour} < 12

... that would evaluate to "true" if it's before noon. (I'm not married to the colon in the middle... Gramatically, I can make that anything.)
Internally to the application, the expression above roughly translates to this:

Plugin p = PluginHost.GetPluginByName("System Clock");
int currentHour = (int) p.GetPropertyValue("Current Hour");
bool result = currentHour < 12;

Many thanks in advance,

Peter
Coordinator
Aug 21, 2007 at 3:23 PM
Hi,

Creating a callback to lookup the value of a variable should be fairly straightfoward to implement. The delegate would have to return the value as an object, so there would be a unbox/cast required to get the strongly-typed value in the expression so it won't be the fastest way to access variable values.

>Also, the names of my plugins and their properties are not ".net compatible" names. They may contain spaces and other characters
I don't know if I can support this as the grammar is currently setup to ignore whitespace. I think a solution would be to take the user's expression string (ie: "{System Clock:Current Hour} < 12") and replace the property lookup with a FLEE compatible lookup (ie: "SystemClock.CurrentHour < 12"}, and then compile and evaluate.
Aug 22, 2007 at 2:44 AM
Edited Aug 23, 2007 at 2:01 AM
Hi Eugene,

Indeed. Not looking for fast, if I have to do bursts of 5 evals per second it's probably a lot. I actually hacked an override of the DynamicExpressionOwner together that works but there are some inefficiencies that cause the variable to have to get looked up multiple times (HasVariable, GetVariableType, etc). Plus it's a really ugly hack :)

I can share with you, if you'd like. (how?)

As for grammar: Could you specify an IDENTIFIER as <<[a-z_]\w*|{[a-z_ ]\w*}>>?
I'm sure I have that syntax wrong....

Thanks!

Peter
Coordinator
Aug 22, 2007 at 8:12 PM
Hi Peter,

>but there are some inefficiencies that cause the variable to have to get looked up multiple times
Yeah, variables are looked up twice during compilation. I'll see if I can change the code around to only do it once.

With the expression {System Clock:Current Hour} < 12: How are you looking up the "Current Hour" property? The variable callback would only work on the first variable ("System Clock"). After that, the second variable ("Current Hour") would be looked up on the type of the previous member and not using the callback.

Couldn't you do something like this:
public class PluginExpressionOwner
{
   private object GetPropertyValue(string pluginName, string propertyName)
   {
      Plugin p = PluginHost.GetPluginByName(pluginName);
      return p.GetPropertyValue(propertyName);
   }
}

Pass that owner to an expression and you could use an expression like: cast(GetPropertyValue("System Clock", "Current Hour"), int) > 12 to achieve what you want. The cast clutters up the expression but is required since the expression language is strongly-typed. You could also pre-parse the expression, replace all property calls with the correct function call, and then pass the string to an expression.

Maybe you could post your hack in a reply so I can see how you went about it.

Thanks,

Eugene
Aug 22, 2007 at 9:02 PM
Edited Aug 23, 2007 at 2:00 AM
Hi Eugene,

Hadn't thought of doing THAT! It's certainly a pretty clean way of doing it, but, as you say, the cast does clutter up the expression. I'm envisioning the {plugin:value} as one "fully qualified variable" to my end user. For instance, I could have two instances of the clock plugin, each running at the different timezone. They'd both have a "Current Hour" property, but their fully qualified names would be different. Your idea is still valid, though.... I could do a something like: cast(Variable("System Clock:Current Hour"), int) and simply do the split myself. Speaking of which, could I write the cast as (int)(Variable("System Clock:Current Hour") ? Still... it looks ugly in the expression.

As to how I actually do the lookup...
Each plugin exposes a list of PropertyDescriptors. The PropertyDescriptor contains things like the name of the property, type, read only, etc., but also a reflected PropertyInfo that points to the actual property that has the value. To the writer of the plugin, this is all abstracted by base classes and attributes. The clock actually looks something like this:
public class ClockPlugin : Plugin
{
  [ExposedProperty("Current Time", readonly=true)]
  public DateTime CurrentTime
  {
    get{ return DateTime.Now; }
  }
 
  [ExposedProperty("Current Hour", readonly=true)]
  public int CurrentHour
  {
    get{ return DateTime.Now.Hour; }
  }
}

The plugin host simple enumerates all the properties that have a ExposedPropertyAttribute and maintains a map.
Aug 22, 2007 at 9:02 PM
Hmm... How do I do the fancy formatting like you did?
Coordinator
Aug 22, 2007 at 10:10 PM


pzand wrote:
Hmm... How do I do the fancy formatting like you did?


See the Guide
Coordinator
Aug 22, 2007 at 10:38 PM
>Still... it looks ugly in the expression.
Yeah, you're doing things dynamically while the expression language is statically typed. The cast is required.
I could implement generic method calls and then the expression becomes GetPropertyValue<int>("System Clock", "Current Hour") > 12. Is that a good compromise?

Regardless of the approach, you will will always have to specify a type when you reference a property. If the type is wrong, you will get an InvalidCastException when you evaluate the expression. Or you can switch to a dynamic .NET language like IronPython which will eliminate the cast altogether.
Aug 23, 2007 at 1:41 AM
Getting so close!
Remember that my plugin host already knows the type of the object that it would return from the GetValue call, so the end-user shouldn't have to specify it.
My ugly hack looks like this:
Imports System.Reflection.Emit
Imports System.Reflection
 
Public MustInherit Class LookupExpressionOwner
  Inherits DynamicExpressionOwner
 
  Public Sub New()
  End Sub
 
  Friend Overrides Sub EmitVariableLoad(ByVal name As String, ByVal ilg As System.Reflection.Emit.ILGenerator, ByVal services As System.ComponentModel.Design.IServiceContainer)
    Dim mi As MethodInfo = Me.GetType().GetMethod("GetValue", BindingFlags.Instance Or BindingFlags.Public)
    ilg.Emit(OpCodes.Ldstr, name)
    ilg.Emit(OpCodes.Call, mi)
 
    Dim nopElement As New NopExpressionElement(Type.GetType("System.Object"))
    Dim castElement As New CastOperator(nopElement, GetValueType(name))
    castElement.Emit(ilg, services)
  End Sub
 
  Friend Overrides Function HasVariable(ByVal name As String) As Boolean
    Return VariableExists(name)
  End Function
 
  Friend Overrides Function GetVariableValueType(ByVal name As String) As System.Type
    Return GetValueType(name)
  End Function
 
  Public MustOverride Function GetValue(ByVal name As String) As Object
  Public MustOverride Function VariableExists(ByVal name As String) As Boolean
  Public MustOverride Function GetValueType(ByVal name As String) As Type
End Class

I'm a C# guy, so I struggled a little with how VB wants its overrides. Plus I didn't want to completely mess up your DynamicExpressionOwner and change Friends into Publics, since that causes all kinds of other bad things. Also, since I've never done anything directly in IL, I ran into some trouble there and just bluntly followed your pattern.

What I wanted to do, but couldn't figure out:
Friend Overrides Sub EmitVariableLoad(ByVal name As String, ByVal ilg As System.Reflection.Emit.ILGenerator, ByVal services As System.ComponentModel.Design.IServiceContainer)
  dim value as object
  value = GetValue(name)
  
  '-- have ilg emit an object. How? OpCodes.Ldobj does something else?
  '-- ilg.Emit( OpCodes.Ldobj, value )
 
  Dim nopElement As New NopExpressionElement(Type.GetType("System.Object"))
  Dim castElement As New CastOperator(nopElement, value.GetType())
  castElement.Emit(ilg, services)
End Sub

Now, if you can make anything between curly braces appear as one variable, I'm set :)

Looked at grammatica, but couldn't figure out how to regenerate grammar. Or I would have tried it!

Appreciate the quick replies, btw...
Coordinator
Aug 23, 2007 at 7:00 PM
Hi Peter,

After sleeping on it, it dawned on me that the properties you are talking about represent real properties on an object. In that case, Flee can just access them directly.
DynamicExpressionOwner owner = new DynamicExpressionOwner();
owner.DefineVariable("SystemClock1", typeof(Plugin));
Plugin p = PluginHost.GetPluginByName("System Clock");
owner.SetVariableValue<Plugin>("SystemClock1", p);
 
Expression e = new Expression("SystemClock1.CurrentHour + 1", owner);

The only thing left is to map from friendly plugin names to flee names. Same for properties. The plugin host should be able to use the propertyDescriptor to get the actual name of a property given its friendly name. Here's some pseudo-code to explain what I'm getting at:
// Get expression string entered by user
string userExpression = mytextbox.text;
// Create a regex to manipulate the string and replace PluginName and PropertyName with flee-friendly equivalents
Regex replacer = new Regex("{PluginName:PropertyName}");
// Replace all plugin names and properties with flee friendly names
string fleeExpression = replacer.Replace(userExpression, replaceCallback);
// Create the expression using the flee-friendly text
Expression e = new Expression(fleeExpression, owner);
// Evaluate, etc

This solution requires that you load all your plugins in to the owner's variable collection. If you only want to load plugins on demand, we'll have to do some tweaks.
Aug 24, 2007 at 3:04 AM
Oh... DUHHH!! I never even considered the possibility of presenting one thing to the user, but presenting something completely different to flee! I could have the user write anything, as long as I can parse it and create something that flee can understand. The expression to flee can then be as complex or as simple as needed, including any "ugly" casts or method calls.

I could take your concept one step further and replace the whole {PluginName:PropertyName} deal with something like cast(LookupVariable("PluginName", "PropertyName"), type) before I give it to flee. After all, during the parse, I already know the type! All I'd have to do is give the expression owner a method like you described earlier in your post of Wed at 4:12 PM.

I can't believe it took me this long....
Aug 24, 2007 at 4:11 AM
Edited Aug 24, 2007 at 4:13 AM
SUCCESS!

Figured I'd share the solution that works for me. YMMV.

I created a "GenericExpressionOwner" that exposes a "GetPluginPropertyValue" method as described above. I've also added a couple of static methods that allow for easy usage in the main code.
Main code:
string userExpr = "{System Clock:Current Date Time}.Hour > 12";
Expression expr = GenericExpressionOwner.CreateExpression(userExpr);
ExpressionEvaluator<bool> evaluator = (ExpressionEvaluator<bool>)expr.Evaluator;
bool result = evaluator();

And GenericExpressionOwner:
class GenericExpressionOwner
{
  public object GetPluginPropertyValue(string PluginName, string PropertyName)
  {
    PluginBase plugin = PluginHost.GetPluginByInstanceName(PluginName);
    return plugin.GetPropertyValue(PropertyName);
  }
 
  public static string PreparseExpression(string expr)
  {
    // Replace all instances of {Plugin Name:Property Name}
    // with:
    //   cast(GetPluginPropertyValue("Plugin Name", "Property Name), type)
 
    string Result = expr;
 
    Regex re = new Regex(@"\{([^:]*):([^:]*)\}");
    MatchCollection matches = re.Matches(expr);
    foreach (Match m in matches)
    {
      string MatchedString = m.Groups[0].Value;
      string PluginName = m.Groups[1].Value;
      string PropertyName = m.Groups[2].Value;
 
      // we could iterate through the collection of properties and find the 
      // type that's defined for this property, but it's much easier
      // to just get the value and determine the type this way.
 
      PluginBase plugin = PluginHost.GetPluginByInstanceName(PluginName);
      object Value = plugin.GetPropertyValue(PropertyName);
      string PropertyType = Value.GetType().ToString();
 
      string NewString = String.Format(
        "cast(GetPluginPropertyValue(\"{0}\",\"{1}\"),{2})",
        PluginName,
        PropertyName,
        PropertyType);
 
      Result = Result.Replace(MatchedString, NewString);
    }
    return Result;
  }
 
  public static Expression CreateExpression(string expr)
  {
    string fleeExpr = GenericExpressionOwner.PreparseExpression(expr);
 
    ExpressionOptions options = new ExpressionOptions();
    options.Imports.AllowGlobalImport = true;
    options.Imports.ImportBuiltinTypes = true;
    options.Imports.AddNamespace("System");
 
    GenericExpressionOwner owner = new GenericExpressionOwner();
    return new Expression(fleeExpr, owner, options);
  }
}
I'm sure there's a better way to do the replace with the regex, but this is the easy way. Since the main app will cache the created expression anyway, the parsing and replacing only happens once per expression that the user defines. No need for much optimization.

BTW, note how the property is actually of type DateTIme and I'm thus having flee do the call to the "Hour" property on the resulting DateTime object :)

Thanks for all your help and a wonderful library, Eugene!

Peter
Coordinator
Aug 24, 2007 at 4:20 PM
Hi Peter,

Great! Glad to see you got it working.

Eugene