OSLO: Attended A Presentation Last Night

April 28, 2010

Last night, I attended a presentation about OSLO. As you may know from previous posts, I’ve been playing around with it for a few days now. In particular, I’ve been playing with the dsl/parser piece of it. I hoped that I’d be able to ask a couple specific questions at the presentation, but it didn’t happen.

As a text parsing presentation, it was very good. As an OSLO presentation, it wasn’t good.

The speaker obviously knows what he’s talking about. He defined how parsers work, and how they fit into the big scheme of things. OSLO is a very small piece of the big picture (as presented). In the speaker’s opinion, that piece could be better filled by a number of other products including ANTLR.

The general parsing presentation as great. But, he really undersold OSLO. I’ve done little more than goof around with it and watch a bunch of videos, but I have a much greater appreciation for what it’s supposed to do. The languages you write get imported into the database, then you can use EntityFramework to hit it. It gives you a full solution based on domain modeling. Maybe it’s not all working… I don’t know. But, he didn’t speak of or demonstrate anything other than Intellipad, which wasn’t working very well. We really didn’t spend any time on OSLO itself, or any of the moving parts other than Intellipad. I don’t think he even mentioned Quadrant. (If he did, it was a negligible mention.)

Informative, yes. Informative about OSLO? Insightful? Thorough? No. If I hadn’t already played with OSLO, then this would not have generated any interest.

Advertisements

NOODB: “M”

April 26, 2010

I mentioned previously that I’m sure I’m not going to be happy with the query parser I built.

TEXT => NEUTRAL FORMAT => REPOSITORY SPECIFIC QUERY

The NEUTRAL FORMAT is good, at least so far. But, it’s the conversion process from text to the neutral format that’s going to be beat.

I spent some time today working with “M”. Since M allows you to define DSLs and how to parse them, I thought I could use it to parse the text.

The theory is holding up, though progress has been slow.

Given this sample text:

from m in movie select m.Crap
from d in director select d.Name

I created an “M” grammar…

module HelloWorld {
    @{CaseInsensitive{}}
    language HelloWorld {
        syntax Main = 
            s:SelectQuery* => SelectQueries { valuesof(s) };
        
        syntax SelectQuery = From a:Alias In t:Table Select Field
            => { Alias => a, Table => t};
        
        @{Classification["Keyword"]}
        token From = "from";

        @{Classification["Keyword"]}
        token In = "in";

        @{Classification["Identifier"]}
        token Alias = AlphaNumeric;

        @{Classification["Identifier"]}
        token Table = AlphaNumeric;
        
        @{Classification["Identifier"]}
        token Select = "select";
        token AlphaNumeric = ('A'..'Z' | 'a'..'z' | '0'..'9')+;       
        interleave ignore = ' ' | '\r' | '\n';   
                
        token FieldAlias = AlphaNumeric;
        token FieldName = AlphaNumeric;
        syntax Field = 
                FieldAlias + "." + FieldName
                | FieldAlias;
    }
}

That results in this, so far…

SelectQueries{
  {
    Alias => "m",
    Table => "movie"
  },
  {
    Alias => "d",
    Table => "director"
  }
}

I’ve hardly scratched the surface of what I’m going to need, but at least it’s some progress.

The OSLO stuff is pretty amazing. I watched 7 videos today… cool product. I don’t know yet how this grammar is going to convert to code, but we’ll see.


NOODB: Query Update

April 24, 2010

Last night I was preparing a demo of the major working features of NOODB. I wrote a test to demo the query, and one of the asserts failed.

I picked apart the sql and found the problem. I was originally planning to create a pivot table. As I built it, though, I thought it was better to go with another approach. I thought the other approach worked, but it didn’t hold up with more complex queries. After slapping it around, I ended up back with a pivot table.

One key difference is that before, the query attempted to resolve everything by joining to the meta tables. This resulted in some funky queries due to the hierarchical nature of the meta data. Since those intricacies are already worked out in code, I changed it to take advantage of that.

I offloaded the construction of the pivot table to a new object. The table is wrapped up in a CTE that can be queried by things consuming the CTE, such as the query engine.

In the end, it’s a lot cleaner and simpler, and I’m back to the pivot table. The following test now passes. (Prior to the refactor, the third assert failed.)

        [TestMethod]
        public void Query()
        {
            TestUtility.DeleteTestData();
            NoodbClient client = new NoodbClient(new Server(TestUtility.UnityContainer));
            client.RegisterType<Movie>();

            // have to cheat and use single word director.
            // problem with query parser and spaces.
            client.Save(new Movie { Director = "Lucas", Name = "Star Wars", ReleaseYear = 1977 });
            client.Save(new Movie { Director = "Kasdan", Name = "The Empire Strikes Back", ReleaseYear = 1980 });
            client.Save(new Movie { Director = "Marquand", Name = "Return of the Jedi", ReleaseYear = 1983 });
            client.Save(new Movie { Director = "Lucas", Name = "The Phantom Menace", ReleaseYear = 1999 });
            client.Save(new Movie { Director = "Lucas", Name = "Attack of the Clones", ReleaseYear = 2002 });
            client.Save(new Movie { Director = "Lucas", Name = "Revenge of the Sith", ReleaseYear = 2005 });
            client.Save(new Movie { Director = "Whedon", Name = "Serenity", ReleaseYear = 2005 });
            client.Save(new Movie { Director = "Jackson", Name = "King Kong", ReleaseYear = 2005 });

            Assert.AreEqual(4, client.Query<Movie>("from m in Movie where m.Director='Lucas'").Count);
            Assert.AreEqual(3, client.Query<Movie>("from m in Movie where m.ReleaseYear = 2005").Count);
            Assert.AreEqual(3, client.Query<Movie>("from m in movie where (m.ReleaseYear = 2005 && m.Director='Lucas') || m.ReleaseYear = 1999 || m.ReleaseYear = 2002").Count);
        }

NOODB: Query works!

April 21, 2010

I’m pretty pleased right now. I can now run queries in NOODB. Sure, they’re just simple queries, but everything starts with a first step.

            TestUtility.DeleteTestData();
            NoodbClient client = new NoodbClient(new Server(TestUtility.UnityContainer));
            client.RegisterType<QueryPerson>();

            client.Save(new QueryPerson { FirstName = "jay", LastName = "allard" });
            client.Save(new QueryPerson { FirstName = "gina", LastName = "allard" });
            client.Save(new QueryPerson { FirstName = "paul", LastName = "allard" });
            client.Save(new QueryPerson { FirstName = "jack", LastName = "allard" });
            client.Save(new QueryPerson { FirstName = "amy", LastName = "harrington" });
            client.Save(new QueryPerson { FirstName = "dawn", LastName = "almas" });

            Collection<QueryPerson> matches = client.Query<QueryPerson>("from p in QueryPerson where p.FirstName='amy' || p.LastName='almas'");
            var ordered = matches.OrderBy(match => match.FirstName).ToList();
            Assert.AreEqual(2, matches.Count);
            Assert.AreEqual("amy", ordered[0].FirstName);
            Assert.AreEqual("dawn", ordered[1].FirstName);

You can’t do joins or property drilling yet, but hey, let’s focus on what we can do. We can write queries!

I’m going to do some easy stuff for a while; add some more operators; maybe do an IN, stuff like that.

The query that I previously posted had to be changed slightly. As I wrote it, I was expecting there to be a pivot table available, but that ended up not being practical. So, the structure of the query is the same, but instead of saying LASTNAME=, it says VALUESTRING=, or VALUEINT=, etc.

This is what I’ve been working towards since day 1. This is a good day in the NOODB universe (population: 1).


NOODB: Complex Where Building

April 17, 2010

The where parser is complete for AND and OR. I have several samples; this is the most complex so far. The only operator is =. Adding < and > is a no brainer. LIKE will take a bit more work; IN will probably take quite a bit more work. But, it’s getting there.

Remember that this is a two-step process. The first step converts the NOODB query to a object graph that’s more easily convertible to any other format. In this case, the other format is sql. I haven’t touched the Sql convert in a long time; all the work has been in step 1. This proves that step 2 is very simple. I added complexity without having to update the Sql converter.

NOODB Query

from p in QueryPerson 
                where 
                    ((p.FirstName='Jay' && p.LastName='Allard')
                    || (p.FirstName='Amy' && p.LastName='Harrington')
                    || p.Random = 'blah') && ((p.a = 1 || p.b = 2 || p.c = 3) || p.d = 4 && p.e = 5))

SQL Where Clause

(
	
	(
		
		(
			[p].[FirstName] = 'Jay' 
			And [p].[LastName] = 'Allard'
		)		
		Or 
		(
			[p].[FirstName] = 'Amy' 
			And [p].[LastName] = 'Harrington'
		)		
		Or [p].[Random] = 'blah'
	)	
	And 
	(
		
		(
			[p].[a] = 1 
			Or [p].[b] = 2 
			Or [p].[c] = 3
		)		
		Or [p].[d] = 4 
		And [p].[e] = 5
	)	
)

UNITY: Resolve on Demand

April 11, 2010

Unity is the core of most programs that I now write. But, in the 1.x version, it couldn’t do something that I occasionally needed. I wanted a particular dependency to resolve more than once, not just when the object was composed.

Example:

new SomeService(new SomeRepository(new SomeConnectionProvider()))

You can configure Unity to do that for you. The problem is that maybe you don’t want to use the same connection provider over and over; maybe you want it to resolve every time something needs a connection. Lifetime managers don’t help for a single instance, because those only do things during resolve, and you only resolve once.

In Unity 1.x, there were some workarounds. I’ll cover 2 here. (Any work-arounds involving static variables not worthy of comment, are henceforth ignored.)

Old Solution #1: Pass in the Unity Container rather than a Connection Provider.

new SomeService(new SomeRepository(unity))

Now, SomeRepository can resolve IConnectionProvider whenever it wants.

Pros

  • It works. Swell.

Cons

  • It binds the object to Unity.
  • You now have a full container when all you really need is one object.

I used this solution in the AW BUS SERVER, but it was a calculated decision. Everything in the server has lots of shared dependencies, and the whole thing was Unity aware to begin with, so I didn’t mind this approach. However, if I used Unity 2, then I wouldn’t have needed this approach.

Old Solution #2: Pass a connection provider factory rather than a connection provider

In this approach, rather than pass in SomeConnectionProvider(), you pass in SomeConnectionProviderFactory(). The factory will instantiate a new instance of SomeConnectionProvider() each time it needs to. As far as SomeRepository is concerned, the factory is just a provider…

Pros

  • It works. Swell.
  • It doesn’t bind you to Unity

Cons

  • Do you really want to create a new factory class for each class that you want to resolve?
  • It feels a little kludgy… Unity supports lifetime management. By using a factory, you’re not able to take advantage of what unity offers.

Unity 2 Solution: Delegate of Func<TResult>

Unity now recognizes Func<TResult> parameters. If a parameter is a Func<TResult>, then it assigns the delegate to the containers Resolve<T> method. If a parameters if of type IEnumerable<Func<TResult>>, then it performs a ResolveAll<T>.

To revisit our example, SomeRepository accepts a constructor parameter of type IConnectionProvider. To take advantage of the new feature, simply change it to Func<IConnectionProvider>(). Whenever your code executes the delegate, it will resolve.

Pros

  • It works. Swell.
  • It doesn’t bind you to unity
  • It doesn’t require you to write a bunch of factory classes

Cons

  • None, really. It might seem unusual to have a delegate as a parameter, but that’s a result of the requirements of your object, not unity. You have determined that some outside source needs to server you another type of object. That other type of thing is unity in this case, but it can be anything.

The Code

This code demonstrates the following:

  1. The problem. It’s only a problem, though, when you want to resolve after instantiation, which doesn’t come up to often for me.
  2. Unity 1.x solution – pass the container
  3. Unity 1.x solution – pass a factory
  4. Unity 2.x solution – change paramters to Func<TResult>. This shows that it solves the problem.
  5. Unity 2.x solution singleton – This is the same as solution #4, but shows that you can use this solution with a lifetime a manager. (It resolves every time, but the resolves all return the same object.)
namespace UnityTest2
{
    using System;
    using System.Threading;
    using Microsoft.Practices.Unity;

    class Program
    {
        private static void Demo(IUnityContainer unity)
        {
            ConsoleColor original = Console.ForegroundColor;

            Demo demo = unity.Resolve<Demo>();
            Console.ForegroundColor = ConsoleColor.Yellow;
            Console.WriteLine(demo.Title);
            Console.ForegroundColor = original;
            Console.WriteLine();
            Console.WriteLine(demo.Explanation);
            Console.WriteLine();

            Console.WriteLine(demo.Ticker.GetTicks());
            Thread.Sleep(500);
            Console.WriteLine(demo.Ticker.GetTicks());

            Console.WriteLine();
            Console.WriteLine();
        }

        /// <summary>
        /// Shows that the ticker provider is only instantiated once.
        /// It will always use the same ticker provider, so the ticks will never
        /// change despite a sleep
        /// </summary>
        private static void TheProblem()
        {
            IUnityContainer unity = new UnityContainer();
            unity.RegisterType<Demo, DemoTheProblem>();
            unity.RegisterType<ITickProvider, TickProvider>();
            Demo(unity);
        }

        private static void OldSolution_PassTheContainer()
        {
            IUnityContainer unity = new UnityContainer();
            unity.RegisterType<Demo, DemoOldSolution1_PassAContainer>();
            unity.RegisterType<ITickProvider, TickProvider>();
            Demo(unity);
        }

        private static void OldSolution_PassAFactory()
        {
            IUnityContainer unity = new UnityContainer();
            unity.RegisterType<Demo, DemoOldSolution2_PassAFactory>();
            unity.RegisterType<ITickProvider, TickerFactory>();
            Demo(unity);
        }

        private static void NewSolution_NewOneEveryTime()
        {
            IUnityContainer unity = new UnityContainer();
            unity.RegisterType<Demo, DemoNewSolution>();
            unity.RegisterType<ITickProvider, TickProvider>();
            Demo(unity);
        }

        private static void NewSolution_Singleton()
        {
            IUnityContainer unity = new UnityContainer();
            unity.RegisterType<Demo, DemoNewSolution>();
            unity.RegisterType<ITickProvider, TickProvider>(new ContainerControlledLifetimeManager());
            Demo(unity);
        }

        static void Main(string[] args)
        {
            TheProblem();
            OldSolution_PassTheContainer();
            OldSolution_PassAFactory();

            Console.WriteLine("Problem solved...");
            NewSolution_NewOneEveryTime();

            Console.WriteLine("Same as previous, but tell unity it's to be a singleton.");
            NewSolution_Singleton();
            Console.ReadLine();
        }
    }

    public interface ITickProvider
    {
        long GetTicks();
    }

    public class TickProvider : ITickProvider
    {
        private readonly long ticks = DateTime.Now.Ticks;
        public long GetTicks()
        {
            return this.ticks;
        }
    }

    public abstract class Demo
    {
        public abstract ITickProvider Ticker { get; }
        public abstract string Title { get; }
        public abstract string Explanation { get; }
    }

    public class DemoTheProblem : Demo
    {
        private readonly ITickProvider ticks;
        public DemoTheProblem(ITickProvider ticks)
        {
            this.ticks = ticks;
        }

        public override ITickProvider Ticker
        {
            get { return this.ticks; }
        }

        public override string Title
        {
            get { return "The Problem"; }
        }

        public override string Explanation
        {
            get { return "Unity composes the object graph once. But, what if you want to go back to the unity container to resolve dependencies again after the instantiation? There's not a great way to do it in Unity 1.x.\r\n\r\nOur goal is to be able to configure, in unity, if a dependency should resolve on each use or not."; }
        }
    }

    public class DemoOldSolution1_PassAContainer : Demo
    {
        private readonly IUnityContainer unity;
        public DemoOldSolution1_PassAContainer(IUnityContainer unity)
        {
            this.unity = unity;
        }

        public override ITickProvider Ticker
        {
            // resolves every time it calls
            get { return this.unity.Resolve<ITickProvider>(); }
        }

        public override string Title
        {
            get { return "Old solution 1: Pass the container and resolve every time."; }
        }

        public override string Explanation
        {
            get { return "The object has a reference to the unity container, so it resolves things as it needs it. But, now it's bound to unity."; }
        }
    }

    public class TickerFactory : ITickProvider
    {
        public long GetTicks()
        {
            return new TickProvider().GetTicks();
        }
    }

    public class DemoOldSolution2_PassAFactory : Demo
    {
        private readonly ITickProvider ticker;
        public DemoOldSolution2_PassAFactory(ITickProvider ticker)
        {
            this.ticker = ticker;
        }

        public override ITickProvider Ticker
        {
            get { return this.ticker; }
        }

        public override string Title
        {
            get { return "Old solution #2: Pass a factory."; }
        }

        public override string Explanation
        {
            get { return "Unity serves a factory instead of an actual implementation. The factory creates new instance every time. It works, but you'd end up with a bunch of factories. Also, you're not using the Unity Lifetime managers... you've created your own."; }
        }
    }

    public class DemoNewSolution : Demo
    {
        private readonly Func<ITickProvider> tickerGetter;
        public DemoNewSolution(Func<ITickProvider> tickerGetter)
        {
            this.tickerGetter = tickerGetter;
        }

        public override ITickProvider Ticker
        {
            get { return this.tickerGetter(); }
        }

        public override string Title
        {
            get { return "Unity 2 Solution: Pass a delegate that will resolve."; }
        }

        public override string Explanation
        {
            get { return "Unity 2 supports the Func<T> delegate, where T is the type you want to resolve. Each time Ticker is called, it will resolve again. The object itself, though, remains unity ignorant... it only knows to call the delegate. It has no intimate knowledge of what the delegate does."; }
        }
    }
}


NOODB: Query Parsers

April 9, 2010

The conversion from the query to sql is a two step process.

Step 1:  Parse the query into an object graph. This is so that multiple implementations of the repository don’t all have to figure out how to do this; Noodb converts the query to a common format, and each repository can do as it needs.

Step 2: Convert the common graph into Sql.

Step 2 is easier. It just has to walk the graph and output sql as it goes. No big deal for the simple queries I’m starting with.

Step 1 is a lot more difficult. I’m working through the code, and it will work, but I’m not going to be happy with it on this pass. I’m manually parsing the string; there has to be a more elegant way of doing this. Unfortunately, I don’t have any experience to draw on in this case.

At this point, I’m working on parsing single-object queries. I’m only concerned with the where clause With those constraint in mind, step 2 is working.

Here’s a piece of test code:

Query simpleQuery2 = new NoodbQueryParser().Parse("from p in QueryPerson where p.LastName='Allard' && p.FirstName='Jay' || p.MiddleInitial='M'");
string simpleText2 = new SqlQueryConverter().Convert(simpleQuery2);

The resulting text:

(
	[p].[LastName] = 'Allard' 
	And [p].[FirstName] = 'Jay' 
	Or [p].[MiddleInitial] = 'M'
)

Pretty simple. Again, Step 1 isn’t complete, it only does the simple stuff. But, if you skip step 1 (which convers the text to a query object), and just create the query object yourself, then you see that step 2 is more mature.

(
	[p].[LastName] = 'Allard'
	Or 
	(
		[p].[FirstName] = 'Santa'
		And [p].[LastName] = 'Claus'
	)	
	And [p].[random] = 'blah'
)

The sql shown is the actual generated sql; I’m not reformatting it. It’s generated with the tabbing to make it more readable. If that proves to be detrimental to performace, I’ll make the formatting optional.