Swapping out SignalR for RabbitMQ

November 26, 2014

Out of the box, the app server uses signalr for 2 things:

  • The bus – it is how all nodes communicate with each other
  • The website – it is how the website communicates with the bus

Since the beginning, I have always said that “anything in the app server can be swapped out… theoretically”. I always added the “theoretically” disclaimer because I hadn’t actually tried it.

Finally, I tried it with RabbitMQ, and the theory was incorrect. I had to make some changes, and chose to make some others.

The primarily required change: the two things done by SignalR were handled by a single class (with a TODO to break it up.) If you switch the bus to RabbitMQ, then you still need SignalR for the website. Managing those 2 things as one doesn’t work. So, per the TODO, I broke it up.

Secondary required change: the battle for flexibility and simplicity continued. There are server level features and application level features, and they all need to be easily managed via code. I want everything to work out of the box, and then also be modifiable either via configuration on the hub, or via code overrides. The real trick from the code perspective is that the applications are, essentially, remote. Any changes you make need to be made to the server then propagated to the app. I spent a lot of time working on that, and it’s now in a good state. (Ironically, for the purposes of RabbitMQ, I don’t even need that yet. But, it’s done. You can manage application features and server features from the server api.)

Chosen change: On the hub, the SignalRBusHub being hosted as a feature in an application.  It was quick and dirty. The feature simply subscribed to the bus then forwarded messages to SignalR. On the application, though, the SignalR connectivity was built into the bus object itself as a subclass. So, that was inconsistent. I changed the hub level bus to also be a subclass hosted by the server. That cleaned it up.

Chosen change: The SignalR code was all within it’s feature. It wasn’t reusable. I changed that so that SignalR hub hosting is a core feature that other features can opt into. That was already implemented for WebAPI and LocalScheduler, but the ability did not exist when I wrote the SignalR code. The implementation of it was very similar to WebAPI. That’s not because they are both ASP.NET technologies (that’s just coincidence), but because they both involve features registering with other features. The WebApi Feature keeps track of how many other features require it, and manages itself based on that. Anything that uses the OPT-IN approach will need this. I took most of the webapi feature code and moved it up to base classes, and implementation of the signalr feature became pretty insignificant. To register another feature with the signalr feature is just an attribute.

image

The SignaRHubHost attribute says that “when this feature is started, start the BusHub SignalR hub.”

Finally, I had to write the RabbitMQ code. There’s a lot of work to be done on that, but it’s working. (IE: a bus message can expire after a few seconds. Currently, it doesn’t.) At the moment, it’s the bare minimum.

Now, you can simply say UseRabbitMqBus(), and signalr is swapped out for RabbitMQ. (You are reminded that this is a test console app, so don’t pay attention to the static fields, etc.)

image

And here is the implementation of it.

image

The funny thing about that is that it’s specifically disabling SIGNALR. What if you also have a third implementation? To solve that I will introduce a FeatureRole, and then allow for disabling of the entire role. Then you’d be able to do: server.DisableFeaturesOfRole(“BusHost”), or similar.

For SignalR, I wrote the code to receive messages and delegate to clients. For RabbitMQ, I don’t need to do that because RabbitMQ does all of that automatically. So, there isn’t a server component running in the app server for RabbitMQ. It’s just the client bus.

The following screenshot is the admin portal as one of the demo features starts. The feature fires a bunch of useless child activities just to show how features can send messages that are associated to parent activities. In this case, during the start activity it creates a child activity that fires a bunch of messages for no good reason. I was pleased to see that, as expected, the message numbers are all in order. You do not get guaranteed message order, or guaranteed delivery, with SignalR. They are mostly in order, but there are always a few that are out of sorts. With RabbitMQ, they are in order every time.

image

I’ve put a lot of time into this over the last few days. I’m really happy to have RabbitMQ working via a single method call.

This is a good update. Now that RabbitMQ is in the mix, I can start working on the service layer. The service layer will need an out-of-the-box default for when you’re not using RabbitMQ. Usually I do out-of-the box first, but for this I will use RabbitMQ first because I’m looking forward to diving into it. But first I need some sleep.


Would you like to write some code?

November 20, 2014

I’d really love some help if anyone has some free coding time. I have specific things that need to be done. Some big, some small. Some concrete, some vague and requiring research.

If you’d like to participate in this project that will almost certainly amount to nothing (love the enthusiasm), then please ping me.

One of the smaller/concrete things is to rewrite my TypeCache object with a better fluent interface. We can then wrap that up as NUGET package. The TypeCache needs a better name. What it really does is serve as a library for convention based programming. Currently, it’s mostly attribute based, but we can expand it to account for type names too.

The app server uses it very frequently to find mostly classes and some methods based on their metadata and types.


Creating and Starting an Application Server

November 20, 2014

The app server gets everything it needs from an IOC container. The IOC container is configured via extension methods, and may also be tweaked manually if necessary.

I have a console app that I use to host a hub and two nodes. This is my primary tool for testing. The console application itself isn’t important, so disregard the use of static fields, etc.

image

  • new UnityContainer() – I choose unity. Theoretically, you can use any container you want by writing an IDependencyResolver adapter for it. But, that is untested.’’
  • ToDependencyResolver() – that’s the unity adapter of IDependencyResolver
  • AsHub/AsNode – sets up the container with all of the default stuff it needs for each role. For a node, you pass it the server id object. For hub, it will create one on it’s own as the name is known to be HUB.
  • CreateServer – instantiates a server. This is very simple. It’s just new Server(serverId)
  • Start – an awaitable method that starts the server
  • Result – the server

AsHub/AsNode can be combined with CreateServer to do both steps at once. But, then you’d lose tweaking capabilities. Once you initialize it AsNode, you can then change whatever you want in the container before creating the server.

I will likely add a method such as CreateNodeServer() that combines both steps if you don’t require further configuration.

There are two other methods:

  • AsApplication – this is used for the applications hosted within the application server. This is done by the BRIDGE that links the app server to the applications.
  • AsStandAlone – with this configuration, it doesn’t use a hub. It gets it’s configuration, etc, locally. This worked when I wrote it, but i haven’t used it in several months.

Remote Installer, Round 2

November 14, 2014

A key feature of the app server is the ability to install it on a remote machine via the admin website. You fill in a form, hit submit, and the server will be created on a remote machine.

Information required:

  • Installer credentials: the user that will run the install
  • Service credentials: the user the service will run as
  • Remote machine name: where to install to
  • Drive letter: which drive to install it to. (So far, you can’t select the folder, just the drive. The folder is determined by convention.)
  • Serve Type: optional. Select how the server should configure itself.
  • The name of the application server. (There can be multiple per machine, but the name must be unique across the network.)

The hub runs in IIS, which is why we need 2 sets of credentials. IIS isn’t an admin user and doesn’t (or shouldn’t) have enough access to the other machines to install software.

The steps of the install are:

  1. Deploy the files to the remote machine
  2. Grant LogonAsService right to the service user on the remote machine
  3. Create the service on the remote machine
  4. Start the service on the remote machine

When I created the installer, I was still in feature demo mode, and just wanted to get it to work. I thought the quickest way to do that would be to generate a batch file and execute it. Process.Start allows you to pass credentials, so I thought i would use that to execute the batch file as the installer user.

It was a long time ago. I don’t remember the details of all that went wrong, but it turned into a disaster. I ended up with 2 batch files: one to run on the hub, and another to run on the remote machine. I had to use PSEXEC to run those batch files, one locally, one remotely.

After much shenanigans, it worked, but it wasn’t the quick fix I was looking for, and also it was really ugly. But, it worked, which is all that I was after at the time.

The reason I went through that is because I didn’t want to deal with writing all of the impersonation code that would allow me to execute C# code as a different user.

Finally, late last week, I circled back on that and settled in to deal with the impersonation. I properly nourished myself and made mental preparations. Then, all of a sudden, I realized it’s 2014, and we have at our disposal a little thing called NUGET. I quickly found and added the SIMPLE IMPERSONATION library, and there we go… I had impersonation.

And it works great. I haven’t had to play with anything. I’m fully confident that I could write it myself, but I would’ve tripped along the way. It’s nice to have it just done and work. So thanks for that.

Now, the installer is much simpler. I deleted a lot of code.

  • The file deploy is done via C# code. There is a configurable list of files. It iterates the list and copies the files.
  • Grant LogonAsService is accomplished by executing NTRIGHTS.EXE from code. NTRIGHTS.exe can run against a remote machine.
  • CREATE and START the service are accomplished by executing SC from code. SC also works against remote machines.

I’m using PROCESS.START three times. As it evolves, it would be nice to eliminate that and do it all in code, but it’s MUCH better now. I’m able to collect the error output and standard output and save it to the response object.

I haven’t done this yet, but now that it’s back in code, I can use the bus to publish messages as the install proceeds. Those messages will be received by the installer page, which will display the messages to let you know what’s happening as it happens. (Today, it just sits there until the install is complete. It’s boring.)

Finally, this change offered another huge benefit: It’s so much faster. PSEXEC took a long time to run, and I ran it multiple times. It would take 20 or 30 seconds to install. Now, it takes less than 300 milliseconds. That sounds ridiculous, so I confirmed it a few times.

So far, it’s only working as a unit test.  You can kick it off from the page, but you won’t get any feedback just yet.

I have to make one more change. Previously, you would install the app server, then set the server type as a second step. I recently changed that so you that you can, optionally, specify the server type on the setup page. But, I cheated and had the controller set the server type before calling the installer. It’s a hack. The installer needs to be self contained and do it all itself.

image

Here I use my super awesome REALLY pattern. The code that calls this method does error handling and impersonation. Impersonation is only done if demanded by the installer request. That method is pretty ugly, so I’m not including it in order to protect my self esteem.

image

This method is called three times: NTRIGHTS, SC CREATE, SC START.

image

The installer response was previously specific to the batch file implementation, and was marked as a todo. This one is more generic. You add a series of messages to the response, and the message indicates “OK”, or “PROBLEM”, and any user details.

This information will ultimately be published and, possibly, recorded.

Currently, I’m handing uninstalls manually as needed. But, that will have to be coded, too, at some point.


Application Server: Hosting Services

November 7, 2014

I am at QCON SF this week, an like last year, there is a whole lot of talk about microservices.

The application server supports Http services via WEB API. It’s a minimal implementation at the moment. If you associate your feature to an ApiController, then when the feature is started, the ApiController becomes available.

    [WebApiHost(typeof(YabbaDabbaDooController))]
    public class DemoFeature1 : Feature
    {
			public Task OnStart() { .... }
       public Task OnStop() { .... }
    }

The, YabbaDabbaDooController is just an ApIController.

public class YabbaDabbaDooController : ApiController
    {
        [Route("date")]
        public DateTime GetNow()
        {
            return DateTime.Now;
        }
    }

 

Now, the ApiController is bound to the feature. If the feature is started, you can reach the controller. If the feature is stopped, it will respond with a 404 (if any other controllers are started), or it won’t generate a response at all (if the webapi feature isn’t running because nothing needs it.)

The thought process behind that is that there may be several things related to a feature. Scheduled jobs, web api endpoints, custom code, etc. The feature is the in-app container that manages that stuff as a unit.

But, what if you only want to do one thing, and that one thing is host an API Controller? Then you have to create the feature, create the controller, then tie them together. That’s now awful, but you’ll have 2 classes for every service. And, when thinking about microservices and the possibility that there’s going to be many of them, it can be done easier.

My brain has been churning non-stop thinking about this. I had no intention on working on it, but this morning it got the better of me and I accidentally woke up 2.5 hours earlier than I normally would. So, I figured I might as well get to work on it.

I had to make fundamental change to features, which worked out better all around. To implement a feature, you subclass the Feature class, then provide Start and Stop methods, even if you don’t need them. The feature manager uses the TypeCache and DependencyResolver to find and build features based on the understanding that they are all of type Feature. Then, there are attributes on the feature to indicate what type of feature it may be. Currently, Core, Server, Application.

What I want to be able to do is this:

    [HttpFeature]
    public class HttpServiceDemo1 : ApiController
    {
        [Route("date")]
        public DateTime GetNow()
        {
            return DateTime.Now;
        }
    }

It’s a single class with a single attribute. It is the feature and the functionality associated to the feature. Of course, it can’t inherit from ApiController and Feature, and I didn’t want to do a bridge, so the base class had to go. I could have switched it to an interface, but then if I would feel obligated to put the Start and Stop methods on it, as I am not a fan of marker interfaces. And, if Start and Stop are on the interface, then you’d have to implement them even if you don’t need them. Now that I have been doing this a while, I find that often they are not needed, such as in HttpServiceDemo1 class.

So, it’s based on the attribute. The feature manager looks for all types that have an attribute that inherits from FeatureAttribute. Depending on where the feature manager is (primary app domain, or child application), it looks for either ServerFeatureAttribute or ApplicationFeatureAttribute (from which Core and Http descend).

If you need a start method, then add a “void Start()” method. If you need a stop method, then add a “void Stop()”  method. The methods will be found regardless of their access modifier.

Support of all this required some changes. The typecache/dependency resolver stuff had to be changed to work off the attribute rather than the type or type name.

  • The TypeCache had to be updated to return descendants of the requested attribute. Previously, when I said “give me all types that have the ServerFeatureType attribute”, it would only return those that had exactly that attribute, not any of it’s subclasses.
  • I added a method to the type cache to retrieve a type of a particular name that had the requested attribute.
  • The code that calls the OnStart and OnStop methods has been changed to use reflection. The method names have been renamed to just Start and Stop. Previously, these were awaitable methods. That has been lost during this change, but can be put back if needed. But, the feature manager was just waiting on them anway, so no loss. (Because it’s now using reflection, multiple Start/Stop signatures can be supported.)

The first success was eliminating the feature class and get everything working as it did before. But, it wasn’t done quite yet.

The Feature manager would recognize the type as a feature because it has a Feature attribute. But, how does it figure out that that the feature needs to be associated to another feature without another attribute? In this case, we want the HttpService1Demo class, which is an API Controller and also a Feature, to be associated to the WebApiHostFeature, which is what actually hosts the controller. Previously, that was done by adding a [WebApiHost] attribute to the feature, which we can still do, but the point of this exercise was to eliminate as much as possible.

Thus, the HttpFeatureAttribute class was made.

    public class HttpFeatureAttribute : ApplicationFeatureAttribute
    {
        public HttpFeatureAttribute(StartMode startMode = StartMode.Started)
            : base(startMode)
        {
        }

        public override IEnumerable<Attribute> GetOtherAttributes(object attributeOwner)
        {
            if (!attributeOwner.GetType().IsSubclassOf(typeof(ApiController)))
            {
                // TODO:
                Debug.Fail("need to be an api controller");
                return null;
            }
            return new List<Attribute> { new WebApiHostAttribute(attributeOwner.GetType()) };
        }
    }

The GeOtherAttributes method is new. It allows the attribute to accept an instance of a class, and return additional attributes for that class. In this case, it’s returning the WebApiHostAttribute.

The RegisterFeatureWithOtherFeatures method of the feature manager is responsible for the child to parent relationships. I added the second block of code to pull in the additional attributes that are provided by the Feature attribute.

            // this is a super hack. Consolidate with the method version.
            // get the attribute assigned directly to the class.
            var classAttributes =
                Attribute
                    .GetCustomAttributes(state.Feature.GetType(), typeof(DependentFeatureAttribute), true)
                    .OfType<DependentFeatureAttribute>();

            // the feature type attribute of the feature may indicate that additional attributes are needed.
            // IE: the HttpFeatureAttribute returns a WebApiHostAttribute so that features of that type are automatically associated
            // to the web api feature, without requiring a hand-added attribute.
            var featureTypeAttribute = (FeatureTypeAttribute)Attribute.GetCustomAttribute(state.Feature.GetType(), typeof(FeatureTypeAttribute));
            var other = featureTypeAttribute.GetOtherAttributes(state.Feature);
            if (other != null)
            {
                classAttributes = classAttributes.Union(other.OfType<DependentFeatureAttribute>());
            }

And that’s it! Now, you can simply put an HttpFeature attribute on an ApiController, and it is a self-contained feature and feature implementation. Of course, you could still associate the feature to other sorts of things if you want, but the point of this was to quickly stand up a service, as a feature, when you don’t really need anything else.

image

Because HttpServiceDemo1 relies on WebApiHost, you cannot stop WebApiHost unless HttpService1 (and any other WebApi dependent feature) are stopped first. You can hit the stop button, but it will respond and tell you that you can’t because of the dependencies.

If WebApiHost is stopped, and you start a feature that requires it, then it will start automatically.

One thing that i didn’t do, which I only just now realized looking at the screen shot, is give the new feature attribute it’s own group id. The UI orders thing by Group id, which is why all of the core features are always at the bottom of the list and in blue. I can set the group id for the new HttpFeature attribute and have them all grouped together too, and add it to the CSS so that they all show up as muave or something else distasteful. It’s all easy stuff.

In conclusion, this all went quick and easy, and Features are now simpler and cleaner. The thing that started it all was the idea of being able to stand up a service real quick, and that has been achieved. You may use it for a microservice, or any other type of service you like.

If you like the old way of doing things, then you can do that too. The old way is better when your feature does many things. The new way is better if you just want to host a service.