F#: Load Balancing Remote-deployed Actors (Akka.Net)

Intro

This document is meant to document my understanding of load balancing remote-deployed actors using the documented OOP support for the Akka.Net framework.

Setup

My solution had the following projects:

  • System1
  • System2
  • Shared

System1

The System1 project is an executable that remote-deploys actors to a separate process.

The following reflects how remote actors get deployed onto a separate process:


use system = ActorSystem.Create("system1", config)

let reply = system.ActorOf<ReplyActor>("reply")

let props1 = Props.Create(typeof<SomeActor>, [||])

let props2 = Props.Create(typeof<SomeActor>, [||])

let props3 = Props.Create(typeof<SomeActor>, [||])

let remote1 = system.ActorOf(props1.WithRouter(FromConfig.Instance), "remoteactor1")

let remote2 = system.ActorOf(props2.WithRouter(FromConfig.Instance), "remoteactor2")

let remote3 = system.ActorOf(props3.WithRouter(FromConfig.Instance), "remoteactor3")

In order for this code to execute successfully, the following configuration is required:

        actor {
            provider = ""Akka.Remote.RemoteActorRefProvider, Akka.Remote""

            debug {
              receive = on
              autoreceive = on
              lifecycle = on
              event-stream = on
              unhandled = on
            }

            deployment {
                /localactor {
                    router = consistent-hashing-pool
                    nr-of-instances = 5
                    virtual-nodes-factor = 10
                }
                /remoteactor1 {
                    router = consistent-hashing-pool
                    nr-of-instances = 5
                    remote = ""akka.tcp://system2@localhost:8080""
                }
                /remoteactor2 {
                    router = consistent-hashing-pool
                    nr-of-instances = 5
                    remote = ""akka.tcp://system2@localhost:8080""
                }
                /remoteactor3 {
                    router = consistent-hashing-pool
                    nr-of-instances = 5
                    remote = ""akka.tcp://system2@localhost:8080""
                }
            }

After the actors get remote-deployed to a separate process, they can then receive messages. Akka.Net supports load balancing for distributing messages amongst actors. To add load balancing support for the actors, Akka.Net provides router types. This example uses a group-router type called ConsistentHashingGroup. The ConsistentHashingGroup router relies on a hash key to route messages to corresponding actors. The configuration for an actor needs to provide this information as it does in the configuration example above.

Once the configuration is specified like in the example above, a ConsistentHashingGroup instance can be created and configured (specifying the actors):

let hashGroup = system.ActorOf(Props.Empty.WithRouter(ConsistentHashingGroup(config)))
Task.Delay(500).Wait();

let routee1 = Routee.FromActorRef(remote1);
hashGroup.Tell(new AddRoutee(routee1));

            let routee2 = Routee.FromActorRef(remote2);
hashGroup.Tell(new AddRoutee(routee2));

let routee3 = Routee.FromActorRef(remote3);
hashGroup.Tell(new AddRoutee(routee3));

Once we have a message type defined that implements IConsistentHashable, we can send messages:

Task.Delay(500).Wait();

for i = 0 to 5 do
    for j = 0 to 7 do

        let message = new HashMessage(j, sprintf "remote message: %i" j);
                    hashGroup.Tell(message, reply);

System2

System2 represents the process that we want to ultimately deploy our actors to even though we’re on a different process and/or machine. The reason for this is to provide support fault-tolerance and load balancing.

The following code was written for the System2 executable:

open Akka.Configuration
open Akka.Actor
open System

[<Literal>]
let EndWithSuccess = 0

[<EntryPoint>]
let main argv = 

    let config = ConfigurationFactory.ParseString(@"
        akka {
            log-config-on-start = on
            stdout-loglevel = DEBUG
            loglevel = DEBUG
            actor {
                provider = ""Akka.Remote.RemoteActorRefProvider, Akka.Remote""

                debug {
                  receive = on
                  autoreceive = on
                  lifecycle = on
                  event-stream = on
                  unhandled = on
                }
            }
            remote {
                helios.tcp {
		            port = 8080
		            hostname = localhost
                }
            }
        }
        ")

    use system = ActorSystem.Create("system2", config)
    Console.ReadLine() |> ignore

    EndWithSuccess

Shared

Both System1 and System2 reference the same actor type and message type. As a result, in order to remote deploy an actor and have that actor receive messages on a separate process, then we need to ensure that the separate process knows how to construct an instance of that actor as well as the message types that it subscribes to. To do this, we need to provide a shared assembly that harbors the actor type and message type.

The following actor type was added to a shared project for both executables (i.e. processes to reference):

type SomeActor() =

    inherit UntypedActor()

        override this.OnReceive(msg:obj) =

            match msg with
            | 😕 HashMessage as m ->
                let address = this.Self.Path.ToStringWithAddress()
                let content = (sprintf "%s got %s" address m.Content)
                Console.WriteLine(content);
            | _ -> ()

            let sender = UntypedActor.Context.Sender
            let content = (sprintf "Message from %A - %s" sender.Path, msg)
            System.Console.WriteLine(content);

The following message type was added to a shared project for both executables (i.e. processes to reference):

open Akka.Routing
open System

type HashMessage(id:int, content:string) =

    interface IConsistentHashable with

        member this.ConsistentHashKey:obj = box id

    member val Content  = content

Output

The output below reflects the load-balancing of actors based on the message key:

akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 0
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 1
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor1/$e got remote message: 3
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 4
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 5
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor1/$e got remote message: 3
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor1/$e got remote message: 3
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor1/$e got remote message: 3
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor1/$e got remote message: 3
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor1/$e got remote message: 3
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 0
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 4
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 5
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 0
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 4
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 5
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 0
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 4
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 5
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 0
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 4
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 5
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 0
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 4
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor3/$d got remote message: 5
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 2
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 6
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 7
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 1
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 2
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 6
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 7
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 1
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 2
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 6
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 7
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 1
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 2
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 6
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 7
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 1)
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 2
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 6
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 7
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 1
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 2
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 6
akka://system2/remote/akka.tcp/system1@localhost:8090/user/remoteactor2/$b got remote message: 7

Conclusion

In conclusion, I attempted to document my understanding of load balancing remote-deployed actors using the documented OOP support for the Akka.Net framework. Successful deployment of actors onto a separate process and/or machine requires configuration, a router, and a shared reference to the actor and message type. Once these are established, messages can be routed and processed.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: