On May 24th, 2017 the project kickoff for Nikeza was live streamed. The kickoff involved several motivated developers that I’d met during my campaign to learn functional programming. I attempted to assemble these developers into a team so that I could not only advance my campaign to learn Functional Programming, but also to build an application by applying what I’d learned.
I was already familiar with F#. Hence, I had performed several code katas and had even built some sample mobile applications with it using Xamarin tools. However, Xamarin relied on Object Oriented Programming which conflicted with my campaign to learn Functional Programming. Thus, I wanted to build an actual application by applying the lessons I’d learned from Functional Programming. In this article, I will discuss the server that I built using F#.
Here’s a screenshot of the app:
The requirements for the application were the following:
- Browse experts
- Subscribe to an expert’s activity feed
- Access expert’s content (via topic)
- Set featured links
- Set featured topics
- Set data sources (i.e. YouTube, WordPress, StackOverflow, etc.)
- Periodically checks for new content from an expert’s data source collection
Based on the requirements defined above, this application was a typical CRUD app. In addition, this app wasn’t mission critical, didn’t handle money, and did not have long running workflows. In other words, there was no need for audit controls. Therefore, I didn’t have to entertain an architecture dependent on sagas or event sourcing. However, decisions were made for the app to be future proof.
For business and marketing reasons, the team decided that Nikeza should be a .Net Core application. Hence, we weren’t sure if the .Net framework was going to be considered legacy in the next five years. However, we soon realized that the combination of using .Net Core with F# would likely result in not-so-spectacular code at the Data Access Layer of the system. Hence, the SQL TypeProvider wasn’t yet ready for .Net Core when we decided to build Nikeza. We also considered Entity Framework Core as an alternative to using the Sql TypeProvider. Unfortunately, we learned that Entity Framework Core didn’t provide 1st-class support for F# either. As a result, we surrendered our ambitions and resorted to using ADO.Net.
The overall architecture looks something like this:
The following commands support user requirements:
type Command = | UpdateProfile of ProfileRequest | UpdateThumbnail of UpdateThumbnailRequest | Follow of FollowRequest | Unsubscribe of UnsubscribeRequest | AddLink of Link | RemoveLink of RemoveLinkRequest | FeatureLink of FeatureLinkRequest | ObserveLinks of ObservedLinks | UpdateTopics of FeaturedTopicsRequest | AddSource of DataSourceRequest | RemoveSource of RemoveDataSourceRequest | SyncSource of DataSourceRequest let execute = function | UpdateProfile info -> info |> updateProfile | UpdateThumbnail info -> info |> updateThumbnail | Follow info -> info |> follow | Unsubscribe info -> info |> unsubscribe | AddLink info -> info |> addLink | RemoveLink info -> info |> removeLink | FeatureLink info -> info |> featureLink | ObserveLinks info -> info |> observeLinks | UpdateTopics info -> info |> featureTopics | AddSource info -> info |> addDataSource | RemoveSource info -> info |> removeDataSource | SyncSource info -> info |> syncDataSource
Querying data followed a different approach compared to the command approach I used. Hence, instead of leveraging a choice type (aka: Discriminated Union), I instead wrote arbitrary functions. In hindsight and for the future reference, I would stay consistent with the command approach that I used and leverage a discriminated union (aka: Choice Type) to model queries as I did commands.
The following is a high-level example of how I would refactor my Store module:
type Query = | LinksByProvider of ProfileId | LinksLatestByProvider of ProfileId | LinksByProviderPlatform of ProfileId * Platform | Source of SourceId | SourceByProvider of ProfileId | SourceQuery of SourceId * Sql | SourcesAll | Profile of ProfileId | ProfileByEmail of Email | ProfilesQuery of ProfileId * Sql | Topics of ProfileId let query = function | LinksLatestByProvider info -> info |> infolinksLatestByProvider | LinksByProvider info -> info |> infolinksByProvider | LinksByProviderPlatform info -> info |> infolinksByProviderPlatform | Source info -> info |> source | SourceByProvider info -> info |> sourceByProvider | SourceByQuery info -> info |> sourceByQuery | SourcesAll info -> info |> sourcesAll | Profile info -> info |> getProfile | ProfileByEmail info -> info |> profileByEmail | ProfilesQuery info -> info |> queryProfile | Topics info -> info |> getTopics
I believe the example above is more expressive and maintainable compared to my current implementation (that I rather not show). Each query option that the server is required to support is explicit and has a function that’s mapped to it. Hence, my original implementation has arbitrary functions littered throughout the module with no meta-data on how they’re mapped to requirements. Contrary to my actual implementation, the example above has each requirement represented as a choice for executing a query. Thus, the query function above would be the main interface for querying data.
The application that I wanted to build not only needed to provide users with the capability to access content from experts, but it also needed to discover the latest content from experts once their platforms and corresponding access ids were saved into the system. This meant that an expert should only be required to point Nikeza to the platforms that they produce content on. Nikeza would then check for new content periodically on those platforms and update subscribers accordingly.
To support this requirement, I wrote a Platforms module that served as an interface for platform types.
The following components were arranged to support content discovery:
Here’s the abstract code for discovering new content:
getAllSources() |> List.iter (fun s -> s |> syncDataSource |> ignore)
Here’s the implementation details:
let syncDataSource (info:DataSourceRequest) = let notInDatabase link = getLink link.Title |> List.isEmpty getLastSynched info.Id |> function | Some lastSynched -> let newLinks = info |> dataSourceToPlatformUser |> newPlatformLinks lastSynched |> List.filter(fun l -> l |> notInDatabase) let updatedSource = newLinks |> updateSourceRequest info updatedSource.Links |> List.ofSeq |> List.iter (fun link -> link |> addSourceLink updatedSource |> ignore ) updateSyncHistory info.Id |> ignore info.Id |> string | None -> addSyncHistory info.Id |> ignore info.Id |> string
The Platforms module is as follows:
module Nikeza.Server.Platforms open ... let PlatformToString = function | YouTube -> "youtube" | WordPress -> "wordpress" | StackOverflow -> "stackoverflow" | Medium -> "medium" | RSSFeed -> "rss feed" | Other -> "other" let platformFromString (platform:string) = platform.ToLower() |> function | "youtube" -> YouTube | "wordpress" -> WordPress | "stackoverflow" -> StackOverflow | "medium" -> Medium | "rss feed" -> RSSFeed | "other" -> Other | _ -> Other let getKey = function | YouTube -> File.ReadAllText(Path.Combine(Directory.GetCurrentDirectory(),KeyFile_YouTube)) | StackOverflow -> File.ReadAllText(Path.Combine(Directory.GetCurrentDirectory(),KeyFile_StackOverflow)) | WordPress -> KeyNotRequired | Medium -> KeyNotRequired | RSSFeed -> KeyNotRequired | Other -> KeyNotRequired let getThumbnail accessId platform = platform |> function | YouTube -> YouTube .getThumbnail accessId <| getKey platform | StackOverflow -> StackOverflow .getThumbnail accessId <| getKey platform | WordPress -> WordPress .getThumbnail accessId | Medium -> Medium .getThumbnail accessId | RSSFeed -> DefaultThumbnail | Other -> DefaultThumbnail let platformLinks (platformUser:PlatformUser) = let user = platformUser.User platformUser.Platform |> function | YouTube -> platformUser |> youtubeLinks | StackOverflow -> platformUser |> stackoverflowLinks | WordPress -> user |> wordpressLinks | Medium -> user |> mediumLinks | RSSFeed -> user |> rssLinks | Other ->  let newPlatformLinks (lastSynched:DateTime) (platformUser:PlatformUser) = let user = platformUser.User platformUser.Platform |> function | YouTube -> platformUser |> newYoutubeLinks lastSynched | StackOverflow -> platformUser |> newStackoverflowLinks lastSynched | WordPress -> user |> newWordpressLinks lastSynched | Medium -> user |> newMediumLinks lastSynched | RSSFeed -> user |> newRssLinks lastSynched
Each function above is essentially a Strategy pattern that’s dependent on the platform type provided.
The web services were implemented in Giraffe. The decision to use that particular framework was based on the team’s desire to use a lightweight framework that embraced idiomatic F# syntax.
The services are as follows:
let webApp: HttpHandler = choose [ GET >=> choose [ route "/" >=> htmlFile "index.html" route "/options" >=> setHttpHeader "Allow" "GET, OPTIONS, POST" // CORS support routef "/syncsources/%s" syncSources routef "/bootstrap/%s" fetchBootstrap routef "/providers/%s" fetchProviders routef "/links/%s" fetchLinks routef "/suggestedtopics/%s" fetchSuggestedTopics routef "/recent/%s" fetchRecent routef "/followers/%s" fetchFollowers routef "/subscriptions/%s" fetchSubscriptions routef "/sources/%s" fetchSources routef "/thumbnail/%s/%s" fetchThumbnail routef "/provider/%s" fetchProvider routef "/removesource/%s" removeSourceHandler ] POST >=> choose [ route "/register" >=> registrationHandler route "/login" >=> loginHandler route "/logout" >=> signOff AuthScheme >=> text "logged out" route "/follow" >=> followHandler route "/unsubscribe" >=> unsubscribeHandler route "/featurelink" >=> featureLinkHandler route "/updateprofile" >=> updateProfileHandler route "/updateprovider" >=> updateProviderHandler route "/addsource" >=> addSourceHandler route "/addlink" >=> addLinkHandler route "/removelink" >=> removeLinkHandler route "/updatethumbnail" >=> updateThumbnailHandler route "/featuredtopics" >=> featuredTopicsHandler ] setStatusCode 404 >=> text "Not Found" ]
In conclusion, I wanted to continue my campaign of learning functional programming. To support my learning, I built a web app using Elm and F#. I then discussed why the team (at the time) chose .Net Core, ADO.Net, and Giraffe as the core frameworks to build Nikeza. Lastly, I discussed the app’s architecture as well as some implementation details for commands and queries. I must admit, Nikeza is still a prototype and requires significant work for my vision to be realized. However, I learned a lot working on it.