{
  "$type": "site.standard.document",
  "canonicalUrl": "https://johnnyreilly.com/posts/azure-container-apps-pubsub",
  "description": "This post shows how to build and deploy two Azure Container Apps using Bicep and GitHub Actions which communicate using daprs pubsub building block.",
  "path": "/posts/azure-container-apps-pubsub",
  "publishedAt": "2022-06-21T00:00:00.000Z",
  "site": "at://did:plc:yy3apqjlms24kso7ahn7lbmb/site.standard.publication/3mova7c4nho2b",
  "tags": [
    "azure container apps",
    "bicep"
  ],
  "textContent": "This post shows how to build and deploy two Azure Container Apps using Bicep and GitHub Actions. These apps will communicate using dapr's publish & subscribe (pubsub) building block.\n\n\n\nThis post will build upon code written in a previous post which built and deployed a simple web application to Azure Container Apps using Bicep and GitHub Actions using the GitHub container registry. Behind the scenes, that app was made up of a .NET app and a Node.js app communicating via dapr's service invocation building block.\n\nThere's a good chance you've just googled \"pubsub dapr azure container apps\" and you don't want to read through all this. You just want the code. That's fine. The code for this blogpost is here.\n\nYou got mail: service invocation\n\nRight now we have a:\n\n- Node.js web app and a\n- .NET app\n\nThe web app, when called, uses dapr service invocation to acquire a weather forecast from a .NET app.\n\nWhat we want to investigate is dapr's pubsub building block. But pubsub doesn't really \"fit\" into our current app. Let's alter it. Instead of showing users a weather forecast when they browse to the site, we'll instead look for our users to provide an email address, and we'll mail them a weather forecast.\n\nThis kind of app could work both using dapr service invocation or using pubsub. We're going to implement using our current service invocation approach first. Once that works, we'll then pivot that into using dapr pubsub.\n\nThis isn't rocket surgery; this is playing around with dapr and Azure Container Apps and seeing how they all hang together.\n\n.NET meet mailgun\n\nOur existing .NET app needs the ability to send email. For that we're going to reach for mailgun, and we'll use RestSharp to call it. Let's add RestSharp as a dependency:\n\nWith that in place, let's turn to our WeatherForecastController and make it send an email.\n\nIn our new and improved controller we:\n\n- Switch our GET endpoint to be a POST one instead, to reflect that we're going to take an action (sending an email) each time it's hit. (RESTful to the end)\n- Rather than returning the weather forecast to our caller, we take the email address supplied and we send the weather forecast to it\n\nYou'll also notice we're passing a IOptions<MailConfig> to the constructor of our class, it's in this configuration object we store our Mailgun api key. So we're going to need to define a MailConfig class:\n\nAnd we need to update our Program.cs so it recognises MailConfig and configures it:\n\nThanks to the default setup of .NET, we'll now be able to populate this using appsettings.json files and environment variables. Since our API key is a secret we'll avoid putting it in source control, and instead populate an environment variable that .NET can read:\n\nThe __ above is the convention that .NET follows for nesting with environment variables; this is equivalent to the following structure in an appsettings.json file:\n\nWebservice gets a form\n\nNow that we've tweaked our WeatherService, we need to tweak the web site that calls it. We'll do that by first adding some dependencies that allow our Koa web service to handle routing a little easier:\n\nThen we'll tweak our index.ts like so:\n\nThe above leaves us with a very simple form based web app that sends an email containing weather forecast:\n\nSecrets in Bicep\n\nWhilst we can run locally, we want to be able to deploy this. So we need to update our Bicep template to receive a MAIL__MAILGUNAPIKEY parameter:\n\nWe can see that we treat the MAIL__MAILGUNAPIKEY as a secret. It's passed in using the @secure decorator and it's configured as a secret inside the weatherServiceContainerApp Azure Container App.\n\nWe have a GitHub Action that handles our deployment. We'll need to introduce the MAIL__MAILGUNAPIKEY secret both to the deploy step of the build-and-deploy.yaml:\n\nAnd we'll need to create the associated secret as well:\n\nYou got mail: pubsub!\n\nSo we're now at the point where we have a pubsub style app - but still implemented using the service invocation approach. It's time to start migrating to using dapr's pubsub capabilities. Now, caveat emptor, pivoting from service invocation involves a fair amount of code. I'll try and be as brief as I can as we make the switch. However there will be big ol' lumps of code blocks as we do this. You may find it easier to just examine the finished code. I will in no way feel bad if that's the path you follow.\n\nPublishing with dapr-client\n\nThe first thing we need, is for our Node.js app to publish using the dapr pubsub mechanism. The easiest way to do that is with the dapr-client:\n\nWe then switch out using axios to send our email command, to use dapr-client instead:\n\nThe thing to note above is the client.pubsub.publish; our WebService will now be publishing using pubsub, instead of using axios and service invocation.\n\nSubscribing\n\nOur WeatherService needs to be able to receive what is published. To make that happen, we'll make use of the following NuGet package in WeatherService:\n\nOur Program.cs is adjusted to cater for this:\n\nThe significant pieces above are:\n\nWe'll also need to update our WeatherForecastController.cs:\n\nReally the only new thing here is the Topic attribute on the SendWeatherForecast endpoint:\n\nThis is used (as you might imagine) to route messages.\n\nThe real difference to call out in what we've done so far, is that both our publisher (Node.js) and our subscriber (.NET) have become \"dapr aware\". Although there have been changes in our code to achieve this, they have not been extensive. Noisy, yes. But not big changes.\n\nComponents\n\nIn order to communicate via pubsub, dapr needs some components in place. We'll create a folder in the root of our project named components, and in there create three files:\n\npubsub.yaml\n\nstatestore.yaml\n\nsubscription.yaml\n\nThese three files are fairly self-explanatory. It's worth drawing attention to the following though:\n\n1. We're going to use these components when running locally and so we'll use Redis for our persistence. When we deploy to Azure Container Apps we'll use something more Azure specific.\n2. We're granting access in these components to our node-app (WebService) and our dotnet-app (WeatherService)\n3. We're wiring up our subscription in subscription.yaml- it's this that will be used to route traffic from publishing to subscription.\n\nWith the above in place we're almost ready to be able to run this locally and debug using VS Code. The final tweak is to make our apps aware of the dapr components. This is achieved by adding \"componentsPath\": \"./components\", to the entries in our tasks.json file. In full it looks like this:\n\nWith this in place we're ready to run our apps locally using pubsub. We can publish from the WebService and receive in the WeatherService. This results in the expected email being sent, as we would hope.\n\nBicep\n\nThe missing piece is Azure. How do we deploy this to Azure Container Apps? Well, we have everything we need to do this, save for the Bicep. We need to augment the Bicep we already have to include our Azure Container Apps dapr components. The full template looks like this:\n\nNow this is undeniably a big lump of Bicep. Let's drill into the significant differences:\n\n1. We're creating an Azure storage account.\n2. We're creating an Azure Service Bus and a topic under it named 'weather-forecasts'.\n3. Underneath our managed environment, we're creating a statestore (using the storage account) which is the Azure equivalent of our statestore.yml, but using Azure storage.\n4. Also underneath our managed environment, we're creating a pubsub (using the service bus) which is the Azure equivalent of our pubsub.yml, but using our Azure ServiceBus.\n\nIt's also worth noting that we always have an instance of the services running; minReplicas: 1. This is because when we dial it down to 0, the Weather Service will stop running. Probably there's a fancy KEDA trigger that prevents this; I haven't figured it out.\n\nNo declarative pubsub subscription support\n\nWhilst you might be thinking \"we're home free now!\" - it turns out we're not. Whilst we'd created Azure equivalents of our statestore.yml and pubsub.yml, you'll note there didn't seem to be an equivalent of the subscriptions component in Bicep.\n\nIt turns out support for declarative pub/sub subscriptions is not yet available:\n\n> Known limitations\n> Declarative pub/sub subscriptions\n\nSo whilst we can take the code we have here and run locally, we cannot deploy it to Azure.\n\nHowever, whilst there's no declaritive support for subscriptions, there is programmatic support. It involves more of a pivot in how we put together our code. But since it's the only game in town, we'll give it a go.\n\nYou got mail: programmatic subscriptions!\n\nWe can get rid of our subscriptions.yaml file now - we're going programmatic instead of declarative.\n\nWe're going to replace our WeatherForecastController.cs with a WeatherForecastEndpoints.cs which contains very similar code, but uses the .NET 6 minimal API approach instead: (There appears to be a way to work with MVC but it's not clear how to use it, and it appears to be a more confusing approach than the .NET 6 minimal API approach.)\n\nThe significant thing to note above is the [Topic(\"weather-forecast-pub-sub\", \"weather-forecasts\")] that we're adding to our MapPost in the MapWeatherForecastEndpoints method. This is the equivalent of the subscriptions component that we wanted to create in Bicep but couldn't. This is our programmatic subscription.\n\nWe also need to tweak our Program.cs to cater for the new WeatherForecastEndpoints class:\n\nSo the app.MapWeatherForecastEndpoints(); is what wires up our WeatherForecastEndpoints class.\n\nWith that in place, we're ready to deploy to Azure.\n\nWe now have Azure Container Apps running in Azure, using the dapr pubsub component. Hopefully in future declarative subscribtions will be available also, but for now we can use the programmatic approach.",
  "title": "Azure Container Apps: dapr pubsub"
}