Data Persistence Models in Docker Containers

A container has different layers starting with Minimal Subset of OS topped by Container Filesystem topped by Application layer topped by Hosting layer. All these layers are read-only.

There is a top layer called Container Runtime layer which will be in a Read/Write state. 

The data on Container Runtime layer is persistent only when the container is stopped/started . If a continer is deleted, this data will be lost forever. Also this data is isolated only to that continer and cannot be shared with other containers.

So lets look at better data persistance models to share data between different containers on a Host.

Volume and Bind Mount are two ways of persistent data storage thats avilable on a Host, which can be accessed (read/write) by multiple containers.

Volume is the storage created and managed by Docker. This means no containers can go beyond the boundaries of docker while working with volumes.

Bind Mount is the storage directly from file system of the host file. So if there is a malicious code deployed in a container, it can break the host by manipulating the host filesystem.

By now looking at color coding you should have understood that Volume is a better/safer way of storing and sharing data between containers. Let me show you by a demonstration.

Containerize ASP.Net Core app on Azure Kubernetes Cluster

In my earlier post, we have deployed ASP.Net Core application to a Container hosted by a Linux Server.

There are some problems with this approach.

1. What if the Host VM is stopped?
2. What if Container Instance is stopped?
3. How do we manage the deployment of any app changes?
4. Even when we stop Host VM, you still be paying for the Disk allocated. How we can avoid that?

This is where Azure Kubernetes comes into the picture. 

Azure Kubernetes provide serverless CI/CD experience which also manages Health, Security, Auto-Scaling, Deployment and Governance aspects. More details can be found here. 

In this article we will be:

• Deploy a ASP.Net Core App to a Container
• Create a Container Image from .Net Core Container
• Push the Image to Azure Container Registry
• Use Kubernetes to Pull that image and create mutliple instances of the container in Kubernete Pods.
• Expose the .Net Core App via Azure Load-Balancer.

Containerizing ASP.Net Core Application on a Linux Host

In this article we will see how to deploy a ASP.Net Core 3.1 Webapplication to a container hosted on a Linux Host.

Beofore we jump into implementation, we need to look at every component and know what it does.

1. Host: This can be a Linux or Windows server. Considering Linux servers have used from long time for contanerization, i picked it. But Windows is very close choice considering all the capabilities it acquired in last 2 years.

2. Docker: It is the engine we use to host our containers. It will package the application with its dependecies and run them in isolated containers on the host.

3 Reverse Proxy Server: This will be sitting behind the firewall and redirects the user requests to appropriate apps/containers. This will provide extra layer of abstraction and reduces the public exposure of the containers. 

4. Kestrel Web Server: When you install a .Net Core SDK, it will internally create a webserver to act as a backend server for the .Net Application. Proxy server will be sending the client calls to Kestrel Web server and responds back with the output from .Net App.

5. App: .Net Core Application we deploy to be containerized. 

Azure Service Bus - Queues and Topics - Part 2

Use Case: Organization X has a large-scale distributed serverless application environment. There are some azure components that sends a private messages which need to be accessed only once by other components using RoleBasedAccessControls. There are some messages that need to Fan-Out(one-to-many) to large number of systems, where sender doesnt need to know the receiver's details. The message should stay until time out, for receiving components to consume when they are avilable.

In the earlier article, we have seen how we can use Azure Service Bus Queues for integration of serverless application environment. Now we will explore Service Bus Topic and how it will be used.

We have already created Azure service Bus in earlier article, you can reffer the link provoided at the top for details.

Azure Service Bus - Queues and Topics - Part 1

Use Case: Organization X has a large-scale distributed serverless application environment. There are some azure components that sends a private messages which need to be accessed only once by other components using RoleBasedAccessControls. There are some messages that need to Fan-Out(one-to-many) to large number of systems, where sender doesnt need to know the receiver's details. The message should stay until time out, for receiving components to consume when they are avilable.

In this article, we will learn what is Azure Service Bus, Service Bus Queues, Service Bus Topics, How to access them programatically using .Net. Once you do it practically following my instrctions, you will understand the theory behind it. So follow me.

Why Azure Service Bus?

Service Bus is a message broker which handles Integrity, Security, Communication protocols of Messages at Enterprise level.

Azure Messaging Models

 All of this is straight from Microsoft Documentation. I am just grouping them for better reference. 

Every time i say, "This is what Microsoft says, and then i validate the theory with a practicle implementation". This time, this article is full of thoery, so this is all about what Microsoft says. I take no credit for this infromation.

Azure Durable Functions - Starter, Orchestrator and Activity Fucntions

Use Case: A user starts to book a trip, which consists booking a Flight and Hotel and then pay for complete trip. 

Back in the day, the answer used to be a single server-side application which is complex enough to take care of everything. Now we call it Monolith application. 

Now we use microservices architecture to seperate the concerns at domain level. With Function as a Service, it even evolved to a Serverless state, where you can provision the services even without using any infrastructure.

Azure Functions is one of the critical components of serverless designs, and here is what we know about them:

Even-Driven: Azure functions are invoked based on a trigger caused by events like making a http call (or) adding a message to a Queue (or) adding a object to a blob storage (or) adding a new record to CosmosDB and many others.

Short-Lived: Azure functions are meant to for short period of excutions and to address a simple tasks which makes it more suitable for Microservices part of the design. Keep it simple and seperated.

Stateless: Azure Functions doesnt save any state infirmation of objects, thus the trigger mechanism will have both inputs and outputs. 

Now considering above facts, i have created a design for above Use Case.

Now the deisgn is pretty much simple and self-explanatory. But here is the catch.

Azure Functions Basics - Part 2

In earlier post, we have built the green part of below app. Part-1

Now lets continue building red part of the design.

Azure Functions Basics - Part 1

Azure Function: Azure Functions is a serverless application platform. It allows developers to host business logic that can be executed without provisioning infrastructure. Functions provides intrinsic scalability and you are charged only for the resources used. You can write your function code in the language of your choice, including C#, F#, JavaScript, Python, and PowerShell Core. Support for package managers like NuGet and NPM is also included, so you can use popular libraries in your business logic.

Says Microsoft, now we will build some simple stuff shown below and understand basics while we do so.

Azure Policy and Compliance Management

Azure Policy helps to enforce organizational standards and to assess compliance at-scale. Through its compliance dashboard, it provides an aggregated view to evaluate the overall state of the environment, with the ability to drill down to the per-resource, per-policy granularity.

Thats what Microsoft documentation says.  Now let me say in my way.

I work in a bank and the regulatory compliance says, data of our bank shouldn't leave Autralia.

So if one of our developers deployed a production azure resource in any other AZ Regions, our organization has to pay big penalties for not complying with regulations.

How do we do it? Using Azure Policy Assignment.

Before we jump on to implemntation, you need to know about difference between Policy and Initiative.

To keep it simple, Policy is a rule and Initiave is a collcetion of policies which set a standard.

Eg: ISO(standard) is a Intiative, which comprises of  many policies which makes that standard.