Top 15 Docker Runtime Issues - Reproduce - Resolve

 In this post we will go through top 15 runtime issues you might face in managing docker containers. Re-create those issues on your docker environment , debug step by step and resolve them.

For this i have created a Github repo which got a docker-compose file which will re-create the issue and a README file which will describe what exactly the issue we created, and step by step guide to debug the issue, and how to fix it. We got 15 folders one for each runtime issue.

Github Repo : https://github.com/pratappilaka24/Top-15-Docker-Runtime-Issues

Below are the Runtime issues we will be looking at.

1. OOM (Out-of-Memory) Killer
2. CPU Throttling
3. Disk IO Exhaustion
4. DNS Resolution Failure
5. Volume Permission Denied
6. Port Binding Conflicts
7. Health Check & Restart Loops
8. Docker Socket Exposure
9. Zombie Processes
10. Container Logs Exhaustion
11. Network MTU Mismatch
12. Environment Variable Secret Leakage
13. Timezone Mismatch Inside Containers
14. Open File Limit Exhaustion
15. SELinux Blocking Container Operations

I have a RHEL 10 vm which i am using as host and mounted the shared drive with all the samples on it.

Lets start recreating the runstime issues one by one.

Docker Containers Troubleshooting - Debug , Fix , Run , Scale

Yeah i know, I will ask my Gen AI agent to write , debug, fix my code. 

Thats what many of us think now a days. What if your AI agent dint see some issue before hand and you need to fix its mistakes. What if you need to review the fix before you roll out to prod environment.

Being a developer never stops, may that be working with your old school IDEs writing your own code, or working with your code writen by a Gen AI agent. Always you should know what you are doing.

Just like everyone else i asked Claude to create me a sample Docker compose project and it did a great job.

It started with a simple arhcitecture diagram, explined me what different components are doing.

Then it spit out the code , and in first glance it all looks so perfect. I moved the code to my RHEL10 VM on Virtualbox to test it out.

Data Persistence Models in Docker Containers

A container has different layers starting with Minimal Subset of OS topped by Container Filesystem topped by Application layer topped by Hosting layer. All these layers are read-only.

There is a top layer called Container Runtime layer which will be in a Read/Write state. 

The data on Container Runtime layer is persistent only when the container is stopped/started . If a continer is deleted, this data will be lost forever. Also this data is isolated only to that continer and cannot be shared with other containers.

So lets look at better data persistance models to share data between different containers on a Host.

Volume and Bind Mount are two ways of persistent data storage thats avilable on a Host, which can be accessed (read/write) by multiple containers.

Volume is the storage created and managed by Docker. This means no containers can go beyond the boundaries of docker while working with volumes.

Bind Mount is the storage directly from file system of the host file. So if there is a malicious code deployed in a container, it can break the host by manipulating the host filesystem.

By now looking at color coding you should have understood that Volume is a better/safer way of storing and sharing data between containers. Let me show you by a demonstration.

Containerize ASP.Net Core app on Azure Kubernetes Cluster

In my earlier post, we have deployed ASP.Net Core application to a Container hosted by a Linux Server.

There are some problems with this approach.

1. What if the Host VM is stopped?
2. What if Container Instance is stopped?
3. How do we manage the deployment of any app changes?
4. Even when we stop Host VM, you still be paying for the Disk allocated. How we can avoid that?

This is where Azure Kubernetes comes into the picture. 

Azure Kubernetes provide serverless CI/CD experience which also manages Health, Security, Auto-Scaling, Deployment and Governance aspects. More details can be found here. 

In this article we will be:

• Deploy a ASP.Net Core App to a Container
• Create a Container Image from .Net Core Container
• Push the Image to Azure Container Registry
• Use Kubernetes to Pull that image and create mutliple instances of the container in Kubernete Pods.
• Expose the .Net Core App via Azure Load-Balancer.

Containerizing ASP.Net Core Application on a Linux Host

In this article we will see how to deploy a ASP.Net Core 3.1 Webapplication to a container hosted on a Linux Host.

Beofore we jump into implementation, we need to look at every component and know what it does.

1. Host: This can be a Linux or Windows server. Considering Linux servers have used from long time for contanerization, i picked it. But Windows is very close choice considering all the capabilities it acquired in last 2 years.

2. Docker: It is the engine we use to host our containers. It will package the application with its dependecies and run them in isolated containers on the host.

3 Reverse Proxy Server: This will be sitting behind the firewall and redirects the user requests to appropriate apps/containers. This will provide extra layer of abstraction and reduces the public exposure of the containers. 

4. Kestrel Web Server: When you install a .Net Core SDK, it will internally create a webserver to act as a backend server for the .Net Application. Proxy server will be sending the client calls to Kestrel Web server and responds back with the output from .Net App.

5. App: .Net Core Application we deploy to be containerized.