Windows Containers in Kubernetes

You've got a .NET Framework monolith that's been running on bare metal for a decade. Your team wants Kubernetes, but every time you try to containerize it, you hit the wall: Windows containers. They're not just Linux containers with a different base image. They're a different beast—heavier, slower to start, and with networking quirks that will make you question your life choices. But if you're running Windows workloads in production, you don't have a choice. This article is about making Windows Containers in Kubernetes work without losing your sanity. We'll cover the gotchas that the docs gloss over: image sizes that break your CI pipeline, networking that requires manual hacks, and scheduling constraints that turn your cluster into a puzzle. By the end, you'll know exactly when to use Windows containers, how to set them up, and what to avoid.

Why Windows Containers Are Not Linux Containers

Windows containers share the same kernel as the host OS, unlike Linux containers which use a shared kernel with isolation. This means a Windows container running on Windows Server 2019 cannot run on Windows Server 2022 without recompilation. The base image sizes are massive—Windows Server Core is around 3.5 GB, and Nano Server is about 1 GB. This impacts image pull times, storage, and CI/CD pipeline speed. Additionally, Windows containers have limited support for certain Linux kernel features like cgroups v2, which affects resource limits and monitoring. In production, this translates to longer deployment times and higher disk usage. You must plan for image caching and use smaller base images like Windows Server Core or Nano Server where possible. Also, Windows containers require Windows nodes, which are more expensive and less common than Linux nodes in cloud providers.

# Check Windows version on a node
$os = Get-WmiObject Win32_OperatingSystem
Write-Host "OS: $($os.Caption), Version: $($os.Version)"
# Output: OS: Microsoft Windows Server 2019 Standard, Version: 10.0.17763

Setting Up a Mixed Cluster with Windows Nodes

To run Windows containers, you need at least one Windows node in your cluster. Most cloud providers offer Windows node pools (e.g., AKS, EKS, GKE). When creating a mixed cluster, you must taint the Windows nodes so that Linux pods don't schedule on them. Use node.kubernetes.io/os=windows:NoSchedule taint. Then add a toleration to your Windows pod specs. Also, you need to install a CNI plugin that supports Windows, like Flannel or Calico. Flannel is the simplest but has limitations with host-gateway mode. Calico offers better network policy support but requires extra configuration. In production, we recommend Calico for its flexibility. Ensure your cluster's control plane is Linux-based—Windows nodes cannot run control plane components. Also, consider using a separate node pool for Windows to simplify management and scaling.

apiVersion: v1
kind: Node
metadata:
  name: win-node-1
  labels:
    kubernetes.io/os: windows
spec:
  taints:
  - key: node.kubernetes.io/os
    value: windows
    effect: NoSchedule

Choosing the Right Windows Container Base Image

Windows offers three base images: Windows Server Core (3.5 GB), Windows Nano Server (1 GB), and Windows Server (full desktop, 8+ GB). For Kubernetes, use Nano Server for .NET Core or .NET 5+ apps, and Server Core for .NET Framework apps. Avoid the full Windows Server image—it's too large and has unnecessary components. The image size directly impacts deployment speed: pulling a 3.5 GB image on a cold node can take minutes. Use image caching strategies like pre-pulling images on node startup or using a container registry with geo-replication. Also, consider using multi-stage builds to reduce final image size. For example, build your app on a full SDK image, then copy the output to a smaller runtime image.

# Multi-stage build for .NET Framework app
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8 AS build
WORKDIR /app
COPY . .
RUN msbuild MyApp.csproj /p:OutputPath=out

FROM mcr.microsoft.com/dotnet/framework/runtime:4.8
WORKDIR /app
COPY --from=build /app/out .
ENTRYPOINT ["MyApp.exe"]

Networking Challenges with Windows Containers

Windows containers have different networking models than Linux. They support two modes: transparent (using host networking) and overlay (using a virtual switch). In Kubernetes, overlay networking is required for pod-to-pod communication across nodes. However, Windows lacks native support for some CNI features like network policies and service mesh sidecars. For example, Calico's eBPF dataplane doesn't work on Windows. You must use the standard iptables mode. Also, Windows containers cannot use hostNetwork mode for privileged pods—they require a different approach. In production, we've seen issues with DNS resolution and port exhaustion due to Windows' ephemeral port range. Increase the ephemeral port range on Windows nodes to avoid conflicts.

# Increase ephemeral port range on Windows node
netsh int ipv4 set dynamicport tcp start=10000 num=55535
# Verify
netsh int ipv4 show dynamicport tcp
# Output: Protocol tcp Dynamic Port Range
# ----------------------------------
# Start Port : 10000
# Number of Ports : 55535

Storage Considerations for Windows Containers

Windows containers support volumes, but with limitations. They cannot use certain volume types like hostPath with specific mount options, and they have issues with file permissions. For persistent storage, use Azure Disk or AWS EBS with the appropriate CSI driver. Windows containers also have a 260-character path limit, which can cause issues with long file paths. Use short paths or enable long path support via Group Policy. Additionally, Windows containers cannot use tmpfs mounts. In production, we recommend using StatefulSets with persistent volume claims for stateful Windows apps. Be aware that resizing volumes on Windows nodes may require a reboot.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: win-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: managed-premium

Resource Management and Limits

Windows containers support CPU and memory limits, but with caveats. CPU limits are enforced using Windows job objects, which may not be as precise as Linux cgroups. Memory limits are enforced, but Windows containers can still experience memory pressure if the limit is too low. Also, Windows containers have a minimum memory requirement: Server Core needs at least 512 MB, Nano Server 256 MB. Set requests and limits appropriately. In production, monitor memory usage closely—Windows containers tend to have higher memory overhead due to the kernel. Use the windows.alpha.kubernetes.io/memory annotation for older clusters. Also, note that Windows containers cannot use hugepages or certain resource types.

apiVersion: v1
kind: Pod
metadata:
  name: win-pod
spec:
  containers:
  - name: win-container
    image: mcr.microsoft.com/windows/servercore:ltsc2019
    resources:
      requests:
        memory: "1Gi"
        cpu: "500m"
      limits:
        memory: "2Gi"
        cpu: "1"

Security and Authentication

Windows containers support Active Directory integration via Group Managed Service Accounts (gMSA). This allows containers to authenticate to domain resources. To use gMSA, you need to install the gMSA credential spec on each Windows node and reference it in the pod spec. Also, Windows containers run with the same security context as the host by default. Use Pod Security Policies or OPA to enforce restrictions. Note that Windows containers cannot run as non-root user in the same way as Linux—they use a different user model. In production, always use gMSA for domain-joined apps and avoid running containers as Administrator.

apiVersion: v1
kind: Pod
metadata:
  name: win-gmsa-pod
spec:
  securityContext:
    windowsOptions:
      gmsaCredentialSpecName: "webapp-spec"
  containers:
  - name: win-container
    image: mcr.microsoft.com/windows/servercore:ltsc2019

Monitoring and Logging

Windows containers have limited monitoring compared to Linux. Tools like Prometheus and cAdvisor have partial support. Use the Windows exporter for Prometheus to collect metrics like CPU, memory, and disk. For logging, Windows containers output logs to stdout/stderr, but some apps write to Event Log. Use a sidecar container to forward Event Log entries to stdout. Also, Windows containers do not support Linux-specific system calls, so some monitoring agents may not work. In production, we recommend using a dedicated Windows monitoring solution like Azure Monitor for containers or Datadog with Windows support.

apiVersion: v1
kind: Pod
metadata:
  name: win-logger
spec:
  containers:
  - name: app
    image: myapp:windows
  - name: log-forwarder
    image: mcr.microsoft.com/windows/servercore:ltsc2019
    command: ["powershell", "-Command", "Get-WinEvent -LogName Application | ForEach-Object { Write-Host $_.Message }"]

CI/CD Pipeline Considerations

Building Windows container images in CI/CD pipelines requires Windows build agents. Most CI platforms support Windows runners (e.g., GitHub Actions, Azure DevOps). However, image sizes cause long build and push times. Use caching layers and multi-stage builds to speed up. Also, ensure your registry supports Windows images (most do). Tag images with the Windows version (e.g., ltsc2019) to avoid kernel mismatch. In production, we recommend using a dedicated Windows build agent pool and pre-pulling base images to reduce build time.

trigger:
- main
pool:
  vmImage: 'windows-2019'
steps:
- task: Docker@2
  inputs:
    containerRegistry: 'myRegistry'
    repository: 'myapp'
    command: 'buildAndPush'
    Dockerfile: '**/Dockerfile.windows'
    tags: '$(Build.BuildId)-ltsc2019'

Common Pitfalls and How to Avoid Them

1. Kernel version mismatch: Always match container base image to node OS version. 2. Image size: Use multi-stage builds and smaller base images. 3. Networking: Increase ephemeral port range and use a CNI that supports Windows. 4. Storage: Avoid hostPath and use CSI drivers. 5. Resource limits: Set appropriate memory limits to avoid OOM. 6. Security: Use gMSA for domain auth. 7. Monitoring: Use Windows-specific tools. 8. CI/CD: Use Windows build agents and tag images. 9. Scheduling: Taint Windows nodes and add tolerations. 10. Logging: Forward Event Logs to stdout. By addressing these, you can run Windows containers in production reliably.

apiVersion: v1
kind: Pod
metadata:
  name: win-pod
spec:
  tolerations:
  - key: "node.kubernetes.io/os"
    operator: "Equal"
    value: "windows"
    effect: "NoSchedule"
  containers:
  - name: win-container
    image: myapp:ltsc2019
    resources:
      requests:
        memory: "1Gi"
        cpu: "500m"
      limits:
        memory: "2Gi"
        cpu: "1"
    env:
    - name: LOG_PATH
      value: "C:\logs"
  volumes:
  - name: log-volume
    persistentVolumeClaim:
      claimName: win-pvc

Performance Tuning for Windows Containers

Windows containers have higher overhead than Linux. To improve performance: use process isolation instead of Hyper-V isolation (default in Kubernetes). Process isolation shares the kernel, reducing memory usage. Also, optimize your application for containerization: reduce memory footprint, use asynchronous I/O, and avoid heavy dependencies. Tune the Windows node: disable unnecessary services, optimize TCP settings, and use SSD storage. In production, we've seen significant improvements by using Nano Server for .NET Core apps and tuning the .NET garbage collector for containers.

# Disable unnecessary services
Stop-Service -Name wuauserv -Force
Set-Service -Name wuauserv -StartupType Disabled
# Optimize TCP settings
Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters -Name TcpTimedWaitDelay -Value 30
# Set power plan to high performance
powercfg /setactive 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c

When to Avoid Windows Containers

Windows containers are not a silver bullet. Avoid them if: your app can be rewritten for Linux (e.g., .NET Core vs .NET Framework), you need advanced Linux kernel features (e.g., eBPF, seccomp), or your team lacks Windows expertise. Also, avoid if you need fast scaling—Windows nodes take longer to provision. Consider using a hybrid approach: run stateless Linux microservices alongside a few Windows containers for legacy apps. In production, we've seen teams waste months trying to containerize apps that should have been rewritten. Evaluate the cost-benefit before committing.

Use Windows Containers if:
- App requires .NET Framework or Windows-specific APIs
- App cannot be rewritten
- Team has Windows admin skills

Avoid if:
- App can run on .NET Core/Linux
- Need fast scaling or low overhead
- Lack Windows expertise