Based on the Cloud Adoption Framework (CAF) and Well-Architected Framework (WAF), this project automates the audit of an entire Azure tenant in minutes, using AI and a sovereign environment to keep data confidential.

Introduction

Nowadays, auditing an Azure tenant can take several days. A tenant is composed of numerous resources : subscriptions, resource groups, security policies, network configurations, identity and access management (IAM), etc. Each element needs to be manually reviewed, navigating from service to service in the Azure portal, exporting data, cross-referencing it with CAF and WAF recommendations, and producing a coherent report. It is a long, tedious process, often prone to errors and oversights.

The goal here is to let artificial intelligence go through all these steps, generating a complete audit report in just a few minutes, while staying within a sovereign environment to ensure the confidentiality of your data.

To build this solution, we started from the foundation of one of our previous projects : Building the N1 Agent : An AI-Powered Kubernetes Assistant.

We highly recommend reading it first, as it covers the fundamental concepts and detailed technical implementation that this project builds upon. Here, we will only go through the specific adjustments made for this new use case.

Architecture Overview

Here's the architecture we've implemented :

Since the core infrastructure was already battle-tested with the N1 Agent, we kept it as-is and focused on what this use case actually needed. Same network layout, same machine segmentation, same core stack : OpenWebUI as the chat interface, n8n for workflow automation, Azure AI Foundry for sovereign model deployment, MCPO as a proxy layer for MCP Servers, and the MCP protocol itself.

The main addition for this use case is the Azure MCP Server, which exposes Azure resource management capabilities as HTTP endpoints through MCPO. We also integrated Open Terminal, which we'll cover further down.

Technical Implementation

Before the AI can audit anything, it needs access to the tenant. That means setting up an App Registration on the target tenant and granting it Read-Only access, following the principle of least privilege, the AI should only be able to see, never touch.

Azure MCP Server

Everything runs in Docker. Once the stack is up, the key step is configuring the mcpo-config.json file with your MCP Server settings. You'll point it to the official Microsoft Azure MCP package and provide your App Registration credentials, either inline or via environment variables :

{
  "mcpServers": {
    "azure-tenant": {
      "command": "npx",
      "args": ["-y", "@azure/mcp@latest", "server", "start"],
      "env": {
        "AZURE_TENANT_ID": "your-tenant-id",
        "AZURE_CLIENT_ID": "your-client-id",
        "AZURE_CLIENT_SECRET": "your-client-secret",
        "AZURE_TOKEN_CREDENTIALS": "prod"
      }
    }
  }
}

A couple of things worth noting here :

• The server start subcommand tells the Azure MCP package to run as an MCP Server and expose its endpoints. Without it, the package simply exits.

• AZURE_TOKEN_CREDENTIALS: "prod" tells the Azure Identity library to authenticate using a client secret, the right mode when working with an App Registration.

From there, the AI can query the tenant through direct prompts, or you can wire it into an n8n workflow to schedule regular audits and auto-generate reports :

Open Terminal

The Azure MCP Server gets you far, but it has limitations. Not every Azure resource has a dedicated MCP endpoint, notably around network security, Entra ID auditing, and fine-grained RBAC. On top of that, some resource queries require many sequential calls (resources within a resource group, for instance), which burns through tokens fast and risks hitting context limits.

For a truly exhaustive audit, you want full Azure CLI access. That's where Open Terminal comes in.

Open Terminal is a tool built by the OpenWebUI team, released in early February 2026. It's a self-hosted remote shell server that exposes a REST API, letting AI agents run shell commands, manage files, and interact with a live environment.

Setup is minimal. On your local machine :

# One-liner with uvx (no install needed)
uvx open-terminal run --host 0.0.0.0 --port 8000 --api-key your-secret-key

# Or install with pip
pip install open-terminal
open-terminal run --host 0.0.0.0 --port 8000 --api-key your-secret-key

Or as a Docker container if you'd rather not expose your host :

docker run -d --name open-terminal --restart unless-stopped -p 8000:8000 -v open-terminal:/home/user -e OPEN_TERMINAL_API_KEY=your-secret-key ghcr.io/open-webui/open-terminal

Once it's running, connect it to OpenWebUI via Settings > Integrations > Open Terminal (+), fill in the fields, and the terminal becomes available directly inside your conversations.

From there, ask the AI to install the Azure CLI, authenticate with your tenant (personal account or App Registration), and it'll start running az commands, giving it significantly more coverage than MCP endpoints alone.

Challenges

If Open Terminal gives the AI full CLI access, why use the Azure MCP Server at all ?

This is actually a broader question the community has been debating : is MCP becoming obsolete now that tools like Open Terminal exist ?

Our take : Open Terminal gives you more coverage, but it comes with real trade-offs. It either runs on your local machine or requires a dedicated container on your infra, and crucially, access cannot be scoped. Whatever account was used to set it up is the one the AI has access to. A poorly written prompt or a hallucination could result in arbitrary packages being installed on the machine. On on-premises infrastructure, that's a serious risk.

MCP, on the other hand, gives the AI a well-defined set of tools with clear descriptions. It knows what it can call, and nothing else. That's what makes it valuable : standardization and constraints. In security-sensitive contexts, both matter.

Beyond security, the Azure MCP Server is actively evolving, and what it already delivers is genuinely useful. It produces a solid report that surfaces the areas worth digging into, and for most use cases, knowing where to investigate is already the hard part. The Azure CLI adds more detail, but either way, an expert review is always the final step.

In practice, our answer is : not obsolete, just different. MCP and Open Terminal solve different problems, and understanding that distinction is what makes the combination powerful. We found that using both together works well : give the AI access to both tools in its system prompt, and let it pick the right one depending on the situation.

Conclusion

What a Cloud Engineer or Architect typically spends ~5 days on, inventorying resources, running a full CAF/WAF audit across every layer, writing the report, the AI completes in under 20 minutes :

This isn't about replacing experts. It's about eliminating the repetitive, time-consuming work that buries them.

The goal is to enable pre-auditing : quickly identifying the critical points that need deeper investigation, so the expert can focus directly on what matters. Beyond one-off audits, we also generate weekly reports on our tenants through n8n workflows, so nothing stays hidden for long.

The N1 agent transforms how DevOps teams interact with Kubernetes infrastructures by combining conversational prompts and advanced automation, all powered by artificial intelligence. Concretely, it enables teams to communicate with their K8s cluster in natural language, while automatically and autonomously analyzing incidents reported in ITSM tools.

Introduction

What if managing Kubernetes was as simple as having a conversation ? For most DevOps teams, diagnosing a production incident means :

• Switching between multiple tools (Rancher, Lens, monitoring dashboards, etc.) ;

• Executing 20+ kubectl commands to isolate the root cause ;

• Spending 30-60 minutes on what should be resolved in 5.

This is the main point driving the N1 Agent project : making infrastructure accessible through natural language and intelligently automating incident handling.

To achieve this, we deployed the OpenWeb UI application, an open-source web interface for interacting with AI models through a chat interface, for the conversational component. For automated incident management, an n8n workflow, an open-source automation tool operating on a principle of connected visual nodes, was set up to retrieve incidents from the ITSM platform, then query our Kubernetes clusters to perform contextual analysis and generate recommendations.

Both tools rely on an AI model hosted on Azure Foundry, Microsoft's platform for developing and deploying AI models, which allows us to leverage advanced artificial intelligence capabilities while maintaining control over our data in a sovereign cloud environment.

But then, does the AI directly access our Kubernetes clusters to execute kubectl commands in real-time ? Well no, it doesn't access them directly. This is where the final component comes in : MCPO, developed by the OpenWeb UI team, a proxy that bridges HTTP requests and the MCP protocol, there by securing and standardizing communication between the conversational interface and Kubernetes resources, based on the MCP protocol.

What is the Model Context Protocol (MCP) ?

The Model Context Protocol, or MCP, is an open-source standard protocol for connecting AI applications to external systems, developed by Anthropic and released in November 2024.

This protocol enables AI models such as GPT-4, Claude, or other LLMs to securely interact with external data sources, APIs, and tools in a standardized way. Rather than each AI application implementing its own custom integration methods, MCP provides a unified framework that allows models to access databases, execute commands, retrieve files, or interact with various services while maintaining security and control over these interactions.

An official list of MCP Servers for various platforms has been compiled and is available here.

Architecture Overview

REMARK : This is the configuration we have implemented in our infrastructure. However, there are many features available across the different components to improve or adapt this setup to your specific needs.

Here's the architecture we've implemented :

MCPO acts as the heart of this architecture as a bidirectional translator between HTTP and the MCP protocol. It loads MCP Servers, in our case, the Kubernetes MCP Server, and exposes their native capabilities as HTTP endpoints. This design allows any HTTP-capable client to interact with Kubernetes through standardized REST calls, while MCPO handles all protocol conversion behind the scenes.

For security reasons, since this component contains sensitive information, particularly the Kubernetes cluster kubeconfigs, we have applied dedicated host and network segmentation. This machine is completely isolated and exposed only through a specific port, accessible exclusively from the host running OpenWeb UI and n8n.

To simplify kubeconfig exports and user access management on the Kubernetes clusters, kubeconfigs are created and exported using Rancher, our Kubernetes management platform. Rancher provides a centralized interface to manage multiple Kubernetes clusters, handle authentication and role-based access control (RBAC), and securely generate kubeconfig files for users and services.

On a separate host, for the conversational component, users (Operator) connect to OpenWeb UI, which is configured with two key integrations : an AI model on Azure Foundry and MCPO as a tool provider. When a user submits a prompt requesting Kubernetes operations, the AI model interprets the intent and generates an HTTP call to the appropriate MCPO endpoint. MCPO converts this into MCP protocol messages, forwards them to the Kubernetes MCP Server for execution, and returns the results through the reverse path.

n8n follows a similar pattern but through visual workflow nodes. First retrieves Kubernetes cluster–related incidents from the ITSM tool. Then, a node calls the Azure Foundry model to analyze incidents, while HTTP Request nodes interact with MCPO, first discovering available endpoints, and finally executing specific Kubernetes commands. Once the analysis is complete, the incident is updated with the analysis and recommendations (or even automatically resolved if it is granted more than read-only permissions!).

Finally, Traefik is a reverse proxy that routes incoming traffic to the appropriate service based on the domain name, enabling multiple applications (Open WebUI, n8n) to be accessed through different URLs while running on the same infrastructure.

Technical Implementation

All components run in Docker containers. Below is the docker-compose.yaml file for the machine that hosts Open WebUI, n8n, and Traefik :

version: '3.8'

services:
  traefik:
    image: traefik:latest
    container_name: traefik
    restart: unless-stopped
    command:
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.file.filename=/traefik-dynamic.yml"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"

    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./certs:/certs:ro
      - ./traefik-dynamic.yml:/traefik-dynamic.yml:ro
    networks:
      - mcphub-network

  open-webui:
    image: ghcr.io/open-webui/open-webui:0.6.41
    container_name: open-webui
    environment:
      # SSO feature (Microsoft)
      - MICROSOFT_CLIENT_ID=${MICROSOFT_CLIENT_ID}
      - MICROSOFT_CLIENT_SECRET=${MICROSOFT_CLIENT_SECRET}
      - MICROSOFT_CLIENT_TENANT_ID=${MICROSOFT_CLIENT_TENANT_ID}
      - MICROSOFT_REDIRECT_URI=${MICROSOFT_REDIRECT_URI}
      - OPENID_PROVIDER_URL=${OPENID_PROVIDER_URL}
      - ENABLE_OAUTH_SIGNUP=true
    env_file:
      - .env
    volumes:
      - open-webui-data:/app/backend/data
    extra_hosts:
      - "host.docker.internal:host-gateway"
    restart: unless-stopped
    networks:
      - mcphub-network
    labels:
      - "traefik.enable=true"
      # Adapt this to your domain
      - "traefik.http.routers.owui.rule=Host(`owui.your.domain`)"
      - "traefik.http.routers.owui.entrypoints=websecure"
      - "traefik.http.routers.owui.tls=true"
      - "traefik.http.services.owui.loadbalancer.server.port=8080"
      # Adapt this to your domain
      - "traefik.http.routers.owui-http.rule=Host(`owui.your.domain`)"
      - "traefik.http.routers.owui-http.entrypoints=web"
      - "traefik.http.routers.owui-http.middlewares=redirect-to-https"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"

  n8n:
    image: n8nio/n8n:stable
    container_name: n8n
    environment:
      # Adapt this to your domain
      - N8N_HOST=n8n.your.domain
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      # Adapt this to your domain
      - WEBHOOK_URL=https://n8n.your.domain/
      - GENERIC_TIMEZONE=Europe/Zurich
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=n8n-postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${N8N_POSTGRES_PASSWORD}
      - N8N_CUSTOM_EXTENSIONS=/app/custom
      - N8N_SECURE_COOKIE=false
      - N8N_SKIP_AUTH_ON_OAUTH_CALLBACK=true
      - N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true
      - N8N_RUNNERS_MODE=external
      - N8N_RUNNERS_BROKER_LISTEN_ADDRESS=0.0.0.0
      - N8N_RUNNERS_BROKER_PORT=5679
      - N8N_RUNNERS_AUTH_TOKEN=${N8N_RUNNER_TOKEN}
      - N8N_NATIVE_PYTHON_RUNNER=true
    env_file: ".env"
    volumes:
      - n8n-data:/home/node/.n8n
      - ./n8n-custom:/app/custom:ro
    restart: unless-stopped
    networks:
      - mcphub-network
    extra_hosts:
      - "host.docker.internal:host-gateway"
    depends_on:
      - n8n-postgres
    labels:
      - "traefik.enable=true"
      # Adapt this to your domain
      - "traefik.http.routers.n8n.rule=Host(`n8n.your.domain`)"
      - "traefik.http.routers.n8n.entrypoints=websecure"
      - "traefik.http.routers.n8n.tls=true"
      - "traefik.http.services.n8n.loadbalancer.server.port=5678"
      # Adapt this to your domain
      - "traefik.http.routers.n8n-http.rule=Host(`n8n.your.domain`)"
      - "traefik.http.routers.n8n-http.entrypoints=web"
      - "traefik.http.routers.n8n-http.middlewares=redirect-to-https"

  n8n-postgres:
    image: postgres:15-alpine
    container_name: n8n-postgres
    environment:
      - POSTGRES_DB=n8n
      - POSTGRES_USER=n8n
      - POSTGRES_PASSWORD=${N8N_POSTGRES_PASSWORD}
    env_file: ".env"
    volumes:
      - n8n-postgres-data:/var/lib/postgresql/data
    restart: unless-stopped
    networks:
      - mcphub-network

  n8n-runners:
    image: n8nio/runners:stable
    container_name: n8n-runners
    environment:
      - N8N_RUNNERS_TASK_BROKER_URI=http://n8n:5679
      - N8N_RUNNERS_AUTH_TOKEN=${N8N_RUNNER_TOKEN}
    env_file: ".env"
    restart: unless-stopped
    networks:
      - mcphub-network
    depends_on:
      - n8n

volumes:
  open-webui-data:
  n8n-data:
  n8n-postgres-data:

networks:
  mcphub-network:
    driver: bridge

The n8n-runners service is responsible for executing workflows and tasks outside the main n8n process, allowing for better scalability and parallel execution. It is therefore recommended to include it in any deployment where workflow performance and load distribution are important.

Here is the traefik-dynamic.yaml file for SSL certificates, which also needs to be configured to ensure HTTPS :

tls:
  certificates:
    # Adapt this to your domain
    - certFile: /certs/your.domain.crt
      keyFile: /certs/your.domain.key

Then, here is the docker-compose.yaml for the isolated host, with the MCPO component :

version: '3.8'

services:
  mcpo:
    build:
      context: .
      dockerfile: Dockerfile.mcpo
    container_name: mcpo
    command: mcpo --config /app/mcpo-config.json
    ports:
      - "8000:8000"
    volumes:
      - ./mcpo-config.json:/app/mcpo-config.json:ro
      - ./configs:/app/configs:ro
    restart: unless-stopped
    networks:
      - mcpproxy-network
    extra_hosts:
      - "host.docker.internal:host-gateway"
      # Connection to Rancher
      - "rancher.your.domain:IP"

networks:
  mcpproxy-network:
    driver: bridge

Next, you need to create a Dockerfile.mcpo for the MCPO container, which adds kubectl to the container to interact with Kubernetes clusters :

FROM ghcr.io/open-webui/mcpo:main

RUN apt-get update && \
    apt-get install -y curl && \
    curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && \
    chmod +x kubectl && \
    mv kubectl /usr/local/bin/ && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

You also need to create the mcpo-config.json file, which contains the configuration for the MCP servers. This file defines each MCP server, the command to run, its arguments, and the environment variables required to connect to it :

{
  "mcpServers": {
    "your-kubernetes-cluster": {
      "command": "npx",
      "args": ["-y", "mcp-server-kubernetes"],
      "env": {
        "KUBECONFIG": "/app/configs/kubernetes/your-kubernetes-kubeconfig"
      }
    }
  }
}

For Kubernetes, we use the mcp-server-kubernetes project. In this setup, npx (a Node.js package runner that allows you to execute npm packages without installing them globally) downloads and runs the mcp-server-kubernetes package inside the container. This exposes the necessary tools to interact with the Kubernetes clusters defined in the configuration, allowing MCPO to manage and execute commands on them.

To allow MCPO to connect to your Kubernetes cluster, you need to export your cluster’s kubeconfig and place it in the appropriate directory, in ./configs/kubernetes/ in this configuration. This directory will then be mounted into the MCPO container.

Finally, you can simply run your docker-compose to start all the services.

OpenWeb UI

Once the services are running, you can access the OpenWeb UI interface at https://owui.your.domain/. You can then create an administrator account as your first step.

Importing a Model

To integrate a pre-generated LLM from Azure AI Foundry, follow these steps :

1. Go to Settings > Admin Settings > Connections, then click Add Connection ;

2. Change the Provider Type to Azure OpenAI (if you are using Azure Foundry) ;

3. Fill in the fields : URL, Auth Bearer, API Version, and Model IDs ;

4. Save the configuration.

To control the LLM’s behavior and handle errors (so it does not stop if a command fails), you need to activate specific settings and provide a system prompt. For the Kubernetes agent, use the following system prompt :

You are a Senior SRE with over 10 years of experience.

You have access to MCP tools for Kubernetes.

ABSOLUTE RULE: In a SINGLE response, make multiple attempts if a tool fails.

If a tool fails:
  Try 3-5 different variants IN THE SAME RESPONSE.
  Never say "I’ll try later" or "I will come back."
  Call the tools successively until one succeeds.

Example:
❌ Attempt 1: kubectl_logs_post (failed: unsupported resource)
❌ Attempt 2: kubectl_get_logs (failed: tool not found)
✅ Attempt 3: get_pod_logs (success!)

Result: [display the result]

You must perform all attempts in the same message, not across multiple turns.

A system prompt allows you to define the AI's behavior, role, and constraints, essentially providing instructions that guide how the model should respond and interact with users.

The configuration of these parameters and the system prompt is done as follows:

1. Go to Settings > Admin Settings > Connections, then Models, and select your model ;

2. Under System Prompt, add the above message to frame the LLM's behavior ;

3. In Advanced Params on the same page :

   • Temperature : 0.0 ;

   • Function Calling : Native ;

4. Save the configuration.

The temperature setting controls the randomness of the model's responses, a lower value (like 0.0) makes outputs more deterministic and focused, while higher values increase creativity and variability.

Function calling enables the AI to invoke multiple endpoints within the same request, allowing it to retry commands in case of failure.

Integrating an MCP Server

To add the MCP server for a Kubernetes cluster, follow these steps:

1. Go to Settings > Admin Settings > External Tools, then click on Add Connections ;

2. Select the OpenAPI type, then configure :

   • URL : http://mcpo-host:8000/XXX (the URL to the MCP server followed by the cluster name) ;

   • Name : A descriptive name ;

3. Save the configuration.

You can then select the cluster in a chat by clicking on the Integration > Tools button.

n8n

As with Open WebUI, once the services are started, you can access it at https://n8n.your.domain/. You can then create an administrator account.

Key components

For the n8n configuration, this obviously depends on your infrastructure and the tools you use. However, the three key nodes to use are as follows :

• Azure OpenAI Chat Model (or equivalent depending on what you use) : This node connects to Azure's OpenAI service, allowing the workflow to send prompts and receive AI-generated responses for tasks like incident analysis or generating recommendations.

• HTTP Request : It makes HTTP calls to external APIs, in this case, to interact with MCPO endpoints for discovering and executing Kubernetes operations.

• AI Agent : This node enables the workflow to leverage AI capabilities for decision-making, analysis, and intelligent task execution within the automation pipeline. You can define instructions for it, such as the system prompt to frame the AI's behavior and the user message to provide the specific context of the request. Then, you can connect it to other components like the HTTP Request node so it can retrieve endpoints, as well as to your Azure OpenAI Model (or equivalent).

Despite the fact that n8n has an MCP node, we cannot use it in our case because, as a reminder, MCPO serves to translate between HTTP ←→ MCP. Therefore, in reality, the AI never directly contacts the MCP protocol.

You should be able to achieve the entire automation without having to code anything (except for very specific configurations). Indeed, if you manage your prompts correctly, the AI Agent component completely replaces the coding part.

For more details about n8n, we have written a dedicated article on this technology here.

Use Cases and Demonstrations

The main use cases for having an N1 Agent are primarily to improve DevOps team efficiency. Whether through the chat system, where you simply interact with the AI for debugging or deployment, or through n8n incident automation, where initial analysis or incident resolution can be done autonomously.

Additionally, if we want to open access to our Kubernetes clusters to other teams who may not have the necessary system knowledge, they can attempt application debugging through chat with an AI configured as a "Senior SRE".

Beyond basic operations, the N1 Agent can handle more advanced scenarios such as automated scaling decisions based on resource usage patterns, security audits and compliance checks across your infrastructure, or cost optimization recommendations by identifying underutilized resources.

Here is a demonstration of an interaction with the AI on a Kubernetes cluster through OpenWeb UI :

In this example, the AI successfully :

1. Listed the existing pods in a specific namespace ;

2. Created a new pod with the requested configuration ;

3. Re-listed the pods to verify successful creation ;

4. Provided clear explanations at each step.

Then, for the n8n part, here is a demonstration of our workflow, with an alert raised in our ITSM tool due to a pod in error :

In order, here are the steps :

1. Incidents related to Kubernetes clusters are retrieved from the ITSM tool ;

2. A first AI Agent “Structuration”, connected to Azure, analyzes the incident to structure it clearly for the next step ;

3. Then, "Analyze (K8s)" is configured as a "Senior SRE" and uses the Azure model as well, stores the information, and retrieves the MCPO endpoints for the MCP Server ;

4. Next, the command is sent to “Execute” AI Agent, which executes the command on MCPO through an HTTP Request (POST), and the returned result is sent back to “Analyze (K8s)” ;

5. This loop continues until the AI determines it has found the source of the problem. If found, it proceeds to the next step ;

6. “Final Answer” node simply reformats the response correctly and posts it to the ITSM tool.

Challenges and Conclusion

What’s the final result ? To measure the real-world impact of the N1 Agent, we compared incident resolution times between traditional manual workflows and our automated approach :

The results speak for themselves : what traditionally takes 45 minutes is now resolved in approximately 8 minutes, an 82% time reduction. The most significant gain occurs during the diagnostic phase, where the AI simultaneously executes multiple checks that would normally require sequential manual investigation.

Beyond the time savings, this automation eliminates the need to switch between multiple tools and enables continuous incident response, even outside business hours.

However, the AI era is still in its infancy, even more so for the MCP protocol, which celebrated its first anniversary in November 2025. Although development is growing rapidly, we still question whether we can truly grant the AI more than read-only access to our clusters ? And can we even trust its analysis compared to that of an expert SRE ? Indeed, during our tests, we have encountered instances where the AI hallucinated and provided incorrect information about the wrong cluster for instance.

Additionally, we have seen that there is an extensive list of MCP servers available for various use cases. Our objective moving forward is to extend this technology to other teams, enabling broader automation capabilities and improving operational efficiency across the organization. As the ecosystem matures and best practices emerge, we anticipate gradually expanding the AI's permissions while maintaining strict safeguards and monitoring.

These two days were full of discussions, workshops (we had the opportunity to participate to the Exoscale capture the flag workshop), and technical talks focused on Kubernetes and the cloud-native ecosystem.

Rather than diving deeply into low-level technical details, we would like to highlight three key topics that particularly caught our attention during the event; each reflecting important trends in how Kubernetes is evolving broader and wider on the market.

KServe: Hosting LLMs and ML Models on Kubernetes

One of the most hot topics was KServe, an open-source project that belongs to the Knative ecosystem and enables deployment, scaling, and serving of machine learning models directly on Kubernetes clusters.

Through a single Custom Resource Definition (CRD), InferenceService, KServe exposes models via HTTP or gRPC endpoints while dynamicly handling autoscaling, resource allocation, networking, rolling updates, and revision management.

It supports multiple popular ML runtimes as well as fully custom containerized models. Thanks to Knative, KServe can scale workloads down to zero pods, significantly reducing infrastructure costs when models are idle and not actively used.

Architecture:

More info here : https://kserve.github.io/website/

KubeVirt: Running Virtual Machines on Kubernetes

Another strong theme was KubeVirt, a Kubernetes extension that allows virtual machines to run natively alongside containers within the same cluster.

By introducing virtualization primitives through CRDs such as VirtualMachine and VirtualMachineInstance, KubeVirt makes it possible to orchestrate VMs using standard Kubernetes tools and workflows.

Under the hood, it relies on KVM/QEMU, while integrating with Kubernetes concepts like RBAC, CNI networking, and PersistentVolumeClaims.

This approach helps organizations to unify containerized and legacy workloads under a single control plane!

Features such as live VM migration, GitOps-driven lifecycle management, and gradual modernization of existing applications make KubeVirt a powerful path between traditional virtualization and cloud-native platforms.

Example:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: test-vm
spec:
  running: true                 # Vm auto starts
  template:
    spec:
      domain:
        devices:
          disks:
            - name: containerdisk
              disk: {}
        resources:
          requests:
            memory: 256Mi
      volumes:
        - name: containerdisk
          containerDisk:
            image: quay.io/kubevirt/cirros-container-disk-demo

Cluster API: Kubernetes Managing Kubernetes

The third topic that stood out was Cluster API (CAPI), an open-source framework designed to manage the full lifecycle of Kubernetes clusters using Kubernetes itself.

So, clearly, the cluster becomes a K8s object itself !

Cluster API introduces a set of declarative CRDs ( Cluster, Machine, and MachineDeployment, etc) that allow infrastructure provisioning, node bootstrap, upgrades and multi-cloud management to be expressed as code.

One of Cluster API’s major aspects is its provider model: the same API can manage clusters across platforms like vSphere, Azure, AWS, Nutanix, and others. This brings a uniform, automated, and repeatable approach to Kubernetes operations, regardless of the underlying infrastructure! A real game changer !

Conclusion

We would like to thank all the participants and organizers for these two instructive days! It was a great opportunity of exchanges, new product discovery and cool talks with a lot of different people, expanding capabilities and community aspects !

For anyone curious about Kubernetes, cloud-native technologies, or the future of infrastructure and AI platforms, we can only encourage you to join future editions of this event. The community, the knowledge sharing, and the discussions make it well worth the time.

For those who would like to see all the talks, everything is available here:
https://www.youtube.com/playlist?list=PLg2OjtyfIbOGVN6AIa-BVgL9i0SpR_tFD

Cert-manager now supports ACME HTTP-01 challenges via Gateway API, allowing us to issue and renew Let’s Encrypt certificates without Ingress resources dependance.

In this guide, we will:

• Install cert-manager with Gateway API support

• Configure an ACME ClusterIssuer

• Wire cert-manager to Envoy Gateway

• Issue a Let’s Encrypt certificate

• Inspect the certificate lifecycle (CertificateRequest, Order, Challenge)

• Quick bonus: Add HTTP → HTTPS redirection using an HTTPRoute

Context: what runs in which namespace ?

In order to have a better understanding of our main resources, here’s a small schema to visualize what runs in which namespace:

Installing cert-manager with Gateway API Support

First, we install cert-manager with its CRDs and enable Gateway API integration with the correct config flag:

helm repo add jetstack https://charts.jetstack.io
helm repo update

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.crds.yaml
kubectl create namespace cert-manager

helm upgrade --install cert-manager oci://quay.io/jetstack/charts/cert-manager \
  --namespace cert-manager \
  --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \
  --set config.kind="ControllerConfiguration" \
  --set config.enableGatewayAPI=true

This enables cert-manager to generate HTTPRoute objects to serve ACME HTTP-01 challenges directly through Envoy Gateway.

Configuring the ACME ClusterIssuer (Let’s Encrypt)

To issue certificates, cert-manager requires an issuer. So we need to create a production Let’s Encrypt ClusterIssuer using the Gateway API HTTP solver. We’ve provided some additional informations, as the mail adress and the server used to contct Let’s Encrypt. Additionally, we need to specify which gateway (with the namespace) will be used with the received certificate.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-http01
spec:
  acme:
    email: your@mail.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-http01
    solvers:
    - http01:
        gatewayHTTPRoute:
          parentRefs:
          - name: gateway1
            namespace: test

This instructs cert-manager to dynamically create an HTTPRoute during ACME validation.

Preparing Envoy Gateway to Terminate TLS

Next, as Envoy Gateway needs a TLS listener, we’ll patch the gateway with the TLS listener (if it wasn’t already done before).

spec:
  gatewayClassName: gatewayclass1
  listeners:
  - allowedRoutes:
      namespaces:
        from: Same   # Only routes in "test" namespace can attach
    hostname: myhostname.com
    name: https
    port: 443
    protocol: HTTPS
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: eg-https

The Gateway will terminate TLS using the Kubernetes secret eg-https.

Observing Certificate Issuance

When cert-manager begins issuing a certificate, it performs several steps:

1. Generate a private key

2. Create a CertificateRequest

3. Create an ACME Order

4. Create an ACME Challenge

5. Create a temporary HTTPRoute to serve the HTTP-01 challenge

6. Wait for Let’s Encrypt validation

7. Fetch the certificate

8. Store it in the target Kubernetes secret

9. Clean up Order, Challenge, and temporary HTTPRoute

When all those steps are done, we’ll receive the certificate.

Viewing the issued certificate

To get the certificate’s details, we just need to use

kubectl describe certificate eg-https -n test

Example output:

Name:         eg-https
Namespace:    test
...
Status:
  Conditions:
    Message:               Certificate is up to date and has not expired
    Reason:                Ready
    Status:                True
  Not After:               2026-02-26T13:44:40Z
  Revision:                1
Events:
  Normal  Issuing     Issuing certificate as Secret does not exist
  Normal  Generated   Stored new private key in temporary Secret
  Normal  Requested   Created new CertificateRequest resource "eg-https-1"
  Normal  Issuing     The certificate has been successfully issued

This output proves that cert-manager successfully:

• created a key

• created a CertificateRequest

• completed ACME validation

• issued the certificate

Inspecting CertificateRequests

kubectl get certificaterequests -n test

Expected output:

NAME          APPROVED   DENIED   READY   ISSUER                 AGE
eg-https-1    True                True    letsencrypt-http01     2d18h

If we need detailed inspection to show the full ACME validation journey:

kubectl describe certificaterequest eg-https-1 -n test

Additional feature: Enforcing HTTP → HTTPS Redirection

In our example, we use only HTTPS but if a redirection is necessary, Gateway API allows explicit redirect policies using filters.

To force HTTPS globally, we can add a route with the correct filter (request redirect) and refere the gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: redirect-to-https
  namespace: test
spec:
  parentRefs:
  - name: gateway1
  hostnames:
  - "yourdomain.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    filters:
    - type: RequestRedirect
      requestRedirect:
        scheme: https
        statusCode: 301 # moved permanently

All HTTP requests now receive a 301 Permanent Redirect to their HTTPS equivalent.

Conclusion

This workflow demonstrates how Envoy Gateway + Gateway API + cert-manager can provide letsencrypt certs, in the same way as an it does with Ingress Nginx. Some aspects changed comparing to the “old” way:

• No annotations needed

• Explicit API resources (Certificate, ClusterIssuer, HTTPRoute)

As Gateway API continues to be adopted, this integration is becoming a new standard for Kubernetes ingress security.

Official source here : https://kubernetes.io/blog/2025/11/11/ingress-nginx-retirement/#:~:text=In%20March%202026%2C%20Ingress%20NGINX,and%20left%20available%20for%20reference.

Of course, it will still be possible to use it but it will no longer receive maintenance, patches or security updates.

The reason are quite simple. The traditional Ingress API:

• Only exposes hostnames, paths, and backends

• Does not handle advanced TLS scenarios

• Does not offer traffic policies or advanced L7 behavior

• Cannot support multi-tenant or complex routing without an overload of annotations

• Annotations (while powerful) became difficult to maintain, inconsistent, and controller specific.

So, what should we do to avoid CVEs and upcoming troubles ? The answer is quite clear : use a new standard, Gateway API.

In fact, that’s not a new concept : the Gateway API appeared in 2020 and a lot of controllers appeared from 2021 (Envoy, Istio, Nginx via their own provider, Cilium) and started to be massively used since 2022.

Gateway API is a new architecture, not a drop-in replacement

This new approach introduces a richer and more structured model with objects such as:

• GatewayClass

• Gateway

• HTTPRoute, TCPRoute, GRPCRoute, etc.

This brings far better separation of concerns compared to Ingress.

New features to implement more advanced traffic control

Gateway API supports various features such as:

• Header matching

• Weighted routing (example here)

  

• Traffic splitting (for blue / green or canary)

• Multi-listener setups

• Better TLS configuration

• Multi-protocol routing

With Ingress, enabling the same functionalites usually required multiple and controller-specific annotations.

Support for diverse traffic types

Gateway API handles not only HTTP, but also:

• gRPC

• TCP

• TLS passthrough

• UDP (via implementations that support it)

Ingress is fundamentally HTTP-centric and lacks this protocol diversity.

A practical, real-world migration path: Ingress NGINX to Envoy Gateway

So, now, let’s switch to the best part: a practical (and rollback friendly) way to go from Ingress Nginx to Envoy Gateway. We choose Envoy for this example but we could have chosen Istio, Kong, Traefik, and many more.

Step 1 — Install Envoy Gateway

helm install eg oci://docker.io/envoyproxy/gateway-helm \
    --version v1.6.0 \
    -n envoy-gateway-system \
    --create-namespace

We choose to use Envoy Gateway to become the control-plane responsible for processing Gateway API.

It’s quite simple to install with the helm install command.

Step 2 — Create a GatewayClass

We now have to create an GatewayClass

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: gatewayclass1
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller

This ensures that all converted resources referencing gatewayClassName: gatewayclass1 are correctly handled by Envoy Gateway.

Step 3 — convert your objects with ingress2gateway

This tool scans existing Ingress objects and generates Gateway API equivalents:

• Gateway

• HTTPRoute

• BackendRefs

• TLS references

• Host rules

The project code sources are available here:

https://github.com/kubernetes-sigs/ingress2gateway

To install it, just run this command

go install github.com/kubernetes-sigs/ingress2gateway@latest

Once it’s install, you can run this command to first get the converted resources taken from your ingress related objects and thenmove it to api Gateway objects.

In this example, we took the objects from our "gitlab” namespace.

#first, list
/root/go/bin/ingress2gateway print \
    --namespace gitlab \
    --providers ingress-nginx 

#then,  apply
/root/go/bin/ingress2gateway print \
    --namespace gitlab \
    --providers ingress-nginx \
    | kubectl apply -f -

Expected output:

• A new Gateway resource in the target namespace

• A set of HTTPRoute objects matching the original ingress rules

• Listeners and TLS configured under the Gateway

• Routing logic now controlled by Envoy Gateway

Final step — Verify IP assignment and Gateway readiness

Run this command to list the gateways and services.

kubectl get gateways -A
kubectl get svc -A | grep envoy

This should confirm that:

• The Gateway shows PROGRAMMED=True , meaning that the gateway is ready and running

• An IP is assigned via LoadBalancer (or any IP provider, for example MetalLB)

• Routes are attached to the appropriate listeners

• DNS or host file entries point to the correct address

Warning: This will not simply “move” your existing Ingress resources into the Gateway API. As explained earlier, this process is reversible because we are not deleting the existing Ingress objects. This means that the new LoadBalancer services created by the Gateway will receive new IP addresses and will not replace the existing Ingress LoadBalancers!

If you need to keep the same IP address, just edit que “loadBalancerIP” parameter in your LoadBalancer services. If you’re using Metallb, don’t forget to restart your controller pod in order to avoid IP attribution errors.

Lessons learned during the migration

Here’re different aspect to care about when preparing your gateway API adoption.

Gateway listeners must include all hostnames

If the Gateway does not list a hostname, Envoy will reject the traffic, even if the HTTPRoute is valid.

TLS is configured on the Gateway, not the HTTPRoute

This is a key difference from Ingress and a common migration pitfall. Here’s the example we used to reference our secret.

Namespace scoping matters

allowedRoutes: Same restricts routes to the Gateway’s namespace.
Incorrect scoping causes silent routing failures.

Ingress2gateway accelerates the process but a manual review is still required

Of course, this tool is very useful and helps to gain a lot of time when enabling gateway APIs, but don’t trust it too much. You need to check all your objects with validations including:

• Correct backends

• Correct hostnames

• Presence of path / matchers

• TLS secret validity

• Route-to-listener alignment

What is n8n and how to install it?

N8n is a workflow automation tool that lets you schedule tasks and connect different applications, platforms, and services. n8n can be used as Saas but has also a free version that can be used on your own infra. In this example, we’ll install it on our own server (with traefik as reverse proxy) with a docker-compose file :

services:

  traefik:
    image: traefik:v3.0
    restart: always
    command:
      - "--log.level=DEBUG"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:8080"
      - "--entrypoints.websecure.address=:443"
    ports:
      - "8080:8080"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./certs:/certs
    networks:
      - traefik-net

  n8n:
    image: docker.n8n.io/n8nio/n8n:1.82.1
    restart: always
    user: "1000:1000"
    labels:
      - traefik.enable=true
      - traefik.http.routers.n8n.rule=PathPrefix(`/`)
      - traefik.http.routers.n8n.entrypoints=websecure
      - traefik.http.routers.n8n.tls=true
      - traefik.http.services.n8n.loadbalancer.server.port=5678
    environment:
      - N8N_HOST=your_IP
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - NODE_ENV=production
      - WEBHOOK_URL=https://your_ip_address/
      - GENERIC_TIMEZONE=Europe/Paris
      - N8N_BASIC_AUTH_ACTIVE=true
      - N8N_BASIC_AUTH_USER=admin
      - N8N_BASIC_AUTH_PASSWORD=XXXXXXXXXXXXXXXX
      - N8N_SECURE_COOKIE=true
    volumes:
      - ./n8n_data:/home/node/.n8n
    networks:
      - traefik-net

n8n – Workflow Automation Made Simple

With its visual editor, you can design workflows by assembling nodes, each representing an action such as calling an API, transforming data, or triggering another service.

There’re some concurrents, for example Zapier that uses the same principe; but n8n can be self-hosted (for exemple using the docker image), in addition to the SaaS version.

To begin our first workflow configuration, let’s take the case of retrieving email attachments

The principle is straightforward: each block (or node) represents an action, such as connecting to the Microsoft Graph API to fetch emails and attachments. Nodes are then linked together, passing inputs and outputs to each other. For instance, an attachment retrieved from an email can be sent to another service to be reformatted before being stored in a different application.

Because n8n is a no-code workflow automation platform, all nodes are configured visually. This makes it possible to build complex workflows without writing custom code.

To help you, there’re tons of integrations and examples here : https://n8n.io/integrations/

Adding AI into the Workflow

Now let’s extend this workflow with an AI component. By integrating Azure OpenAI into n8n, AI becomes just another node in the chain. Since Azure OpenAI provides an API endpoint, you can send information to an AI agent and use its response to enrich or transform your workflow.

In our example, we’ll connect to an AI agent configured in the Azure portal and managed through Azure AI Foundry.

Azure AI Foundry allows you to:

• Manage API keys and model selection

• Compare performance across multiple AI models (screenshot here)

• Build and customize your own AI agents

• Index and query documents

• Integrate connectors such as SharePoint, Azure Blob Storage, SQL databases, and more

• Control access and monitoring through RBAC and Azure Monitor

When adding an OpenAI node in n8n, you can define the behavior of the agent directly in the node configuration, and adjust it as needed (response format, retries, model, prompt, etc)

Workspace integration with “LLM” and “Azure OpenAI” nodes.

Conclusion

By combining n8n with a custom AI agent hosted in Azure OpenAI, you can design workflows that go far beyond basic task automation. This approach lets you build processes to your exact needs, offload repetitive work, and make your workflows smarter and more efficient!

To solve this problem, Platform Engineering is emerging as a structured approach to standardize and streamline the developer experience by building internal platforms tailored to development teams’ needs.

Platform Engineering: Definition and Approach

Platform Engineering is the practice of designing, building, and maintaining an internal developer platform (IDP), treated as a product for developers.

Core principles include:

• Standardized environments and workflows

• Self-service access to development, testing, and deployment workflows

• Built-in observability (logs, metrics, traces, alerting)

• Security and governance applied consistently without blocking teams

The aim is not to eliminate DevOps or SRE, but to evolve those practices in a way that reduces complexity and operational overhead for developers.

Why Platform Engineering Has Become Necessary

Two main developments have driven its adoption:

1. Toolchain proliferation

Cloud-native promised simplicity but sometimes resulted in the opposite: developers now face a growing list of tools—Kubernetes, ArgoCD, Vault, Terraform, Velero, and more. Developers are expected to understand and operate infrastructure, pipelines, monitoring, and deployment tooling, often beyond their core responsibilities.

This leads to fragmentation, a steep learning curve, and a loss of focus. In many cases, productivity and security suffer as a result. I personally had the opportunity to give lessons to devs and it was often challenging to make them quick integrated and “DevOps-ready”.

2. The incomplete transition to DevOps

Many organizations adopted DevOps in name only. Teams were renamed but workflows didn't change. Developers were given full infrastructure ownership without adequate tooling, guidance, or automation. Non-standard pipelines, environments, and monitoring setups became the norm.

The outcome: more autonomy, but also more inconsistency and operational burden.

What makes a Functional Internal Developer Platform?

A modern internal platform should provide:

• On-demand provisioning of development, test, and staging environments

• Standardized CI/CD pipelines, based on defined “golden paths”

• Integrated observability tools that don’t require manual configuration

• Self-service interfaces (portals or APIs) for common tasks such as deployments, rollbacks, or namespace creation

• Reusable and documented service templates

• Centralized management of identity, secrets, and access control

Crucially, the platform should be managed like an internal product: versioned, evolving, and focused on the developer experience.

Exemple product: Backstage

Backstage, an open source project from Spotify, is increasingly used to implement internal developer portals.

It provides:

• A service catalog for better visibility and documentation

• Scaffolding templates for consistent project creation

• Integration with CI/CD, monitoring, and alerting tools in a unified interface

• A flexible plugin system to connect with existing tools like GitLab, ArgoCD, Vault, and others

Backstage offers a central access point for developers, without hiding the underlying systems.

Key Steps to Building a Platform

Start by understanding what developers actually need:

• Interview teams

• Identify common pain points in daily workflows

Define golden paths—clear, optimized workflows that abstract complexity without removing control:

• For example: “Create a REST API with Node.js, Postgres, and CI/CD in three clicks”

Assemble the necessary building blocks:

• Kubernetes, Terraform, GitLab, ArgoCD, Vault, Velero, Prometheus, etc.

Expose functionality through clear APIs or developer portals such as Backstage or Port.

Automate as much as possible:

• Environment provisioning, secret management, backup/restore processes

Adopt a product mindset:

• Evolve the platform based on developer feedback

• Version and maintain features just like any internal product

Platform Engineering Is Not Just a Stack of Tools

A common mistake is to assume that deploying a few popular tools (Kubernetes, GitLab, ArgoCD) constitutes a platform. Without consistent workflows, shared patterns, and a developer-focused interface, this approach usually results in a fragmented and fragile setup.

The real value of Platform Engineering lies in reducing unnecessary complexity while enabling teams to remain autonomous and efficient.

Conclusion

Platform Engineering isn’t a buzzword—it’s a strategic response to the operational challenges of cloud-native and multi-cloud environments. It provides organizations with a way to maintain speed and flexibility while regaining control over tooling, workflows, and infrastructure.

With tools like Backstage, ArgoCD, Terraform, and Velero, an internal developer platform becomes a true product—designed to support developers while aligning with organizational goals.

OpenTelemetry (OTel) is a suite of tools, APIs, and SDKs designed to collect, process, and export telemetry data such as traces, metrics, and logs to improve observability in modern distributed systems.

Initially focused on traces and metrics, OpenTelemetry is now evolving into a unified standard for observing the complete behavior of applications. It provides a standardized protocol, OTLP (OpenTelemetry Protocol), which is increasingly replacing traditional formats like Jaeger or Prometheus.

(image taken from Grafana website)

What is the difference with tools like Prometheus ?

While OpenTelemetry and Prometheus are both related to observability, they all have different roles:

• OpenTelemetry is a framework for collecting telemetry data (traces, metrics, logs, profiling). It focuses on instrumentation and exporting data. It does not store or visualize data itself.

• Prometheus scrapes metrics data from targets, stores them locally, and allows for querying and alerting. So, OpenTelemetry can export metrics to Prometheus. As I thought that they were competitors, they’re in fact complementary

Why should you adopt OpenTelemetry?

• Unified signals (traces, logs, metrics, and now profiling, for example CPU or RAM usage over time for analysis): ability to collect and process different types of data in a standardized way

• Multi-cloud interoperability: Large organizations typically use more than one observability platform. Instrumenting for each vendor separately leads to complexity, redundancy, and performance issues (like double tracing). OpenTelemetry enables you to instrument your code once and export the data to multiple observability backends with minimal overhead.

• Widespread adoption by cloud providers: AWS, Azure, and GCP integrate OpenTelemetry natively

How could you try it ?

Basically, you have to use the SDKs available here : https://opentelemetry.io/docs/languages/

When you have your application ready, here’s an example in a K8s cluster (with sidecar to collect the data).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-with-otel
  labels:
    app: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: my-app
          image: my-org/my-app:latest
          ports:
            - containerPort: 8080
          env:
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://localhost:4317"
            - name: OTEL_RESOURCE_ATTRIBUTES
              value: "service.name=my-app"
            - name: OTEL_TRACES_SAMPLER
              value: "always_on"

        - name: otel-collector
          image: otel/opentelemetry-collector:latest
          args: ["--config=/etc/otel-collector-config.yaml"]
          ports:
            - containerPort: 4317 # OTLP gRPC
            - containerPort: 4318 # OTLP HTTP (optional)
          volumeMounts:
            - name: otel-config
              mountPath: /etc/otel-collector-config.yaml
              subPath: otel-collector-config.yaml

      volumes:
        - name: otel-config
          configMap:
            name: otel-collector-config

What is the future of this solution ? Here’s what the KubeCon Europe taught us…

Current situation

• OpenTelemetry is the second most contributed project in the CNCF, after Kubernetes (over 7000 contributions per week!)

• Many CNCF projects are actively adopting OpenTelemetry

• Increasing integration into open source libraries and frameworks

New features and development directions

• Profiling support in OTLP (v1.30.0) Profiling has been added as a first-class signal in OpenTelemetry, alongside traces, metrics, and logs. This allows low-overhead CPU profiling to be collected and exported via OTLP, enabling developers to correlate performance bottlenecks with application behavior more easily.

• Standardized Logs API in progress Although OpenTelemetry was initially not designed to include logs, community demand has pushed the project to develop a vendor-neutral Logs API. The aim is to unify logs with traces and metrics so they can all be processed in a single telemetry pipeline.

• OTLP/HTTP as the emerging standard format The OpenTelemetry Protocol (OTLP) over HTTP is becoming the preferred transport format for telemetry. It is designed to replace older, tool-specific formats such as Jaeger’s native format and Prometheus exporters, providing a more consistent and interoperable standard across observability platforms.

• Expanded and stabilized semantic conventions OpenTelemetry continues to refine and expand its semantic conventions. Those conventions are used by the developpers to standardize the information sent to Opentelemetry. This includes stabilizing attributes like code.*, database-related tags, and introducing conventions for Kubernetes resources, messaging systems, RPC communication, and profiling events. These conventions help ensure consistent labeling and correlation across all telemetry signals. Examples :

Conclusion

OpenTelemetry continues to grow as a key pillar of cloud-native observability. With profiling support, the upcoming logs API, adoption of the OTLP/HTTP protocol, and deeper integration into clouds and frameworks, OpenTelemetry is on track to become the unified standard for cross-platform telemetry.

The community is highly active, with many contributors, and the tools are evolving to make adoption easier and more valuable in production environments.

Further resources:

• https://opentelemetry.io/

• https://bindplane.com/blog/opentelemetry-in-production-a-primer/

• https://bindplane.com/blog/kubecon-europe-2025-opentelemetry-recap-from-london

In this article, we’ll share a high-level overview of the main trends we observed. We’ll publish more detailed articles on some of these topics in the upcoming weeks.

Trend number 1: Platform Engineering

Platform engineering was everywhere this year. Organizations are looking for scalable ways to streamline their development and operations to have a better global approach. A lot of new tools are coming and trying to meet the needs of this specific topic.

First, Backstage, an open-source developer portal, stood out as a key enabler. Paired with Crossplane, it allows platform teams to expose complex infrastructure as simple, self-service components. The goal is to create a modular, declarative, and developer-friendly experience.

Trend number 2: OpenTelemetry has become the Standard

OpenTelemetry is becoming the cornerstone of observability. It’s now the second most contributed CNCF project after Kubernetes and was featured in multiple sessions. It’s literaly everywhere.

Key takeaways:

• OpenTelemetry Protocol (OTLP) over HTTP is becoming the default.

• Logs, metrics, traces, and even profiling are now unified under one framework.

• New semantic conventions are being defined, including for CI/CD pipelines.

• Native support is growing across cloud providers like AWS, Azure, and GCP.

• Certification programs and new discovery methods (e.g., Kubernetes annotation-based discovery) are being introduced.

We had the opportunity to talk a lot with people working with Dash0, a tool that leverages OpenTelemetry for vendor-neutral observability.

Trend number 3: Cloud-Native Security

Security, as always, is a hot topic, especially when it comes to secrets management and policy automation.

GitGuardian revealed shocking statistics: over 35,000 hardcoded secrets were found across 180,000 public Docker images. This shows how critical secret scanning and proper image hygiene have become.

Policy-as-code also drew attention, with tools like Kyverno or Gatekeeper offering flexible and scalable ways to manage RBAC, network rules, and admission controls through Kubernetes.

We also discovered tools like Tetragon, which brings powerful eBPF-based observability and runtime security to Kubernetes.

Global Trend: AI is everywhere

Like already said and written last year, the AI acts as a common aspect in addition to all the tendencies. In London this year, the AI and its integration into developer workflows was a central theme. An impressive part of the sessions touched on AI/ML topics, with more than 120 sessions focusing directly on areas like Kubernetes for AI workloads and integrating AI with cloud-native technologies.

Large Language Models (LLMs) and generative AI were particularly highlighted too. Tools such as Ray and Kubeflow were discussed for their roles in scaling AI applications and managing end-to-end machine learning workflows on Kubernetes.

What’s Next?

KubeCon 2025 confirmed the direction the industry is heading: toward platform-centric, observable, secure, and AI-augmented cloud-native environments.

This is just the beginning and we’ll be diving deeper into various topics in future articles. Plaform engineering, OpenTelemetry…stay tuned for more!

Personnal conclusion

It’s been a real pleasure meeting so many people involved in the CNCF ecosystem. We encourage everyone to take part in the upcoming KubeCons in Europe (Amsterdam 2026 and Barcelona 2027). It’s truly impressive to see how much knowledge is shared at these events. To illustrate that, we’ll simply leave you with this image…

This is where the Azure Landing Zone (ALZ) Accelerator comes in. It provides an automated, Microsoft-recommended way to establish the foundational structure of an Azure tenant, including:

• Management group hierarchy – Organizing subscriptions under a structured governance model.

• Policy and compliance enforcement – Predefined Azure Policies to maintain security and best practices.

• Identity and access control – Implementing RBAC and integrating Azure AD for governance.

• Networking and connectivity – Standardized hub-and-spoke networking configurations.

By leveraging the ALZ Accelerator, organizations can set up their Azure environment efficiently using Infrastructure as Code (IaC) with Terraform and Azure DevOps. This ensures a repeatable, scalable, and compliant deployment, making it easier to manage and extend Azure environments over time.

In this blog post, we’ll walk through how to use the ALZ Accelerator in Azure DevOps to bootstrap an enterprise-ready Azure tenant.

Why Use the ALZ Accelerator to build your tenant foundations?

The ALZ Accelerator helps organizations lay down the groundwork for a well-governed Azure tenant by automating key elements such as:

Management Group Hierarchy

Instead of manually structuring management groups, ALZ defines a pre-configured model based on Microsoft best practices:

• Root Management Group (Tenant Root Group)

• Platform Management Groups (Identity, Connectivity, Management)

• Landing Zone Management Groups (Production, Non-Production, Sandbox)

This helps enforce governance while keeping workloads isolated.

Policy-Driven Compliance

The ALZ Accelerator automatically deploys Azure Policy Assignments to enforce security, monitoring, and cost management best practices.

Examples include:

• Tag enforcement for resource consistency.

• Security baselines for subscriptions.

• Networking policies to prevent non-compliant configurations.

Identity and Access Management (IAM)

With built-in role-based access control (RBAC) configurations, the ALZ Accelerator helps define:

• Scoped permissions for different teams.

• Privileged Identity Management (PIM) recommendations.

• Integration with Microsoft Entra ID (Azure AD) for authentication.

Standardized Networking Architecture

ALZ supports multiple network topologies (hub-and-spoke, virtual WAN) to create a secure and scalable foundation for application workloads.

What does the ALZ Accelerator deploy?

Once the ALZ Accelerator is deployed using Terraform and Azure DevOps, the following foundational elements are created within your Azure tenant:

• Management Groups – A hierarchical structure organizing Azure subscriptions for governance and isolation.

• Azure Policies and Initiatives – Predefined governance rules ensuring compliance with security best practices.

• Role-Based Access Control (RBAC) Assignments – Scoped access control across management groups and subscriptions.

• Subscription Assignments – Assigning subscriptions to management groups for logical separation.

• Networking Baseline – Configurations for a hub-and-spoke or virtual WAN networking model.

• Logging and Monitoring Configurations – Integrations with Azure Monitor and Log Analytics for visibility.

With these elements in place, organizations have a ready-to-use tenant that enforces security, governance, and operational best practices across all workloads.

You can find the detailed list of deployed items here: https://azure.github.io/Azure-Landing-Zones/accelerator/#azure-devops

Step-by-step deployment: ALZ Accelerator via Azure DevOps

The usage of the accelerator is split in 4 phases: Planning, Prerequisites, Bootstrap and Run.

The first phase being the planification (naming convention, network choices, design of the architecture and tools), we will skip it. However, it’s a very important phase and you should not take it lightly.

Here a quick schema of what is deployed at each step:

Phase 1 - Prerequisites

Tools

• PowerShell 7.4 (or newer): Link

• Azure CLI 2.55.0 (or newer): Link

• Git (any supported version): Link

You will also need open access to the internet to download tools, terraform providers and connect to Azure and your Version Control System.

Azure subscriptions and permissions

In addition to the tools, you’ll need in your tenant to have a minimum of:

• Management group: In which management group will be place the Landing Zone Structure. In our example we’ll use the Tenant Root Group.

• Azure Subscriptions: Depending on the scenario you choose, you’ll need to manually create subscriptions. In our case, we need 3 Subscriptions :

  1. Management: This is used to deploy the bootstrap and management resources, such as log analytics and automation accounts.

  2. Identity: This is used to deploy the identity resources, such as Azure AD and Microsoft Entra Domain Services (formerly Azure AD DS) .

  3. Connectivity: This is used to deploy the hub networking resources, such as virtual networks and firewalls.

• Management Group Subscription Placement: Make sure the subscriptions you just created are placed in the management group you choose previously

• Azure Authentication and Permissions: To interact with Azure, you need either an Azure User Account or a Service Principal. Both options are drastically different in implementation so be sure to check the official documentation. In both case, you’ll need those permissions:

  • Owner on your chosen parent management group.

  • Owner is required as this account will be granting permissions for the identities that run the management group deployment. Those identities will be granted least privilege permissions.

  • Owner on each of your 3 Azure landing zone subscriptions.

If you choose the authentification with a User Account, you can check your permissions by running this commands:

az login
az account set --subscription "<subscription id of your management subscription>"
az account show

Version control systems

Of course, a very important choice to make is which platform you’ll use store and run your configuration. Your options are Azure DevOps, GitHub or Local File System. For our demonstration we choose Azure DevOps but feel free to check the official documentation for others platforms.

Azure DevOps prerequisites

You’ll need to setup billing for you Azure DevOps organization and billing for parallel jobs. Check Microsoft documentation for detailed steps.

Azure DevOps Personal Access Token (PAT)

A PAT is necessary for the accelerator script to create the base project for your ALZ. Here is the scope it needs:

• Agent Pools: Read & manage

• Build: Read & execute

• Code: Full

• Environment: Read & manage

• Graph: Read & manage

• Pipeline Resources: Use & manage

• Project and Team: Read, write & manage Service

• Connections: Read, query & manage

• Variable Groups: Read, create & manage

A second PAT might be needed if you use self-hosted runners (not our case).

Phase 2 - Bootstrap

First things first, install the ALZ PowerShell module:

Install-Module -Name ALZ

The accelerator consist of running a one-time command that will deploy the base architecture. So the next step is to build your configurations files (Bootstrap configuration and deploy configuration). You’ll need to know which IaC (Infrastructure a Code) tool, in our case we use Terraform but you could choose Bicep.

Now it's getting a little bit more tricky, that the templating and specifications of the different configurations files. Please, keep in mind that you usecase might differ from ours so check the official documentation to be sure.

Create the arborescence of the accelerator project, as it a “one-time” execution, you don’t need to git it.

New-Item -ItemType "file" c:\accelerator\config\inputs.yaml -Force
New-Item -ItemType "file" c:\accelerator\config\platform-landing-zone.tfvars -Force  # Exclude this line if using FSI or SLZ starter modules
New-Item -ItemType "directory" c:\accelerator\output

For the bootstrap, edit the inputs.yaml file.

# For detailed instructions on using this file, visit:
# https://aka.ms/alz/accelerator/docs

# Basic Inputs
iac_type: "terraform"
bootstrap_module_name: "alz_azuredevops"
starter_module_name: "platform_landing_zone"

# Shared Interface Inputs
bootstrap_location: "<region-1>"
starter_locations: ["<region-1>", "<region-2>"]
root_parent_management_group_id: ""
subscription_id_management: "<management-subscription-id>"
subscription_id_identity: "<identity-subscription-id>"
subscription_id_connectivity: "<connectivity-subscription-id>"

# Bootstrap Inputs
azure_devops_personal_access_token: "<token-1>"
azure_devops_agents_personal_access_token: "<token-2>"
azure_devops_organization_name: "<azure-devops-organization>"
use_separate_repository_for_templates: true
bootstrap_subscription_id: ""
service_name: "alz"
environment_name: "mgmt"
postfix_number: 1
azure_devops_use_organisation_legacy_url: false
azure_devops_create_project: true
azure_devops_project_name: "<azure-devops-project-name>"
use_self_hosted_agents: true
use_private_networking: true
allow_storage_access_from_my_ip: false
apply_approvers: ["<email-address>"]
create_branch_policies: true

# Advanced Inputs
bootstrap_module_version: "latest"
starter_module_version: "latest"
#output_folder_path: "/accelerator/output"

For the deploy, you can choose between multiple scenarios template to help you to build your configuration. In this example, we choose the “Single-Region Hub and Spoke Virtual Network with Azure Firewall”. Due to the size and variability of the file, please check the official template for customization. It refers as platform-landing-zone.tfvars file.

Once you put the right informations and configuration in those files, you’re ready to run the bootstrap!

Here the command to run it:

Deploy-Accelerator `
  -inputs "c:\accelerator\config\inputs.yaml", "c:\accelerator\config\platform-landing-zone.tfvars" `
  -output "c:\accelerator\output"

This might take a while but you' will see a Terraform Init and plan, If you’re happy with it, just hit it and it will apply the bootstrap.

Now, you have all the pieces ready to build your Tenant ALZ. Your DevOps organization is ready to run. 🎉

Phase 3 - Run

Last part of this implementation is the deployment of the “actual” ALZ. To do this, you’ll need to run the different pipeline that the Bootstrap juste created in your Azure DevOps project.

Running the ALZ Pipelines in Azure DevOps

1. Navigate to dev.azure.com and sign in to your organization.

2. Navigate to your project.

3. Click Pipelines in the left navigation.

4. Click the 02 Azure Landing Zones Continuous Delivery pipeline.

5. Click Run pipeline in the top right.

6. Take the defaults and click Run.

7. Your pipeline will run a plan. If you provided apply_approvers to the bootstrap, it will prompt you to approve the apply stage.

8. Your pipeline will run an apply and deploy an Azure landing zone based on the starter module you choose.

Now, your ALZ should be populated with those resources:

• Management group hierarchy

• Subscription assignments

• Azure policies and RBAC

• Networking configurations

Conclusion

Using the ALZ Accelerator with Terraform and Azure DevOps provides a repeatable, automated way to establish a secure and scalable Azure tenant foundation.

• ✅ Governance & Compliance – Predefined management groups, IAM roles, and Azure Policies.

• ✅ Security Best Practices – RBAC and policy-driven controls for workloads.

• ✅ Scalability – A structured framework that supports future growth.

Further reading and official resources

For additional details and customization, refer to the following Microsoft documentation:

🔗 Azure Landing Zones Accelerator Overview – https://azure.github.io/Azure-Landing-Zones/
🔗 Terraform Implementation Guide for ALZ – https://github.com/Azure/terraform-azurerm-alz
🔗 Azure Enterprise-Scale Documentation – https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/enterprise-scale/
🔗 Azure Landing Zones GitHub Repository – https://github.com/Azure/Azure-Landing-Zones

What are interactive tasks?

In a pipeline, interactive tasks/tests are automated processes that require a graphical user interface to simulate user interactions with an application. These tests often:

• Validate GUI components.

• Require input simulations like mouse clicks or keystrokes.

• Run on applications that depend on display rendering.

While headless environments work well for many scenarios, certain use cases, such as UI testing or hardware integration tests, necessitate a fully interactive session.

How to configure it?

We first tried to add a VMSS as Azure Devops Agents and use the “tick” to run it as interactive and use a standard W10 / server VMSS to run the tasks.

Sadly, we’ve never been able to run interactive tests with this option(or at least for the moment ! Keep an eye on it if there’s something new / working better that could be coming soon)…

To address this, we had to configure interactive agents on a Windows 10 Virtual Machine or a Virtual Machine Scale Set (VMSS) to run pipeline tasks that require a graphical user interface (GUI). The pool type has been changed (we cannot refer to a “simple” VMSS and we had to choose the “self hosted” agent pool type instead.)

Prepare your Windows 10 VM/VMSS Instances for Interactive tasks

To set up interactive tasking on VM/ VMSS instances, we’ll use the following custom script at VM / VMSS startup (you can configure it through the Azure UI or directly with IaC, referencing the script in the “custom script” extension). This scrpit (that will be run at VM’s creation) ensures the system remains logged in, avoiding interruptions like sleep mode or screen savers, and installing the Azure DevOps agent with interactive mode enabled. The downside of this method is that we’ll have to add our agent as “self hosted” , even if we’re using a VMSS on Azure.

Custom Script for Interactive Agent Configuration

Warning : this script will setup the machine as always on, always run and always logged in with an admin user. Customize it if such features aren’t needed !

# Creds: User / pass
$user = "${var.admin_username}"
$password = "${var.admin_password}"

# Disable standby mode
powercfg -change -standby-timeout-ac 0
powercfg -change -monitor-timeout-ac 0

# Disable automatic session close
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" -Name "ScreenSaveActive" -Value "0" 
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" -Name "ScreenSaverIsSecure" -Value "0"

# Disable hibernation
powercfg -h off
powercfg -setacvalueindex SCHEME_CURRENT SUB_SLEEP STANDBYIDLE 0
powercfg -setdcvalueindex SCHEME_CURRENT SUB_SLEEP STANDBYIDLE 0
powercfg -setacvalueindex SCHEME_CURRENT SUB_SLEEP HIBERNATEIDLE 0
powercfg -setdcvalueindex SCHEME_CURRENT SUB_SLEEP HIBERNATEIDLE 0

# Never stop the display
powercfg -change -monitor-timeout-ac 0
powercfg /change standby-timeout-dc 0

powercfg -setacvalueindex SCHEME_CURRENT SUB_SLEEP STANDBYIDLE 0
powercfg -setdcvalueindex SCHEME_CURRENT SUB_SLEEP STANDBYIDLE 0
powercfg -setacvalueindex SCHEME_CURRENT SUB_DISK DISKIDLE 0
powercfg -setdcvalueindex SCHEME_CURRENT SUB_DISK DISKIDLE 0


# Disable the NIC-standby
Get-NetAdapter | ForEach-Object { Set-NetAdapterPowerManagement -Name $_.Name -AllowWakeArmed Only -WakeOnMagicPacket $true -Force }

# Auto login for interactive sessions
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" -Name "AutoAdminLogon" -Value "1"
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" -Name "DefaultUserName" -Value $user
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" -Name "DefaultPassword" -Value $password

# Disable display of error reporting
Set-ItemProperty -Path "HKLM:\Software\Microsoft\Windows\Windows Error Reporting" -Name "DontShowUI" -Value 1
Set-ItemProperty -Path "HKCU:\Software\Microsoft\Windows\Windows Error Reporting" -Name "DontShowUI" -Value 1

# Disable OOBE notifs
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" -Name "EnableOOBE" -Value 0 -Force
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" -Name "NoOOBE" -Value 1 -Force

# Disable LUA and admin approvals for software install
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" -Name "EnableLUA" -Value 0 -Force
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" -Name "ConsentPromptBehaviorAdmin" -Value 0 -Force

# Disable OOBE privacy settings screen
$regKey = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\OOBE"
Set-ItemProperty -Path $regKey -Name "PrivacySetting" -Value 0 -Force
Set-ItemProperty -Path $regKey -Name "NoPrivacySettings" -Value 1 -Force
New-ItemProperty -Path $regKey -Name "PrivacyConsentStatus" -Value 1 -PropertyType DWORD -Force 
New-ItemProperty -Path $regKey -Name "SkipMachineOOBE" -Value 1 -PropertyType DWORD -Force 
New-ItemProperty -Path $regKey -Name "ProtectYourPC" -Value 3 -PropertyType DWORD -Force 
New-ItemProperty -Path $regKey -Name "SkipUserOOBE" -Value 1 -PropertyType DWORD -Force 

# Disable "Send Microsoft info about your device" at startup
Set-ItemProperty -Path $regKey -Name "OOBECompleted" -Value 1 -Force

# Disable post upgrade reboot
Set-ItemProperty -Path "HKLM:\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU" -Name "NoAutoRebootWithLoggedOnUsers" -Value 1 -Force

# Force activating user account at startup (avoid login prompt) 
Set-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" -Name "DontDisplayLastUserName" -Value 1 -Force


# Variables for the agent
$AgentURL = "https://vstsagentpackage.azureedge.net/agent/4.248.1/vsts-agent-win-x64-4.248.1.zip"
$AgentDirectory = "C:\agentinteractive"
[System.Environment]::SetEnvironmentVariable("AGENT_USERCAPABILITY_Interactive", "true", [System.EnvironmentVariableTarget]::Machine)

# Download / install agent
New-Item -ItemType Directory -Path $AgentDirectory -Force
Invoke-WebRequest -Uri $AgentURL -OutFile "$AgentDirectory\agent.zip"
Expand-Archive -Path "$AgentDirectory\agent.zip" -DestinationPath $AgentDirectory

# Agent configuration
cd $AgentDirectory
.\config.cmd --unattended --addcap InteractiveMode true --auth pat --token "${var.token}" --pool YOUR_POOL --url https://dev.azure.com/yourOrganization --agent $env:COMPUTERNAME-interactive --acceptTeeEula --runAsAutoLogon --windowsLogonAccount $user --windowsLogonPassword $password

#restart 
Restart-Computer -Force

Key Points in the Script

1. Preventing Sleep and Timeout:

   • Sleep and display timeouts are disabled using the powercfg command.

   • Screen savers and session logouts are also disabled.

2. Auto-Login Configuration:

   • Ensures the VMSS instance automatically logs in with the specified user.

3. Azure DevOps Interactive Agent:

   • Downloads and installs the Azure DevOps agent.

   • Configures it in interactive mode, essential for UI tests.

   • Important: You’ll need to give a token in order to be able to communicate with Azure Devops.

4. Disabling Unnecessary Prompts:

   • Removes OOBE and privacy prompts to streamline the user experience.

5. Restart to Apply Settings:

   • A restart ensures all configurations take effect.

Testing Interactive Agents

Once the script is executed:

1. The VM / VMSS instance will automatically reboot / log in with the specified user.

2. The Azure DevOps agent will register in interactive mode.

3. UI tests can now run seamlessly on the VMSS instance.

To verify:

• Check the Azure DevOps agent pool to confirm the agent is online.

• Run a UI test pipeline and monitor the execution.

If the registration worked, we can see our agents in the agent list of the agent-pool.

Exemple of interactive test

pool:

  name: "YourPool"

steps:

  - task: PowerShell@2
    displayName: "Open Notepad, write 'Hello' and take a screenshot"
    inputs:
      targetType: "inline"
      script: |
        # Launch Notepad
        Start-Process notepad.exe
        Start-Sleep -Seconds 2  

        # write something
        Add-Type -AssemblyName System.Windows.Forms
        [System.Windows.Forms.SendKeys]::SendWait("Hello")

        # Take a screenshot

        $screenshot = [System.Drawing.Bitmap]::new([System.Windows.Forms.Screen]::PrimaryScreen.Bounds.Width, [System.Windows.Forms.Screen]::PrimaryScreen.Bounds.Height)
        $graphics = [System.Drawing.Graphics]::FromImage($screenshot)
        $graphics.CopyFromScreen([System.Drawing.Point]::Empty, [System.Drawing.Point]::Empty, $screenshot.Size)
        $screenshotPath = "$env:BUILD_ARTIFACTSTAGINGDIRECTORY\screenshot.png"
        $screenshot.Save($screenshotPath, [System.Drawing.Imaging.ImageFormat]::Png)
        Write-Output "Screenshot saved to $screenshotPath"

        # Close notepad
        Stop-Process -Name notepad -Force

Conclusion

Running interactive tests on Windows 10 VMSS requires careful configuration to ensure the environment stays active and user session remains open. The script provided simplifies this setup, enabling robust automation of UI-based tasks. By leveraging Azure DevOps agents in interactive mode, you can confidently execute tests that replicate real user interactions.

So, we automated it. By combining the power of Large Language Models (LLMs) with Kubernetes' diagnostic tools, we’ve built a system that lets AI do the boring work while you focus on innovation.

Let’s dive into what we built, how we built it, and why this is your next must-have tool.

Overview

1. The Problem

Diagnosing Kubernetes clusters is a process filled with repetition:

• Running kubectl get pods again and again.

• Parsing logs line by line for the needle in the haystack.

• Writing mental notes about what went wrong, only to rewrite them as fixes.

The real problem? It’s not hard—it’s boring and extremely time-consuming. When you’re firefighting production issues, the last thing you need is a manual slog through the diagnostics process.

2. Our Solution

We built a Kubernetes diagnostic assistant powered by OpenAI’s LLMs via the use of the LangChain project. It automates repetitive tasks like:

• Identifying problematic pods (CrashLoopBackOff, Failed, Pending).

• Analyzing logs, events, and dependencies.

• Summarizing findings into actionable solutions.

In short, it turns Kubernetes troubleshooting into a hands-off operation.

3. Our Objectives

1. Eliminate the Boredom: Let AI handle repetitive commands and diagnostics.

2. Save Time: Quickly identify and resolve critical issues.

3. Actionable Insights: Provide a clear summary with solutions—not just a data dump.

4. Automation at Scale: Diagnose clusters, no matter how large, without manual intervention.

Technical Implementation

1. Automating Kubernetes Diagnostics with Python and LangChain

Our tool uses OpenAI’s GPT-3.5 model to interact with Kubernetes. Here’s the workflow:

Key Features

1. LLM-Powered Command Generation: The AI proposes and executes Kubernetes commands dynamically, such as:

    kubectl get pods --all-namespaces --field-selector=status.phase!=Running --kubeconfig=~/.kube/config --insecure-skip-tls-verify
   

2. Iterative Diagnostics: Each problematic pod is analyzed for:

   • Logs (kubectl logs <pod>).

   • Events (kubectl describe pod <pod>).

   • Dependencies (ConfigMaps, Secrets, etc.).

3. Actionable Summaries: AI synthesizes findings into a Markdown table with:

   • Pod Name

   • Namespace

   • Status

   • Cause

   • Solution

Snippet of the Main Function

def monitor_kubernetes():
    print("\n--- Starting Kubernetes Dynamic HealthCheck ---\n")
    executed_commands = interact_with_llm()
    print("\n--- Executed Commands ---\n")
    for cmd in executed_commands:
        print(f"{cmd}: Success" if cmd else "Failed")
    print("\n--- HealthCheck Completed ---\n")

2. Environment Configuration

We keep sensitive information like API keys and kubeconfig paths in an .env file:

OPENAI_API_KEY="XXXXXXXXXXXXXXXXXXXXXXXXXXX"
KUBECONFIG="~/.kube/config"

3. Dependency Management

The tool uses:

• LangChain: To integrate the LLM and manage interactions.

• pandas: For formatting findings.

• urllib3: To handle Kubernetes HTTPS connections.

• rich: To improve terminal output.

Install dependencies via:

pip install -r requirements.txt

4. Git Hygiene

A clean .gitignore ensures no sensitive files (like .env) are committed:

# Ignore virtual environment directories
env/
.env
venv/
.venv/

How It Works

1. Step-by-Step Workflow

1. Let AI Take the Lead: The LLM proposes commands to diagnose the cluster:

    kubectl get pods --all-namespaces --field-selector=status.phase!=Running --kubeconfig=~/.kube/config --insecure-skip-tls-verify
   

2. Iterative Analysis:

   • For each problematic pod, the AI:

     • Retrieves logs.

     • Examines events.

     • Checks dependencies.

   • Findings are returned as a Markdown table.

3. Summarized Output: The AI synthesizes results into an actionable summary, like:

   | Pod Name | Namespace | Status | Cause Identified | Solution Proposed | | --- | --- | --- | --- | --- | | my-pod-1 | default | CrashLoopBackOff | Missing ConfigMap | Add the required ConfigMap | | my-pod-2 | kube-system | Pending | Node resource issue | Allocate more resources to the cluster |

4. Output Example:

    [DEBUG] LLM Proposes: ### Anomalies Detected in the Kubernetes Cluster:
   
    | Pod Name                  | Namespace         | Status           | Cause Identified                  | Solution Proposed                                      |
    |---------------------------|-------------------|------------------|-----------------------------------|--------------------------------------------------------|
    | loki-test-0               | loki              | CrashLoopBackOff | Failed to start properly          | Check pod logs for specific error messages and fix     |
    | k8s-monitor-54c8bfc448-x4bjh | monitoring      | CrashLoopBackOff | Continuous crashing               | Investigate logs and events for root cause and resolve |
    | loki-test-promtail-8bvb5  | loki              | Running          | -                                 | -                                                      |
    | loki-test-promtail-kv2cd  | loki              | Running          | -                                 | -                                                      |
    | loki-test-promtail-x6jsm  | loki              | Running          | -                                 | -                                                      |
   
    ### Recommendations:
    1. **loki-test-0 (loki)**:
       - **Status**: CrashLoopBackOff
       - **Cause**: Pod is failing to start properly.
       - **Solution**: Check pod logs for specific error messages and take necessary actions to resolve the issue.
   
    2. **k8s-monitor-54c8bfc448-x4bjh (monitoring)**:
       - **Status**: CrashLoopBackOff
       - **Cause**: Pod is continuously crashing.
       - **Solution**: Investigate logs and events for the root cause of the crashes and take corrective actions.
   
    3. **loki-test-promtail-8bvb5, loki-test-promtail-kv2cd, loki-test-promtail-x6jsm (loki)**:
       - **Status**: Running
       - **Cause**: No issues identified currently.
       - **Solution**: Monitor for any future issues.
   
    Ensure to address the critical anomalies promptly to stabilize the cluster.
   
    [DEBUG] No new commands or synthesis proposed. Ending interaction.
   

Reflections and Challenges

1. What Worked

• Boredom Eliminated: AI took over repetitive tasks, freeing us to focus on value-added work.

• Accuracy: The LLM excelled at identifying and prioritizing issues.

• Time Saved: Diagnosis time for complex clusters dropped significantly.

2. Challenges

• Parsing Logs: Kubernetes errors can be cryptic, requiring prompt tuning to improve AI responses.

• Scalability: Handling large clusters requires additional optimization.

Get Started

Ready to automate Kubernetes diagnostics? Clone the repo and start today:

git clone https://github.com/cisel-dev/k8s-healthcheck-llm.git
python k8s_agent.py

A Humble Beginning

We know this tool isn’t perfect—far from it. It’s a starting point, a framework to help you tackle the often tedious task of Kubernetes diagnostics. While the functionality is practical and the prompts are designed to cover common use cases, every environment is unique, and so are its challenges.

We encourage you to adjust the prompts, tweak the workflows, the model, and extend the capabilities to fit your specific needs. This is your opportunity to make it better—more accurate, more insightful, and more tailored to your cluster.

So, get creative, dive in, and make it yours. Let’s automate the boring stuff.

In this example, we use Terraform to automate the creation of Azure Storage Accounts and their containers for each environment (production, pre-production, and non-production). The goal is to implement a scalable, secure, and segmented architecture where each environment is isolated and has its own User Assigned Managed Identity. This identity enables precise access control, which is essential for meeting enterprise security and compliance requirements.

Final Objective: Target Structure for Storage Accounts and Managed Identities

The final objective of this configuration is to create a structure for each environment with storage accounts and dedicated storage containers, each associated with a specific managed identity. This setup is designed to meet the following needs:

• Data Security and Isolation: Each environment (production, pre-production, non-production) has its own storage account, containers, and a dedicated managed identity to isolate data and control access.

• Flexibility and Scalability: Using dynamic variables and loops (for_each), this configuration allows us to add new environments or containers easily without complex code modifications.

• Compliance and Best Practices: Separating resources by environment with dedicated identities aligns with security standards for access management and data isolation.

Structure and Configuration: Overview

The target structure is defined in the terraform.tfvars file, which specifies the storage account names and containers for each environment, along with the names of the managed identities:

# Storage accounts and their containers, must be ordered by environment
storage_accounts = {
  "prd" = {
    sa_name    = "saprojectprd001"
    containers = ["container-xyz-prd"]
  }
  "ppd" = {
    sa_name    = "saprojectppd001"
    containers = ["container-xyz-ppd"]
  }
  "npd" = {
    sa_name    = "saprojectnpd001"
    containers = ["container-xyz-dev", "container-xyz-qas", "container-xyz-sbx", "container-xyz-poc"]
  }
}

# User Assigned Managed Identity
identity_names = {
  "prd" = "id-project-prd-chn-001"
  "ppd" = "id-project-ppd-chn-001"
  "npd" = "id-project-npd-chn-001"
}

• Production (prd): A storage account (saprojectprd001) with a container for production data (container-xyz-prd) and a specific managed identity (id-project-prd-chn-001).

• Pre-production (ppd): A storage account (saprojectppd001) with a container for pre-production data (container-xyz-ppd) and a specific managed identity (id-project-ppd-chn-001).

• Non-Production (npd): A storage account (saprojectnpd001) with multiple containers (container-xyz-dev, container-xyz-qas, etc.) for development and testing environments and a dedicated managed identity (id-project-npd-chn-001).

This organization provides environment-specific isolation while centralizing resources for non-production environments.

Implementation in Terraform

1. Creating Storage Accounts and Containers with for_each

We use the for_each loop in Terraform to automatically deploy Storage Accounts and their containers for each environment defined in terraform.tfvars.

Deploying Storage Accounts

The following code shows the creation of storage accounts, where each environment has its own account:

resource "azurerm_storage_account" "sa_account" {
  for_each                        = var.storage_accounts
  name                            = each.value.sa_name
  location                        = azurerm_resource_group.this_001.location
  resource_group_name             = azurerm_resource_group.this_001.name
  account_tier                    = var.sa_account_tier
  account_replication_type        = var.sa_account_replication_type
  public_network_access_enabled   = var.sa_account_public_network_access_enabled
  allow_nested_items_to_be_public = var.st_nested_public
  min_tls_version                 = var.st_min_tls_version

  queue_properties {
    logging {
      delete                = var.st_logging_delete
      read                  = var.st_logging_read
      write                 = var.st_logging_write
      version               = var.st_logging_version
      retention_policy_days = var.st_logging_retention
    }
  }

  blob_properties {
    delete_retention_policy {
      days = var.st_soft_delete_retention
    }
  }

  tags = {
    Projet       = var.tags_project
    Facturation  = var.tags_facturation
    Souscription = var.tag_sub_re_chn_dig
  }
}

Creating Containers in Each Storage Account

For the containers, we use flatten to combine each storage account and its containers into a single, manageable list:

locals {
  storage_containers = flatten([
    for env, sa in var.storage_accounts : [
      for container in sa.containers : {
        sa_name   = sa.sa_name
        container = container
        env       = env
      }
    ]
  ])
}

resource "azurerm_storage_container" "sc_account" {
  for_each = { for idx, container in local.storage_containers : "${container.sa_name}-${container.container}" => container }

  name                  = each.value.container
  storage_account_name  = azurerm_storage_account.sa_account[each.value.env].name
  container_access_type = "private"

  depends_on = [
    azurerm_storage_account.sa_account,
    azurerm_private_endpoint.st_pep,
    azurerm_private_dns_a_record.st-record
  ]
}

The for_each loop automatically deploys each container for the corresponding storage account based on the environment.

2. Creating and Assigning Managed Identities per Environment

For each environment, a User Assigned Managed Identity is created and assigned to the corresponding resources.

Defining Managed Identities

Each managed identity is created according to its name specified in identity_names:

resource "azurerm_user_assigned_identity" "this" {
  for_each            = var.identity_names
  resource_group_name = azurerm_resource_group.this_001.name
  location            = azurerm_resource_group.this_001.location
  name                = each.value
}

Assigning Roles to Identities

Finally, we assign the necessary roles to each identity, enabling access to storage containers as required for each environment.

• Storage Blob Data Contributor: This allows each identity to access storage blobs.

resource "azurerm_role_assignment" "storage_blob_data_contributor" {
  for_each = {
    for env, id_name in var.identity_names : env => flatten([
      for container in var.storage_accounts[env].containers : {
        sa_name   = var.storage_accounts[env].sa_name
        container = container
      }
    ])
  }

  principal_id         = azurerm_user_assigned_identity.this[each.key].principal_id
  scope                = azurerm_storage_account.sa_account[each.key].id
  role_definition_name = "Storage Blob Data Contributor"

  depends_on = [
    azurerm_storage_account.sa_account,
    azurerm_storage_container.sc_account
  ]
}

Conclusion

In summary, this Terraform configuration allows us to deploy Storage Accounts and Managed Identities in an automated and segmented way for each environment. It ensures data separation, access isolation, and provides maximum flexibility to add or modify environments effortlessly. This architecture offers a robust and scalable solution for managing Azure resources and access controls.

What is MegaLinter?

MegaLinter is an open-source tool designed to automate static analysis and linting for your code and supports a lot of programming languages, file formats and configuration tools. If you're working with JavaScript, Python, Go, or even YAML, Megalinter will find a way to cover it properly and ensure that your code is complliant to best practices.

Why to choose MegaLinter?

Like mentionned before, MegaLinter stands out by supporting various languages and formats. This means you can manage a complex project with multiple tech stacks without having to integrate several specific linting tools.

Integrating MegaLinter into your CI/CD pipeline is easy. Whether you're using GitHub Actions, Jenkins, Gitlab, or Azure DevOps, MegaLinter fits into your workflow to analyze your code .

The best tool to be sure that your installation is ok is their assisted installation tool! It automaticlly generates configuration files!

more details here : https://megalinter.io/latest/install-assisted/

When it's installed (beware that you might have to install nodeJS on your machine before), you can simply run

npx mega-linter-runner --install and it will ask you questions and generate a .mega-linter.yml file with the configurations you need.

Now we can integrate it on our pipeline

Exemple with an Azure Devops yaml file:

  # Run MegaLinter to detect linting and security issues
  - job: MegaLinter
    pool:
      vmImage: ubuntu-latest #we'll use an Ubuntu VM
    steps:
      # First we need to checkout the repo
      - checkout: self

      # Pull MegaLinter docker image
      - script: docker pull oxsecurity/megalinter:v8
        displayName: Pull MegaLinter

      # Run the image and use the Git Token.

      - script: |
          docker run -v $(System.DefaultWorkingDirectory):/tmp/lint \
            --env-file <(env | grep -e SYSTEM_ -e BUILD_ -e TF_ -e AGENT_) \
            -e GIT_AUTHORIZATION_BEARER=$(System.AccessToken) \
            oxsecurity/megalinter:v8
        displayName: Run MegaLinter

      # Upload MegaLinter reports
      - task: PublishPipelineArtifact@1
        condition: succeededOrFailed()
        displayName: Upload MegaLinter reports
        inputs:
          targetPath: "$(System.DefaultWorkingDirectory)/megalinter-reports/"
          artifactName: MegaLinterReport

Output example (in this case using terraform linters):

In addition, the MegaLinter documentation points to a really good and very well detailled example to have it perfectly integrated in azure devops. When you have a pull request, this shows you how to automatically trigger a code analysis with MegaLinter.

https://github.com/DonKoning/megaLinter

If you would like to have great example of github actions or even local use of Megalinter, please go to their official documentation:

https://megalinter.io/latest/install-github/

Customization

The second aspect is that Megalinter is highly customizable. You can enable or disable linters, exclude files simply by using a ".mega-linter.yml" file or via environment variables (FILTER_REGEX_INCLUDE:()and the opposite, FILTER_REGEX_EXCLUDE: ()). You can decide if you let the tool apply fixes or not (direct commit, new PR, etc).

more details on their Github repo:

https://github.com/oxsecurity/megalinter/blob/main/.mega-linter.yml

Detailed Reports

Once configured, MegaLinter generates detailed and clear reports. These reports allow you to identify issues in your code and fix them easily. It's an excellent way to keep your codebase clean and compliant.

THE cool feature regarding these reports is that you can generate them in almost all formats ! The full list is here : https://megalinter.io/latest/reporters/

Conclusion

Adopting MegaLinter within your CI/CD pipeline is a strategic move and can transform your code quality to make it more secure, more readable and more best-practices compliant.

Microsoft patterns & practices : https://github.com/mspnp/AzureNamingTool

The Azure Naming Tool was created to help administrators define and manage their naming conventions, while providing a simple interface for users to generate a compliant name. The tool was developed using a naming pattern based on Microsoft's best practices. Once an administrator has defined the organizational components, users can use the tool to generate a name for the desired Azure resource.

Below is a quick guide to run and use a local Docker version of the tool for testing purpose

1. Get the source code

wget https://github.com/mspnp/AzureNamingTool/archive/refs/tags/v4.2.1.tar.gz
tar -xvf v4.2.1.tar.gz
cd AzureNamingTool-4.2.1/src/

2. Build the image

docker build -t azurenamingtool .

3. Start the container

docker run -d -p 8081:80 --mount source=azurenamingtoolvol,target=/app/settings azurenamingtool:latest

4. Log in to the UI and set the default admin password : http://localhost:8081/

5. Go to the Configuration page and set the Component

   

Generate a name for a resource, below an example for a PrivateDNSZone virtualNetworkLins

Build image and deploy via ArgoCD

Push your builded image to a registry

docker build -t contoso/azurenamingtool:v4.2.1 .
docker push contoso/azurenamingtool:v4.2.1

Now that your image is accessible from a registry, add the registry to your ArgoCD server.

Below is a sample of deployment code to deploy the app using your image (you can either apply the yaml via kubectl or as a project on ArgoCD).

This deployment will deploy the app with a PV, a service and an Ingress.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: azurenamingtool-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: azurenamingtool
  template:
    metadata:
      labels:
        app: azurenamingtool
    spec:
      containers:
        - name: azurenamingtool
          image: contoso/azurenamingtool:v4.2.1
          ports:
            - containerPort: 80
          volumeMounts:
            - mountPath: /app/settings
              name: settings-volume
      volumes:
        - name: settings-volume
          persistentVolumeClaim:
            claimName: azurenamingtool-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: azurenamingtool-pvc
spec:
  #Specify a storageClassName or leave blank to use default one
  #storageClassName: longhorn
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: azurenamingtool-service
spec:
  type: ClusterIP # Service type for Ingress
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: azurenamingtool
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: azurenamingtool-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    # Specify the URL you want the app to be accessible
    - host: azurenamingtool.contoso.domain.lan
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: azurenamingtool-service
                port:
                  number: 80

Don’t worry, it’s easy as using kubectl debug!

So, of course you need to have access to your AKS cluster API. In this example we will access it using kubectl.

Step 1: Get the Node Name

kubectl get nodes -o wide
NAME                                  STATUS   ROLES    AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
aks-nodepool1-12121212-vmss000000   Ready    <none>   8d    v1.29.8   10.10.10.5    <none>        Ubuntu 22.04.4 LTS   5.15.0-1071-azure   containerd://1.7.20-1
aks-nodepool1-12121212-vmss000001   Ready    <none>   8d    v1.29.8   10.10.10.6    <none>        Ubuntu 22.04.4 LTS   5.15.0-1071-azure   containerd://1.7.20-1
aks-nodepool2-13131313-vmss000000   Ready    <none>   8d    v1.29.8   10.10.10.7    <none>        Ubuntu 22.04.4 LTS   5.15.0-1071-azure   containerd://1.7.20-1
aks-nodepool2-13131313-vmss000001   Ready    <none>   8d    v1.29.8   10.10.10.12   <none>        Ubuntu 22.04.4 LTS   5.15.0-1071-azure   containerd://1.7.20-1

Identify on which node you want to connect and then run the Microsoft busybox image on it.

kubectl debug node/aks-nodepool1-12121212-vmss000000 -it --image=mcr.microsoft.com/cbl-mariner/busybox:2.0

Now you are connected to the Busybox that is running on your AKS node.

if you specifically need an image with azure cli installed on it, you need to run the following image.

kubectl debug node/aks-nodepool1-12121212-vmss000000 -it --image=mcr.microsoft.com/azure-cli:cbl-mariner2.0

Step 2: Access the Node OS

To access the node OS, you will need to chroot.

chroot /host

Now you can debug image pull issues, network access issues and others using the default binaries installed on the node ;-)

crictl pull xyz
curl -k https://my.example.com/api/vx
dig @DNS-server-name Hostname
nc -z -v 10.10.8.8 80

Step 3: Clean Up

When you have finished with the debug pod, don’t forget to delete it from the default namespace.

Enjoy!

Get your GitLab project ID : Settings → General → Project ID

Then we need to create a user access token or a project token, depending on your gitlab instance. Settings → Access Token → Add new Token

The Token needs to have at least the Maintainer role and the api scope

Finally, specify your Project ID and Token information in your provider.tf file

Example for a Project ID 6576578493

  backend "http" {
    address        = "https://gitlab.com/api/v4/projects/6576578493/terraform/state/tfstate"
    lock_address   = "https://gitlab.com/api/v4/projects/6576578493/terraform/state/tfstate/lock"
    unlock_address = "https://gitlab.com/api/v4/projects/6576578493/terraform/state/tfstate/lock"
    lock_method    = "POST"
    unlock_method  = "DELETE"
    username       = "tfstate_access"
    password       = "glpat-VMB1DHzAssssswZSTySSxe6a"
  }

Create / use a Rancher integration user

First, you need to create a user in Rancher ('user and authentication' menu > create new user) and give him the standard user role in the Rancher cluster.

Now, you need to add the 'cluster owner' role to this new user on the cluster you would like to add. Go to the cluster parameters > cluster and project members.

Then add the correct right

When it's done, you can create the bearer token for this new user . First disconnect from Rancher and connect to the web interface with the newly created user.

Then, click on the user icon on the top right of the page and go to "Account & API keys"

Now, create the api key that ArgoCD will use to deploy resources on the cluster through the user.

Get the config

Then, connect to your Rancher web interface and get the kubeconfig file of your "destination" cluster (the one you would like to add in ArgoCD).

When you have the file, get the following info from it :

• caData : the certificate authority used to connect to your cluter

• cluster: the URL to connect to the cluster

Add the cluster to argo

Now, we need to add a secret in our argoCD namespace and add the token, tls configuration and URL of the server, previously taken from the Kubeconfig file.

kind: Secret
data:
  # Within Kubernetes these fields are actually encoded in Base64; they are decoded here for convenience.
  # (They are likewise decoded when passed as parameters by the Cluster generator)
  config: "{'bearerToken':{'token-xxxxxx'},
           'tlsClientConfig':{'insecure':'false',
                               'caData':'xxxxxxx'}}"
  name: "your-newcluster"
  server: "https://yourrancherdomain.com/k8s/clusters/c-m-xxxxxxxx"
metadata:
  labels:
    argocd.argoproj.io/secret-type: cluster

Add this secret and go on your argoCD interface and you'll see that your cluster was added ! You can now start to synchronize Git / helm charts on it!

En tant qu'ingénieur, passionné par les technologies open source, j'aimerais partager avec vous comment utiliser Mistral, open-webui et Exoscale pour une utilisation plus open source des outils liés aux technologies de génération de langage naturel (GenAI).

Mistral est un projet open source qui permet de gérer les modèles de langage naturel. Il offre plusieurs avantages par rapport aux modèles et interfaces closed-source, notamment :

• Transparence dans les algorithmes utilisés, ce qui permet de comprendre les décisions prises par les modèles.

• Possibilité de personnaliser les modèles pour répondre à des besoins spécifiques.

• L'accès à une communauté active de développeurs qui contribuent régulièrement à l'amélioration du projet.

Pour accéder à Mistral, vous devez suivre les étapes suivantes :

1. Créez un compte sur le site web de Mistral https://mistral.ai/

2. Connectez-vous à votre compte et cliquez sur "API keys" dans le menu de gauche.

3. Cliquez sur "Create new API key" pour générer une nouvelle clé d'API.

Une fois que vous avez obtenu votre clé API, vous pouvez utiliser open-webui pour interagir avec les modèles de Mistral. open-webui est un outil open source qui permet de visualiser et de manipuler les modèles de langage naturel.

Pour démarrer open-webui en local, vous devez effectuer la commande suivante :

docker run -d -p 3000:8080 \
  -v open-webui:/app/backend/data \
  -e OPENAI_API_BASE_URLS="https://api.mistral.ai/v1" \
  -e OPENAI_API_KEYS="your-api-key" \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Puis, pour accéder à open-webui, aller sur l'URL http://localhost:3000/

Vous trouverez un exemple de setup SSL https en bas de cet article.

Personnelement j'utilise actuellement leur dernier modèle, le plus performant, open-mixtral-8x22b.

Si vous souhaitez héberger l'interface open-webui sur une VM s'exécutant sur un cloud public, je vous conseil d'utiliser Exoscale, un cloud souverain Européen avec des Datacenters en Suisse. Voici comment procéder :

1. Créez un compte sur Exoscale et déployez une VM avec Ubuntu 22.04 LTS.

2. Connectez-vous à votre VM via SSH et installez Docker en suivant les instructions sur leur site web.

3. Exécutez la commande suivante pour télécharger et installer open-webui :

docker run -d -p 80:8080  \
 -v open-webui:/app/backend/data  \
 -e OPENAI_API_BASE_URLS="https://api.mistral.ai/v1"  \
 -e OPENAI_API_KEYS="your-api-key"  \
 --name open-webui  \
 --restart always  \
 ghcr.io/open-webui/open-webui:main

Afin de pouvoir accéder au service open-webui il faut vous assurer d'autoriser vers le port que vous aurez mentionné dans la commande docker run.

Voic un exemple de Security Group à appliquer sur votre instance:

Pour accéder à votre instance open-webui, il suffit de trouver l'IP de votre instance et d'y accèder sur le port sur lequel écoute le service.

Fonctionnalité très utiles dans Mistral, vous pouvez suivre en temps réel l'évolution des coûts de votre compte. Soit en Euro ou en nombre de Token.

Vous pouvez ainsi faire vos tests entre les différents modèles et adapter selon vos besoins et préférences.

Total usage in TOKENS

Total usage in EURO

Et voilà!

C'est si simple de s'affranchir un peu de la Silicon Valley :-)

Exemple de setup SSL https

docker run -d -p 8080:8080   -v open-webui:/app/backend/data   -e OPENAI_API_BASE_URLS="https://api.mistral.ai/v1"   -e OPENAI_API_KEYS="your-api-key"   --name open-webui   --restart always   ghcr.io/open-webui/open-webui:main

sudo apt-get update
sudo apt-get install nginx

server {
    if ($host = ai.example.com) {
        return 301 https://$host$request_uri;
    }
    listen 80;
    server_name ai.example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name ai.example.com;

    ssl_certificate /etc/letsencrypt/live/ai.example.com-0001/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/ai.example.com-0001/privkey.pem; # managed by Certbot

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256";

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

}

Activer la configuration et obtenir le certificat SSL

sudo ln -s /etc/nginx/sites-available/open-webui.conf /etc/nginx/sites-enabled/
sudo systemctl restart nginx

Références et liens

Mistral GitHub : https://github.com/mistralai

Mistral LLM: https://mistral.ai/technology/#models

La Plateforme: https://console.mistral.ai/

Le Chat: https://chat.mistral.ai/chat

Mistral API: https://docs.mistral.ai/api/

open-webui : https://github.com/open-webui/open-webui

Exoscale: https://www.exoscale.com/

Exoscale Compute: https://www.exoscale.com/compute/

Source : Partially generated with Mistral model open-mixtral-8x22b

Indeed, Kubernetes isn't designed with an easy and "implicit" network management approach. This is where Cilium comes in and starts the presentation by introducing some questions:

• Who manages the K8s network?

• Often, network infrastructures are unaware that "pod networks" exist, what to do?

• What tools are available for troubleshooting the K8s network?

• How to check for bottlenecks and performance?

• How to manage traffic encryption?

• How to handle load balancing?

• Other requirements : we need to secure our applications, we need to know what egress IP we're using for our pods in order to manage IP filtering, etc...

Cilium is here for that

Natively in Kubernetes, all network rules are managed by a component called kube-proxy. This component controls iptables across all nodes of the cluster.

However, even thokugh iptables is widely used in the GNU/Linux environment, it's not always ideal.

Indeed, when iptables needs to update a rule, it must recreate and update all rules in a single transaction. This is a real problem when a cluster has a significant number of nodes and Pods running on it.

Moreover, this tool can only filter based on IP addresses or ports, not on potential paths or HTTP methods. This is somewhat inconvenient in a Kubernetes context where many applications are APIs.

Finally, iptables generally consumes a significant amount of CPU when running with Kubernetes.

Most of iptables' shortcomings are addressed with Cilium. It can filter at the Layer 7 (application) of the OSI model and addresses scalability issues that iptables struggled with.

How does it work ?

eBPF is the primary answer. Indeed, eBPF allows direct communication with the kernel. In fact, a slide states "what Javascript is to the browser, eBPF is to the Linux Kernel."

eBPF is used to control the traffic, load balancers, network policies, service mesh, ingress, etc.

eBPF is low level kernel coded and that's the reason it's able to communicate with the kernel and manage traffic very quickly.

Note : ePBF is massively used by Facebook / Google / Netflix to handle the traffic. Think about that the next time you're loading content from them ;)

Here's a performance comparaison between eBPF / ipvs and iptables

The end of the presentation mainly revolves around use cases, showing manifest files after installation of Cilium.

You can find the video of the presentation here .

Addition: how to install Cilium?

The installation can be done through different ways. The first one is using an Helm chart, the second one is using cilium-cli (doc available here ).

Example with helm :

helm repo add ciliumhttps://helm.cilium.io/

helm upgrade --install cilium cilium/cilium

--version 1.xx.x-rcx

--namespace kube-system

--set sctp.enabled=true

--set hubble.enabled=true

--set hubble.metrics.enabled="{dns,drop,tcp,flow,icmp,http}"

--set hubble.relay.enabled=true

--set hubble.ui.enabled=true

--set hubble.ui.service.type=NodePort

--set hubble.relay.service.type=NodePort

When it's done we can check the "cilium status" command to be sure that the cilium CNI was correctly installed.

Known limitations

• AKS supports Cilium but has some limitations (no L7 rules, no hubble) . More details here :

  https://learn.microsoft.com/en-us/azure/aks/azure-cni-powered-by-cilium#limitations

• EKS: many advanced features of Cilium are not yet enabled as part of EKS Anywhere, including Hubble observability, DNS-aware and HTTP-Aware Network Policy, Multi-cluster Routing, Transparent Encryption, and Advanced Load-balancing. More details here :

  https://isovalent.com/blog/post/cilium-eks-anywhere/#:~:text=Many%20advanced%20features%20of%20Cilium,%2C%20and%20Advanced%20Load%2Dbalancing.