Mastering the SKILL.md File in Agentic AI: A Complete Guide

April 5, 2026April 5, 2026Sandeep Mewara Leave a comment

In modern Agentic AI architectures, the primary engineering challenge is no longer generating language, but bridging the gap between conversational intent and reliable, repeatable and unambiguous execution. To achieve this, we must treat agent capabilities not as conversational shortcuts, but as well-defined engineering assets.

This requires a standardized contract for capability execution. That’s where SKILL.md comes in. A formal, machine-parsable definition file that acts as a Standard Interoperability Definition (SID) contract for systematic task execution within an agentic framework.

In this blog, I’ll dive deep into SKILL.md and share how it serves as a single source of truth for both conceptual planning (roles) and procedural execution (workflows) that power an automated, engineering-grade SDLC.

The Architectural Blueprint: The SKILL.md

A SKILL.md is structured as an engineering specification, designed for zero-ambiguity parsing by an LLM like Claude. It defines the contract for interoperability, forcing teams to move from conversational requests to precise capability definitions.

Anatomy of an Engineering Contract

The specification consists of five required metadata fields that are immutable and machine-parsable:

Name: An immutable, unique, system-wide identifier for the capability (e.g., internal-token-manager-v1, exec-raise-github-pr-v1, or sdlc-pm-v1). This is the system’s handle for the skill.
Description: Critically, this is not a summary. It is the definitive Trigger Event Definition. It must be written from the perspective of an event, user query or internal signal that activates this capability, allowing the framework to perform accurate skill matching. Example: “Triggers automatically after a successful code analysis scan…”
Commands: A list of executable operations or prompts defined by the contract. For procedural skills, these map to API endpoints or internal function calls. For conceptual skills, these map to defined prompt sequences. Example: get-linter-report(timestamp) or refresh-token(service_id).
Constraints: A critical safety and resource management section. It defines the limits, rules and error conditions of the contract. Example: “Internal authentication tokens must expire after 1 hour.”
Examples: These are not suggestions but are the gold standard of Expected Behavior. They define the intended output for specific input scenarios, providing the LLM with a definitive blueprint for successful execution and reducing non-deterministic output.

# Code Snippet 1: Sample Procedural SKILL.md (Raise GitHub PR)
---
# REQUIRED METADATA FIELDS (SID CONTRACT)

name: exec-raise-github-pr-v1
description: Triggers automatically after a successful 'exec-linter-code-analyzer-v1' scan or upon user request to systematically raise a new pull request on GitHub for reviewed code.
commands:
  - create-pr(repository_url, head_branch, base_branch, title, body)
constraints:
  - Must use a valid GitHub API token with 'repo' scope.
  - Head branch must differ from the base branch.
---

### Expected Behavior (Examples)

When this skill is matched against a standard JavaScript repository:
  - Input: create-pr("https://github.com/org/repo.git", "feat/new-api", "main", "Feat: Add API v2", "This PR introduces...")
  - Execution: Loads 'scripts/create_pr.py'.
  - Output: New PR URL.

Directory Structure & Progressive Disclosure

The SKILL.md is packaged within a defined directory structure, ensuring all supporting assets are decoupled and version-controlled alongside the specification.

.Sandeep Mewara Github

📄 SKILL.md (The only required asset, containing the definitions and contract).
📁 scripts/ (Optional: Decoupled logic – Python, Bash, Node.js, etc. The implementation details of the contract).
📁 references/ (Optional: Docs, checklists, design patterns or standards the skill must adhere to).
📁 assets/ (Optional: Templates or sample data).

This decoupled architecture enables the Progressive Disclosure Pattern, which is critical for system efficiency and managing token constraints. A high-performance agentic system should not load every asset for every skill simultaneously. Progressive disclosure ensures assets are loaded only when necessary.

Agents don’t load everything at once. They discover and expand context only when needed.

Architecting the Automated SDLC

The standardization offered by SKILL.md allows us to architect and separate the dynamic pillars of an automated SDLC, managing all capabilities via this single specification. In a professional lifecycle, conceptual setup (Defining Roles) always precedes procedural execution (Executing Workflows).

Conceptual Role-Based Skills: Defining the Contract for a Persona (Planning & Setup)

To initiate any SDLC phase (e.g., Requirements), we must first define the conceptual frameworks, knowledge bases and systematic planning workflows of specific roles that help organise content by domain (behaviour-driven). We apply the identical SKILL.md standard to define a persona’s “mindset”.

WHAT: SKILL.md definitions for Product Manager Persona or Lead Developer Persona.
APPLICATION: During the “Requirements” and “Design” phases of the SDLC.
ARCHITECTURAL FLOW: During planning, you activate the Product Manager Persona (Code Snippet 2). Claude adopts this mindset and leverages knowledge references (e.g., Agile standards) and the command contract (draft-prd(user_stories)) to provide focused, high-quality requirements.

Code Snippet 2: Sample Conceptual SKILL.md (Product Manager)
---
# REQUIRED METADATA FIELDS (SID CONTRACT)

name: sdlc-pm-v1
description: Triggers during project initiation to define the persona, responsibilities, knowledge base and systematic planning workflows of a senior Product Manager.
commands:
  - draft-prd(user_stories, acceptance_criteria)
  - run-feature-prioritization(prd_document)
constraints:
  - Must reference files in the optional 'references/' directory (e.g., 'references/agile-standards.md') for all Agile terminology.
---

### Expected Behavior (Examples)

When this skill is matched to a new project request:
  - Input: draft-prd(user_stories, acceptance_criteria)
  - Execution: Loads 'references/agile-standards.md' to define terminology.
  - Output: A structured PRD document based on the internal persona.

External Workflow Execution Skills: Defining the Contract for the Workflow to ‘Do’

Once the groundwork is established and the build begins, the agent’s focus shifts to user-triggered workflows (e.g., after a commit). These skills are guides that help perform specific, measurable steps in the automated pipeline, providing the user with domain-specific results (task-driven).

WHAT:SKILL.md definitions for exec-linter-code-analyzer, exec-raise-github-pr, or jira-ticket-update.
APPLICATION: During the “Build,” “Test” and “Deploy” phases of the SDLC, typically automated by CI/CD events.
ARCHITECTURAL FLOW: After a successful code implementation event, the framework activates the exec-linter-code-analyzer-v1 (Code Snippet 3). Claude reads the inputs and expected behavior. The framework executes the decoupled logic (scripts/) to systematically create the pull request, ensuring a reliable result (the PR URL) is provided back to the user’s workflow or CI/CD pipeline.

Code Snippet 3: Sample Procedural SKILL.md (Code Analyzer Workflow)
---
# REQUIRED METADATA FIELDS (SID CONTRACT)
name: exec-linter-code-analyzer-v1
description: Triggers automatically after a code commit event to execute a static analysis and linter scan on the modified files in a specific repository, providing a systematic JSON report.
commands:
  - run-analysis(repository_url, branch)
constraints:
  - Must use a valid GitHub API token with 'repo' scope.
---

### Expected Behavior (Examples)
When this skill is matched following a code commit:
  - Input: run-analysis("https://github.com/org/repo.git", "main")
  - Execution: Loads 'scripts/run_analysis.py'.
  - Output: Linter report JSON.

Internal Agent Operational Skills: Defining the Contract for the Software to ‘Be’

To ensure system stability, the agent software itself requires precise, standardized contracts for core operational tasks (like authentication, state, error handling, api-call, etc). These skills are operational and invisible to the SDLC workflow itself. They focus on the agent’s internal robustness and platform integrity.

WHAT: SKILL.md definitions for internal-token-manager or agent-state-historian.
APPLICATION: Triggered automatically by the agent’s orchestration layer during defined lifecycle events (e.g., establishing a session state, refreshing an expired 401 token).
ARCHITECTURAL FLOW: When any skill requires access to a restricted API, it activates the internal-token-manager (Code Snippet 4). Claude reads the command contract (refresh-token(service_id)). The framework executes the decoupled logic (scripts/) to refresh the secure token, ensuring the agent software can authenticate without creating brittle, direct credential dependencies in the domain-level skills. This internal complexity is hidden from the user but critical for security and robustness.

Code Snippet 4: Sample Procedural SKILL.md (Token Manager)
---
# REQUIRED METADATA FIELDS (SID CONTRACT)
name: internal-token-manager-v1
description: An internal operational skill that triggers throughout a workflow when the agent detects it requires a secure token to authenticate against an external service (e.g., GitHub, Slack, Splunk).
commands:
  - refresh-token(service_id)
constraints:
  - Must use a valid agent credential secret (e.g., 'agent_platform_secret').
  - Tokens must expire after 1 hour.
---

### Expected Behavior (Examples)

When this skill is matched when a GitHub operation requires auth:
  - Input: refresh-token("github_api")
  - Execution: Loads 'scripts/refresh_token.py'.
  - Output: New OAuth token JSON.

The Boundary of Autonomy and the Expertise Gap

While standardizing capabilities via SKILL.md is essential, I believe it is critical for architects to also define where SKILL.md is not the right tool. My own perspective, based on recent project implementation, is that a common architectural failure is expecting SKILL.md to easily encode true Domain Expertise and Heuristic Judgment.

Offloading Heuristics vs. Offloading Wisdom

A well-defined SKILL.md is designed to be precise, measurable and standardized. It excels at offloading common known items, standard checklists and systematic patterns into reliable workflows (as seen in our Code Snippets 3 & 4). In my recent project, this precision made the skills function as excellent fixed checklists, significantly reducing operational ambiguity.

This same precision, however, means it can appear only as a checklist. A procedural skill like exec-linter-code-analyzer can identify a syntax error based on a rule, but I found it often lacked the domain wisdom to understand the conceptual design decision that led to that error.

Assisting Expertise, Not Replacing It

Based on the experience so far, I believe that you cannot easily encode a senior engineer’s years of nuanced design thinking into a SKILL.md description. The true architectural value of a standardized specification is that it offloads the reliable execution complexity, allowing the Human Expert (or a high-level Agentic Persona) to focus entirely on core domain and design reasoning.

For now, I believe following a model where three distinct pillars of knowledge are defined will work out:

Systematic Workflows (Procedural Skills): Handled perfectly by SKILL.md. (The “What to Do”)
Conceptual Frameworks (Persona Mindsets): Setup by SKILL.md. (How Claude “Thinks”)
Domain Wisdom & Design Reasoning: Passed as the problem context in the main prompt. (Why Claude “Decides”)

Engineering Best Practices for SKILL.md Mastery

Achieving systematic capability definition requires adhering to these foundational best practices:

Strict Decoupling: Never place the execution logic (e.g., Python code) directly within the SKILL.md file. The SKILL.md is the specification & the scripts/ directory is the implementation.
Immutability: Once a skill is deployed, treat its metadata (Name, Description, Commands) as immutable. Any significant change requires a new version (e.g., exec-raise-github-pr-v2). Brittleness often stems from changing definitions in place.
Description as a Trigger: Never write a summary description (e.g., “This skill runs a linter”). It must be written as a trigger definition (e.g., “Triggers automatically after a context save event…”). Skill matching depends entirely on this accuracy.
Token Economy: Adhere to strict size constraints: < 500 lines and < 5k tokens for the SKILL.md. The Progressive Disclosure pattern will handle heavier assets, keeping the SID itself focused and parseable.
Git-Managed Context: Treat SKILL.md files as code. They must be version-controlled in Git, promoting discoverability, reuse and providing a traceable history of how capabilities have evolved throughout the lifecycle.

Final Thought: A Standard for Scaling Autonomy

By adopting the SKILL.md specification, we move from fuzzy conversational AI to a structured engineering discipline, where all agent capabilities, whether they are internal operational requirements, external user workflows or conceptual roles framework – all are defined by precise, version-controlled contracts.

This foundation standardizes reliable execution complexity, not only making your automated SDLC predictable and robust but also ensuring that precious domain expertise remains focused on main design decisions, not common patterns. Mastering the SKILL.md standard is the definitive, interoperable foundation for building scalable, maintainable and engineering-grade AgenticAI architectures.

. Sandeep Mewara Github
News Update
Tech Explore
Trend
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow

[DOWNLOAD: skill.md Quick Reference Guide]

Agentic AI for Beginners: My Journey into Building with Claude

March 30, 2026March 30, 2026Sandeep Mewara Leave a comment

There’s been a lot of buzz around Agentic AI lately, especially around how powerful Claude can be when used beyond simple prompting. Naturally, I got curious.

As an architect, I wanted to understand what “agentic” really means in practice. What changes when we move from prompts to agents? And what does that mean for how we design systems? As I started exploring, it became clear this isn’t just about smarter chatbots, it’s something more.

From Prompts to Agents: What’s the Difference?

Before diving in, let’s distinguish between Generative AI and Agentic AI.

Generative AI (Reactive) – Deals with Prompts, where we provide an input and the model provides a one-time response. We are the orchestrator.
Agentic AI (Proactive) – Deals with Agents, where we provide a goal and the model determines the steps, uses tools and iterates until the goal is met. The model is the orchestrator.

“Agentic” means moving from chatting to delegating. It’s the difference between asking for instructions and having the task completed for you, like getting a recipe vs hiring a chef or asking for directions vs being driven there.

The “Agentic” Starter Pack

I started with the basics to see how Claude handles the “plumbing” of a real-world project. My exploration focused on three core hyped items:

Agentic Implementation: I moved away from “one-off” prompts and built loops where Claude runs a Plan -> Execute -> Test -> Fix cycle autonomously.
Model Context Protocol (MCP): I hooked Claude up to my local filesystem, Slack & GitHub. This was to see how the agent “reaches out” and queries the data it needs directly.
Role-Based Division: I experimented with “Agent Teams” by giving different Claude instances specific roles: one as the Architect to handle planning and another as the Developer to handle implementation. Further, tried to put multiple hats for the clarity of work distribution and decision making for the agent.

My Learning Project: Endpoint Watch Agent (EWA)

The goal of this project was to build a hands-on learning kit for agentic systems. Endpoint Watch Agent (EWA) is a Python-based agent that continuously monitors configurable endpoints (websites or APIs). When an endpoint is down or unhealthy, the agent autonomously evaluates the incident, avoids duplicate alerts, creates a ticket and sends a contextual Slack notification.

Flow Diagram Plan

Structuring the Workflow

Starting from nothing, I worked with Claude itself to set up the structure and segregation of components, defining single responsibility. To keep things simple, created a single agent (Orchestrator) that runs one sequential loop: Check Endpoint 1 → Decide → Act → Check Endpoint 2 → Decide → Act...

The PolicyEngine is not an agent but a pure function called by the agent. The tools are interfaces that the agent dispatches, while the MCP servers are external services.

Explore or build on the Project available here: [Github Link]

What I Learnt: The “Pro” Framework

The real breakthrough wasn’t the model itself, but how I structured the project to guide it. Below project structure can be considered a good architectural template as a baseline start for any agentic development. The architectural pattern supports a clean separation of concerns, where we can add new tools, policy rules or tests without needing to restructure the entire system.

As a production-ready baseline though, it has gaps: no tests, single-threaded endpoint checking, no metrics, no graceful shutdown. These are solvable without rethinking the architecture, but they’d need to be added before shipping anything real.

I found that following four pillars are essential for any agentic workflow:

CLAUDE.md (The Project Brain)

This file lives in the root of your repo as the AI’s operating manual. It tells Claude agent who it is and how it should behave in this specific codebase. Thus, it helps to start with shared context instead of inferring everything from scratch each session.

# Project Context: Endpoint Watch Agent (EWA)

## Role & Mission
You are the **EWA Specialist**. Your goal is to maintain a high-availability monitoring system. You prioritize accuracy in incident detection and clarity in Slack notifications.

## Tech Stack
- **Runtime:** Python 3.12
- **Logic:** Policy-based reasoning (PolicyEngine)
- **Integrations:** Slack (Alerts), Jira (Tickets), GitHub (MCP)

## Architecture Rules
- **Separation of Concerns:** Keep tools in '/tools', logic in '/engine'.
- **Async First:** Use 'asyncio' for all network-bound endpoint checks.
- **No Deletions:** Never delete incident logs, only archive or update status.

## Dev Commands
- **Run:** 'python main.py'

CLAUDE.md is the interface between the human who designed the system and the AI that extends it. It’s not a documentation for users of the tool instead is a documentation for the next builder, human or AI.

SKILLS.md (The Capability Manual)

While CLAUDE.md is about the project, SKILLS.md is about what the agent is capable of doing. It provides pre-verified “recipes” for complex tasks, stopping the agent from hallucinating its own (often broken) logic.

# Agent Skills

## Skill: Incident Evaluation
- **When:** An endpoint returns a non-200 status.
- **Action:** 
  1. Check 'storage/incidents.json' for active tickets.
  2. If new, invoke the 'JiraTool' to create a "Critical" task.

## Skill: Slack Formatting
- **Constraint:** Always include the Status Code, Response Time, and the "Runbook Link" from the configuration file.
- **Tone:** Professional and urgent.

These are the procedural instructions or documentation that teach the agent how to use a tool effectively in a specific context.

“Plan, then Execute” Workflow

I stopped asking Claude to “just do it”. Instead, I enforced a mandatory two-step gate:

The Plan: Claude must output a step-by-step technical plan first.
The Approval: I review the plan for architectural alignment.
The Execution: Only after approval does the agent start writing code. This eliminates 90% of the “rabbit holes” agents often fall into.

Verification Criteria

Never ask an agent to “fix a bug”. Instead, ask it to “Fix the bug and provide the specific CLI command or test case to verify the fix”. It seems an agent that knows it has to prove its work is significantly more accurate and less likely to hallucinate a “done” state!

What I Learnt: Behavioral System Design

EWA is built like a Claude agent where it has a brain (orchestrator), reasoning (policy engine), senses (endpoint checker), hands (Jira + Slack tools) and memory (incident store).

Thus, moving beyond simple monitoring, this system creates a truly agentic closed loop: it observes, reasons, decides, acts and remembers, closing the gap between detection and autonomous resolution. This is what differentiates a single prompt from a system that operates.

If designed properly, the orchestrator never does anything directly. It asks tools to observe, asks the policy engine to reason, then dispatches to tools based on the decision. Every component has one job and knows nothing about the others.

Thus, with agentic systems, we start to define goals, shape decision boundaries, orchestrate tools and design workflows. The unit of design has moved from “What does this function do?” to “How does this system behave over time?”. This is very different and is a significant mindset shift.

What I Learnt: The Operational Reality

This is where Agentic AI gets interesting and at the same time risky. They are not just capable but are also more complex to reason about.

What’s Exciting (The Wins)

Self-Healing Workflows: Automation of operational tasks where systems can adapt to minor changes instead of simply breaking
Engineering Velocity: Drastic reduction in manual intervention for complex, multi-file refactors

What’s Hard (The Risks)

Observability & Non-Linear Debugging: Traditional logs don’t help much when an agent enters a logic loop. It becomes difficult to answer: “Why did the agent choose this specific tool at this specific time?” Tracking these non-linear flows requires a completely different observability stack.
Guardrails & Cost: Without structural “circuit breakers”, agents can enter recursive loops that transform a technical logic error into a financial one. In an agentic world, unguided autonomy doesn’t just crash a service, it can drain token budgets in minutes.

What I Learnt: The Shift to “Specification of Judgment”

The biggest realization was the shift in our roles: The engineer’s job is becoming the specification of judgment.

We are moving away from writing line-by-line code and towards translating domain knowledge (e.g., Don’t auto-close the Jira ticket on recovery instead leave that to humans), operational experience (e.g., What if the MCP server subprocess hangs instead of failing?) and trust calibrations (e.g., Trust the agent to send Slack alerts without human review: yes) into rules the agent can follow.

Claude handles the execution, but its success depends on our ability to articulate why a system should behave a certain way, not just what it should do. This requires architectural experience to anticipate what could go wrong and the clarity to express those constraints precisely.

Final Thoughts: The Evolution of How We Build

It’s only a matter of time. While the technical risks are real today, the pace of advancement is blistering. We are witnessing a total paradigm shift: we aren’t just writing code anymore, instead we are managing a digital workforce.

For architects, this means rethinking system boundaries. For developers, it means thinking in workflows. I am excited to adapt! This isn’t the evolution of standard coding but the evolution of how we build.

.Sandeep Mewara Github

News Update
Tech Explore
Trend
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Dynamic Programming

Explore or build on the Project available here: [Github Link]

Why ‘Service as Software’ is the Industry’s Next Big Bet

March 22, 2026March 22, 2026Sandeep Mewara Leave a comment

I recently caught a presentation by Intuit CTO Alex Balazs, where he described their evolution from a “Do-It-Yourself” software company to an AI-driven expert platform. During the talk, he used a phrase that immediately clicked for me: “Service as Software”.

It was one of those “aha” moments that forced me to pause and re-evaluate the trajectory of the entire SaaS industry. We’ve spent the last twenty years perfecting Software as a Service, but flipping that phrase to Service as Software implies a much deeper shift in how we deliver value. It provoked me to dig into why this isn’t just a trend, but a directional necessity for the next generation of tech.

The Shift: From Passive Tools to Active Experts

For years, the gold standard has been the “System of Record“. We built beautiful digital filing cabinets and powerful calculators, but they were ultimately passive tools. Whether it was an accounting suite or a CRM, the software only provided value if a human expert sat behind the keyboard to drive it. In that model, the value only scales as fast as the person at the controls.

Now, “Service as Software” represents a move toward a “System of Action“. With the rise of agentic AI, software is moving from the “medium” to the “expert.” Recent 2025 research from Capgemini highlights that we are moving beyond “Copilots” to “Agents” where AI that doesn’t just suggest actions but possesses the autonomy to execute end-to-end business processes.

SaaS (The Tool): The software provides the interface where the user performs the labor.
Service as Software (The Outcome): The software acts as an autonomous agent navigating complexity, identifying optimizations and executing tasks on the user’s behalf.

Why this is the Industry’s Directional Need

As I look at the landscape from a leadership perspective, this shift feels inevitable. We are hitting a ceiling with traditional models for a few key reasons:

Solving for “SaaS Fatigue”: The “per-seat” model is under pressure. According to 2026 SaaS pricing forecasts, nearly 60% of enterprise SaaS solutions are shifting toward hybrid or outcome-based pricing. Customers are tired of managing dozens of tools that require constant human attention. They want problems solved, not more licenses to manage.
Bridging the Expertise Gap: We are facing a documented global shortage of human experts in complex fields like finance, specialized engineering and data science. By “coding” that expertise directly into the software, we make high-level results accessible at a scale that human labor simply cannot match.
Accelerating Time-to-Value: Traditional software often has a long “time-to-value” during onboarding, a period where 63% of customers are already deciding whether to churn. A service-oriented model flips this. By having the software perform the initial heavy lifting for the user, you deliver the “aha moment” almost instantly.

Navigating the Transition: A Technical Leader’s View

Transitioning to this model is an architectural marathon. You don’t just “add AI” and call it a service. It requires a fundamental rethink of the stack.

The “Human-in-the-Loop” Bridge: Trust is the primary hurdle. Successful transitions will likely use a hybrid model where AI performs 80% of the work, but human experts remain available for the “gray areas”. This builds the user’s confidence in the system’s autonomy while maintaining a safety net.
Codifying Logic, Not Just Features: We have to shift from building “buttons” to building “agents”. This requires robust reasoning engines that can handle exceptions and ambiguity without breaking.
The Observability Mandate: If the software is performing the service, it cannot be a black box. As architects, we must build in deep transparency providing “reasoning logs” so users can always audit why a specific decision was made.

Closing Thoughts

We are moving away from providing digital tools and toward providing digital results. The most successful companies of the next decade won’t just be selling software but they’ll be selling outcomes and confidence.

The transition from being a vendor of tools to being a partner in results is a massive challenge, but for those of us in technical leadership, it’s easily the most exciting problem to solve in a long time. It’s no longer about what our users can do with our software but it’s about what our software can do for our users.

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Dynamic Programming

Disclaimer: This post represents my personal perspectives and reflections on industry trends. These thoughts are my own and do not necessarily reflect the official positions or strategies of any specific company or employer.

The Great Inversion: Why AI is Moving from Cloud to Desktop

March 15, 2026March 23, 2026Sandeep Mewara Leave a comment

For the better part of a decade, the desktop was largely relegated to a passive terminal, a mere high-resolution viewport for remote cloud services. As the industry mantra shifted to “Cloud-First”, local hardware was often treated as an underutilized abstraction.

However, we are now witnessing The Great Inversion. As AI workloads navigate the practical limits of cloud latency, data privacy and operational costs, the center of gravity is visibly shifting back to the local system. We are moving towards the era of the AI-Native Desktop, where the local machine is no longer just a window to the cloud, but is increasingly becoming the primary engine of intelligence.

The Evolution of the “SaaS Margin”

A primary driver of this shift appears to be fundamental economics. Throughout 2024 and 2025, as software providers integrated Large Language Models (LLMs) into their web platforms, it became clear that inference costs could significantly erode margins. This “Token Tax” has encouraged a strategic reckoning across the industry.

The Data: According to early 2025 fiscal reports from major SaaS players, AI-related compute costs increased OpEx by an average of 25-30% year-over-year.
The Cost Shift: Industry analysis from Deloitte and various independent reports suggests that local NPU inference can reduce AI operational costs by up to 90% ( Medium/Vygha, 2025). By migrating specific compute tasks to the desktop, we can transition from a variable OpEx model towards a more sustainable fixed hardware model.

The Proliferation of the AI PC

The “Inversion” is physically supported by a massive hardware refresh. We are no longer designing for underpowered machines. As of Q1 2026, the “AI PC” has moved from a premium category to the industry baseline.

The Benchmark: The AI PC has evolved from a niche offering into an enterprise standard. Gartner reports that AI PCs now account for over 55% of all shipments, with nearly 100% of new enterprise purchases featuring dedicated NPUs (Gartner, 2025).

Microsoft introduced “Copilot+ PCs” as a new Windows category built around local AI acceleration (NPUs) and has continued to expand GA AI features (some in preview) across this category, emphasizing on-device experiences.
Silicon Supremacy: Standard workstations now ship with 40+ TOPS (Trillion Operations Per Second) capability. This allows for real-time local inference that was previously technically out of reach (Microsoft Learn, 2025).

Chip vendors are also directly pushing the “on-device inference” narrative as a foundational shift (cost, latency, privacy, reliability).

Compliance and the “Privacy Moat”

Regulatory considerations are making the cloud a complex environment for sensitive data. With the EU AI Act entering its critical enforcement phase in August 2026, there is a clear directional pull toward “Zero-Export” AI solutions (EU AI Act Guide, 2026 ).

Apple’s Blueprint: Apple has helped standardize this approach with Apple Intelligence and Private Cloud Compute. Their architecture ensures that if a task can be processed on-device (via the M4’s 38-TOPS Neural Engine), it remains local. Only when necessary does it move to “stateless” servers designed to process data without storing it (Apple Privacy, 2025 ).
Data Sovereignty: Modern desktop apps can index a user’s local files to provide personalized AI insights (Local RAG, i.e. Retrieval-Augmented Generation) without ever exposing that intellectual property to a third-party cloud provider. Local-first patterns are re-emerging because they improve resilience and user trust (data control, offline capability, graceful sync).

Performance: Breaking the Latency Wall

The browser is naturally limited by the “spinning wheel” of network latency. For the next generation of Agentic AI, tools that actively assist by observing screen context and reacting in real-time, the network round-trip is often a bottleneck.

Feature	Web App (Cloud AI)	AI-Native Desktop App (NPU)
Response Latency	200ms – 500ms lag	<20ms (Instant)
Data Privacy	Encrypted in Transit	Zero-Export (Stays on Disk)
Offline Capability	Non-existent	Full Functionality
Operational Cost	Per-token / Monthly	One-time Development
System Access	Sandboxed/Limited	Deep File & OS Integration

Moving Forward: The Architect’s Blueprint

To remain competitive in 2026 and beyond, a forward-thinking desktop strategy should aim to capitalize on this hardware-rich environment. While the web remains vital, relying solely on the browser may now carry missed opportunities. A prepared strategy should consider:

Framework Modernization: Exploring lightweight native cores. This involves moving toward Rust-based frameworks like Tauri that interface directly with the local NPU via DirectML or CoreML, rather than relying on memory-heavy wrappers.
Hybrid Model Deployment: Integrating Small Language Models (SLMs) like Phi-4 or Llama 3-8B inside the desktop installer. These can handle the majority of daily tasks, reserving the cloud for “Heavy Reasoning” only.
Local Vector Databases: Utilizing local databases (e.g., LanceDB) for hyper-personalized, privacy-first “Long-Term Memory” of the user’s local files, all without requiring a cloud sync.

Conclusion: Towards a Structural Evolution

The current landscape suggests we are moving towards more than just a passing trend. We appear to be entering a structural shift in how software is delivered. There seems to be a renewed potential for the desktop to reclaim its significance, as it offers a compelling intersection where Performance, Privacy and Profit can uniquely align.

However, the most promising products in this new era likely won’t be “desktop-only” in the traditional sense. Instead, there is a clear path for the emergence of desktop-first AI workspaces which will act as platforms that leverage cloud augmentation, sophisticated model-routing and seamless OS integration to redefine the modern workflow.

Final Thought: In 2016, we asked, “Why build a desktop app when you can build a website?” In 2026, the question is increasingly, “Why would a user trust a website with their data when their desktop can do it better, faster and more securely?”

AI seems to be shifting software architecture toward hybrid local-cloud models, which is beginning to elevate the strategic importance of desktop environment once again.

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
Word Ladder solution
What is Dynamic Programming

Disclaimer: The views and opinions expressed in this article are strictly my own and reflect my personal belief in current market directions. They do not constitute professional or investment advice. Technology landscapes change rapidly, therefore, readers should perform their own due diligence and assess their specific needs before making any architectural or business decisions. I shall not be held responsible for any actions taken based on the contents of this post.

Decode with Google 2021

August 3, 2021August 3, 2021Sandeep Mewara Leave a comment

Yesterday, I received an email from Google about an upcoming virtual event Decode with Google India 2021. It is a day event, planned to be held on August 26, 2021 compared to a two day event last year.

Details of the event can be found here: Decode with Google

It seems, this time the focus of event is: Contributing from India, Impacting the World

Join us for the virtual edition of Decode with Google India 2021 and learn about what we’re working on, the state of AI, the opportunities it presents—with a focus on the next billion internet users. You will get to hear live from the Google Tech leaders and you will also have the opportunity to ask leaders some questions and potentially have them answered live on air!
Event highlight shared by Google

If it resonates, Register to join in to learn more and ask queries about how Google engineers, researchers, scientists and product managers based in India are addressing unique Indian challenges, and finding solutions that will benefit people across the world.

Event date: August 26, 2021 12:00 AM – 1:10 PM GMT+5:30

Happy connecting!

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming

Microsoft Build 2021 Highlights

June 6, 2021June 13, 2021Sandeep Mewara Leave a comment

As shared in my earlier post, recently Microsoft held its Build Conference 2021 as a virtual event. It spanned across for more than two days that touched upon various topics. It was not as big as it used to be in person for last several years. There were numerous updates that they shared which I have tried to cover here.

To empower developers, to empower you, to empower the world.
Microsoft vision

Core Themes

Microsoft unveiled a handful of news and tools to empower developers. There were multiple Innovative AI, mixed reality solutions, intelligent cloud tools solutions shared. Core themes for the conference were as depicted below:

With so much changing in working mode, Microsoft shared how they are looking to support for Operational excellence with their developer platform. A deep dive session on how to increase developer velocity with Microsoft’s end to end platform was shared during the conference.

Further, data being the key for future transformations, they showcased how Azure Data was being built as a platform that could support AI & data governance.

They now support building deeper Artificial Intelligence infrastructure for everyone. This would help in easy onboard and usage of Azure services to have more proactive approach to solutions. Details about how to harness the power of data in our applications with Azure can be viewed here.

With Microsoft vision to empower developers, there was a session on how to build cloud native solutions that can run on premise, on edge and on multiple cloud. Recording of the session can be seen here.

Covid has probably made the biggest shift in how we work today. A hybrid future work style is being imagined with new operating models to work, learn or collaborate.

Microsoft Teams is one of the Microsofts fastest growing product with 145 million daily active users to date.

With new working style, transformation seems to be an opportunity where cross device collaboration would be the key. Microsoft surely knows the same and are investing in it. Following session shares on how they are progressing on the collaborative applications journey: Build the next generation of collaborative apps for hybrid work

Based on the different kind of services, need of support, scalability options and various other factors puts software as a service, a much needed way. Microsoft looked focused on the same and shared on how they are working towards a cloud native SaaS apps composed on top of other clouds and components.

Microsoft also held a session showcasing how they are working towards helping build Metaverse Apps (digital and physical mixing up) incorporating digital twins, IOT, Autonomous systems, Power platforms and Mesh.

Announcements

Like always, there were few key announcements about new initiatives and solutions as mentioned below:

Real World

Through sessions, Microsoft covered few examples on how various other organizations are leveraging Microsoft solutions to provide awesome customer experience with speed and accuracy.

ms-toyota — Fusion Teams for mission critical delivery app

ms-twitter — Generate captions for live audio conversations using MS Speech service

ms-walmart — Power eCommerce transactions using MS CosmosDB

ms-servicenow — Incident response using MS Teams, Graph & Bot

ms-finastra — Application development using MS Financial Services cloud, Teams & Azure

ms-abinbev — Track bottles till distribution processes using Metaverse stack

Nadella in his keynote also shared that because of Covid, digital transformations have accelerated and has advanced by 10 years.

Post pandemic, virtual world will still have a significant role in the new normal. Solutions across industries would want to be connected and continue use the benefits of digital first responders. This would further fuel the innovations and development across technology.

Windows Update

Microsoft’s Build 2021 event didn’t had much of Windows-specific news. Few that came out were:

21H1 windows update was rolled out with multiple security fixes
Support for Linux GUI apps on Windows 10 will come later this year
Continues promoting ARM – Qualcomm ARM/Snapdragon Developer Kit was announced
Fall update (aka Sun-Valley Update) might have renewed UX

Windows10 used by 1.3 billion – work, learn, connect and play
Microsoft

.NET Update

.NET 6 is the next version of .NET, a modern, open-source development platform for building apps for any OS with the best performance and productivity.

.NET 6 completes the unification of the platform and adds new capabilities for building web, native and hybrid apps for Linux, Windows, Mac, iOS and Android with a single codebase. Details can be viewed here: .NET 6 deep dive – what’s new and what’s coming

.NET 6 preview 4 available, .NET 6 planned to be released during .NET Conf 2021
Visual Studio 2022 under works, preview to be available soon

Imagine Cup 2021

Make an impact through coding, collaboration, and competition. Innovate with passion to tackle global issues and bring your idea to life in the Imagine Cup.
Imagine Cup

One Winner out of four student teams from across the world was announced. Teams brought their innovations focused on four social good categories – Earth, Education, Healthcare, and Lifestyle. With intention to solve issues in their local and global communities, multiple teams participated and final four were selected after evaluation on various parameters.

Team REWEBA from Kenya were announced as the Imagine Cup 2021 World Champions.

What’s next

Learn, connect, and explore all of the sessions and on-demand content from Microsoft Build anytime, anywhere.
Microsoft

With virtual as normal, Microsoft hinted at having multiple What’s next virtual events to share updates through out the year. For now, they have asked to save the date June 24 as event to share all about the operating system’s updates.

All the MS Build sessions are recorded and can be viewed from here.

Reference: https://mybuild.microsoft.com/home

Microsoft Build 2021

May 21, 2021May 21, 2021Sandeep Mewara Leave a comment

Microsoft Build is an online free event planned on May 25-27, 2021. It’s suggestible to register in order to access the full learning experience, including all sessions and the Connection Zone. All the details of the event are here.

At Microsoft Build, you’ll leave a better developer than when you arrived. It’s where you can solve challenges, meet the engineers behind the Microsoft platforms you use every day, and connect with a diverse group of coders who want to hone their skills.

Learn the tools and technologies that help you create solutions you can build today. Explore new developer technologies and techniques that will drive your vision and expand your toolkit. Bring your technical questions and imagination and leave with new knowledge and skills.
Event highlight shared by Microsoft

Whether you are a student or an experienced app developer, it promises technical sessions, various experts, product round-tables, One-on-One consultations and other engaging activities to learn.

Happy connecting!

Microsoft WebView2 – A new friend for native apps!

March 7, 2021May 9, 2021Sandeep Mewara Leave a comment

Now, we can embed web content (HTML, CSS, and JavaScript) in our native applications with Microsoft Edge WebView2. Earlier, it was announced only for Win32 C/C++ apps but later, it was announced available for use in .NET 5, .NET Core, and .NET Framework Windows Forms and WPF applications.

It uses the modern Microsoft Edge (Chromium) platform to host web content within native Windows applications.

In the future, the Evergreen WebView2 Runtime plans to ship with future releases of Windows. Deploy the Runtime with your production app until the Runtime becomes more universally available.
Microsoft recommendation

Power of Native

With growing presence of online world, desktop applications are pushed to use more of online like capabilities. Further, web solution also helps reuse most of the code across different platforms and easy change delivery. This is leading desktop applications more and more towards hybrid approach where best of both (native and web) can be leveraged.

Microsoft WebView2 comes to rescue. It helps build powerful applications with controlled access to native capabilities.

WebView2 uses the same process model as the Microsoft Edge browser. A browser process is associated with only one user data folder. A request process that specifies more than one user data folder is associated with the same number of browser processes.

More details about browser process model can be read here.

WebView2 apps create a user data folder to store data such as cookies, credentials, permissions, and so on. After creating the folder, your app is responsible for managing the lifetime of the user data folder, including clean up when the app is uninstalled
Microsoft – Managing user data folder

Microsoft has laid here some best practices for developing secure WebView2 application.

Distribution

When distributing your WebView2 app, ensure the backing web platform, the WebView2 Runtime, is present before the app starts.

By default, WebView2 is evergreen and receives automatic updates to stay on the latest and most secure platform.

Evergreen Bootstrapper – a tiny installer that downloads the Evergreen Runtime matching device architecture and installs it locally.
Evergreen Standalone Installer – a full-blown installer that can install the Evergreen Runtime in offline environment.
Fixed Version – to select and package a specific version of the WebView2 Runtime with your application.

The WebView2 Runtime is a redistributable runtime and serves as the backing web platform for WebView2 apps.

Download the runtime from here. Supported platforms are mentioned here.

Sample Application

I built a sample WPF application (runs on .NET Framework and not Core) to try WebView2. This was to evaluate how comparatively older .NET applications would work out.

I tried to display my blog in the WPF application using a WebView2 control. Sample application had capabilities to post message to host application and back as well as hook events as per need.

public MainWindow()
{
    InitializeComponent();

    // NavigationEvents
    webView.NavigationStarting += WebView_NavigationStarting; ;
    webView.SourceChanged += WebView_SourceChanged;
    webView.ContentLoading += WebView_ContentLoading;
    webView.NavigationCompleted += WebView_NavigationCompleted;

    // Embedded at CoreWebView2 level
    InitializeOnceCoreWebView2Intialized();
}

/// <summary>
/// initialization of CoreWebView2 is asynchronous.
/// </summary>
async private void InitializeOnceCoreWebView2Intialized()
{
    await webView.EnsureCoreWebView2Async(null);

    // Hook other events
    webView.CoreWebView2.FrameNavigationStarting += CoreWebView2_FrameNavigationStarting;
    webView.CoreWebView2.HistoryChanged += CoreWebView2_HistoryChanged;

    // For communication host to webview & vice versa
    webView.CoreWebView2.WebMessageReceived += CoreWebView2_WebMessageReceived;
    await webView.CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync("window.chrome.webview.postMessage(window.document.URL);");
    await webView.CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync("window.chrome.webview.addEventListener(\'message\', event => alert(\'Message from App to WebView2 on navigation!\'));");
}

/// <summary>
/// Web content in a WebView2 control may post a message to the host 
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void CoreWebView2_WebMessageReceived(object sender, CoreWebView2WebMessageReceivedEventArgs e)
{
    // Retrieve message from Webview2
    String uri = e.TryGetWebMessageAsString();
    addressBar.Text = uri;

    // Send message to Webview2
    webView.CoreWebView2.PostWebMessageAsString(uri);
    log.Content = $"Address bar updated ({uri}) based on WebView2 message!";
}

/// <summary>
/// Execute URL
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void ButtonGo_Click(object sender, RoutedEventArgs e)
{
    try
    {
        Uri uri = new Uri(addressBar.Text);

        if (webView != null && webView.CoreWebView2 != null)
        {
            webView.CoreWebView2.Navigate(uri.OriginalString);
        }
    }
    catch (UriFormatException)
    {
        MessageBox.Show("Please enter correct format of url!");
    }
}

/// <summary>
/// Allow only HTTPS calls
/// WebView2 starts to navigate and the navigation results in a network request. 
/// The host may disallow the request during the event.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void WebView_NavigationStarting(object sender, CoreWebView2NavigationStartingEventArgs e)
{
    String uri = e.Uri;
    if (!uri.StartsWith("https://"))
    {
        e.Cancel = true;
        //MessageBox.Show("Only HTTPS allowed!");

        // Inject JavaScript code into WebView2 controls at runtime
        webView.CoreWebView2.ExecuteScriptAsync($"alert('{uri} is not safe, try an https link please.')");
    }
}

Various events run when specific asynchronous actions occur to the content displayed in a WebView2 instance.

`NavigationStarting`	WebView2 starts to navigate and the navigation results in a network request. The host may disallow the request during the event.
`SourceChanged`	The source of WebView2 changes to a new URL. The event may result from a navigation action that does not cause a network request such as a fragment navigation.
`ContentLoading`	WebView starts loading content for the new page.
`HistoryChanged`	The navigation causes the history of WebView2 to update.
`NavigationCompleted`	WebView2 completes loading content on the new page.
`ProcessFailed`	To react to crashes and hangs in the browser and renderer processes
`Close`	To safely shut down associated browser and renderer processes

Key events

Working with the sample application, I was able to display a webpage, intercept calls both ways and embed message/code to my need. It provides all the capabilities that seems to be needed for a stable web app display control.

Complete sample application can be downloaded from here: https://github.com/sandeep-mewara/WebView2WpfBrowserApp

Reference

https://developer.microsoft.com/en-us/microsoft-edge/webview2/
https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/wpf
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/distribution

Quick look into Machine Learning workflow

December 6, 2020December 13, 2020Sandeep Mewara Leave a comment

Before we jump into various ML concepts and algorithms, let’s have a quick look into basic workflow when we apply Machine Learning to a problem.

A short brief about Machine Learning, it’s association with AI or Data Science world is here.

How does Machine Learning help?

Machine Learning is about having a training algorithm that helps predict an output based on the past data. This input data can keep on changing and accordingly the algorithm can fine tune to provide better output.

It has vast applications across. For example, Google is using it to predict natural disasters like floods. A very common use we hear these days in news are usage in Politics and how to attack the demography of voters.

How does Machine Learning work?

Data is the key here. More the data is, better the algorithm can learn and fine tune. For any problem output, there would be multiple factors at play. Some of them would have more affect then others. Analyzing and applying all such findings are part of a machine learning problem. Mathematically, ML converts a problem output as a function of multiple input factors.

Y = f(x)
Y = predicted output
x = multiple factors as an input

How does a typical ML workflow look?

There is a structured way to apply ML on a problem. I tried to put the workflow in a pictorial view to easily visualize and understand it:

It’s goes in a cycle and once we have some insights from the published model, it goes back into the funnel as learning to make output better.

Roughly, Data scientists spend 60% of their time on cleaning and organizing data

Walk through an example?

Let’s use dataset of Titanic survivors found here and run through basic workflow to see how certain features like traveling class, sex, age and fare are helping us assess survival probability.

Load data from file

Data could be in various format. Easiest is to have it in a csv and then load it using pandas. More details around how to play with data is discussed here.
.

# lets load and see the info of the dataset

titanicdf = pd.read_csv("data-files/titanic.csv")
print(titanicdf.info())

&lt;class 'pandas.core.frame.DataFrame'>
RangeIndex: 1309 entries, 0 to 1308
Data columns (total 14 columns):
 #   Column     Non-Null Count  Dtype  
---  ------     --------------  -----  
 0   pclass     1309 non-null   int64  
 1   survived   1309 non-null   int64  
 2   name       1309 non-null   object 
 3   sex        1309 non-null   object 
 4   age        1046 non-null   float64
 5   sibsp      1309 non-null   int64  
 6   parch      1309 non-null   int64  
 7   ticket     1309 non-null   object 
 8   fare       1308 non-null   float64
 9   cabin      295 non-null    object 
 10  embarked   1307 non-null   object 
 11  boat       486 non-null    object 
 12  body       121 non-null    float64
 13  home.dest  745 non-null    object 
dtypes: float64(3), int64(4), object(7)
memory usage: 143.3+ KB
None

# A quick sample view
titanicdf.head(3)

	pclass	survived	name	sex	age	sibsp	parch	ticket	fare	cabin	embarked	boat	body	home.dest
0	1	1	Allen, Miss. Elisabeth Walton	female	29	0	0	24160	211.3375	B5	S	2	NaN	St Louis, MO
1	1	1	Allison, Master. Hudson Trevor	male	0.9167	1	2	113781	151.55	C22 C26	S	11	NaN	Montreal, PQ / Chesterville, ON
2	1	0	Allison, Miss. Helen Loraine	female	2	1	2	113781	151.55	C22 C26	S	NaN	NaN	Montreal, PQ / Chesterville, ON

Data cleanup – Drop the irrelevant columns

It’s not always that the data captured is the only data you need. You will have a superset of data that has additional information which are not relevant for your problem statement. This data will work as a noise and thus it’s better to clean the dataset before starting to work on them for any ML algorithm.
.

# there seems to be handful of columns which 
# we might not need for our analysis 
# lets drop all those column that we know 
# are irrelevant
titanicdf.drop(['embarked','body','boat','name',
'cabin','home.dest','ticket', 'sibsp', 'parch'],
axis='columns', inplace=True)

titanicdf.head(2)

	pclass	survived	sex	age	fare
0	1	1	female	29	211.3375
1	1	1	male	0.9167	151.55

Data analysis

There could be various ways to analyze data. Based on the problem statement, we would need to know the general trend of the data in discussion. Statistics Probability Distribution knowledge help here. For gaining insights to understand more around correlations and patterns, data visualization based insights help.
.

# let's see if there are any highly corelated data
# if we observe something, we will remove that 
# one of the feature to avoid bias

import seaborn as sns
sns.pairplot(titanicdf)

# looking at graphs, seems we don't have 
# anything right away to remove.

Data transform – Ordinal/Nominal/Datatype, etc

In order to work through data, it’s easy to interpret once they are converted into numbers (from strings) if possible. This helps them input them into various statistics formulas to get more insights. More details on how to apply numerical modifications to data is discussed here.
.

# There seems to be 3 class of people, 
# lets represent class as numbers
# We don't have info of relative relation of 
# class so we will one-hot-encode it
titanicdf = pd.get_dummies(titanicdf,
                              columns=['pclass'])


# Lets Convert sex to a number 
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
titanicdf["sex"] = le.fit_transform(titanicdf.sex) 

titanicdf.head(2)

	survived	sex	age	fare	pclass_1	pclass_2	pclass_3
0	1	0	29	211.3375	1	0	0
1	1	1	0.9167	151.55	1	0	0

Data imputation: Fill the missing values

There are always some missing data or an outlier. Running algorithms with missing data could lead to inconsistent results or algorithm failure. Based on the context, we can choose to remove them or fill/replace them with an appropriate value.
.

# When we saw the info, lots of age were missing
# Missing ages values filled with mean age. 
 
titanicdf.loc[ titanicdf["age"].isnull(), "age" ] = 
titanicdf["age"].mean()

# titanicdf.loc[ titanicdf["fare"].isnull(), "fare"] 
# = titanicdf["fare"].mean() 
# => can do but we will use another way 

titanicdf.info()

&lt;class 'pandas.core.frame.DataFrame'>
RangeIndex: 1309 entries, 0 to 1308
Data columns (total 7 columns):
 #   Column    Non-Null Count  Dtype  
---  ------    --------------  -----  
 0   survived  1309 non-null   int64  
 1   sex       1309 non-null   int64  
 2   age       1309 non-null   float64
 3   fare      1308 non-null   float64
 4   pclass_1  1309 non-null   uint8  
 5   pclass_2  1309 non-null   uint8  
 6   pclass_3  1309 non-null   uint8  
dtypes: float64(2), int64(2), uint8(3)
memory usage: 44.9 KB

# When we saw the info,
# 1 fare was missing
# Lets drop that one record
titanicdf.dropna(inplace=True)
titanicdf.info()

#<class 'pandas.core.frame.DataFrame'>
#Int64Index: 1308 entries, 0 to 1308

Normalize training data

At times, various data in context are of different scales. In such cases, if the data is not normalized, algorithm can induce bias towards the data that has higher magnitude. Eg, feature A value range is 0-10 and feature B range is 0-10000. In such case, even though a small change in magnitude of A can make a difference but if data is not normalized, feature B will influence results more (which could be not the actual case).
.

X = titanicdf
y = X['survived']
X = X.drop(['survived'], axis=1)

# Scales each column to have 0 mean and 1 std.dev
from sklearn import preprocessing
X_scaled = preprocessing.scale(X)

Split data – Train/Test dataset

It’s always best to split the dataset into two unequal parts. Bigger one to train the algorithm and then then smaller one to test the trained algorithm. This way, algorithm is not biased to just the input data and results for test data can provide better picture.
.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = 
                        train_test_split(X_scaled,y)
X_train.info()

&lt;class 'pandas.core.frame.DataFrame'>
Int64Index: 981 entries, 545 to 864
Data columns (total 6 columns):
 #   Column    Non-Null Count  Dtype  
---  ------    --------------  -----  
 0   sex       981 non-null    int64  
 1   age       981 non-null    float64
 2   fare      981 non-null    float64
 3   pclass_1  981 non-null    uint8  
 4   pclass_2  981 non-null    uint8  
 5   pclass_3  981 non-null    uint8  
dtypes: float64(2), int64(1), uint8(3)
memory usage: 33.5 KB

Run ML algorithm data

Once we have our training dataset ready as per our need, we can apply machine learning algorithms and find which model fits in best.
.

# for now, picking any one of the classifier - KN
# Ignore details or syntax for now
from sklearn.neighbors import KNeighborsClassifier

dtc = KNeighborsClassifier(n_neighbors=5)
dtc.fit(X_train,y_train)

Check the accuracy of model

In order to validate the model, we use test dataset where comparing the predicted value by model to actual data helps us know about ML model accuracy.
.

import sklearn.metrics as met

pred_knc = dtc.predict(X_test)
print( "Nearest neighbors: %.3f" 
      % (met.accuracy_score(y_test, pred_knc)))

Nearest neighbors: 0.817

Voila! With basic workflow, we have a model that can predict the survivor with more than 80% probability.

Download

Entire Jupyter notebook with more samples can be downloaded or forked from my GitHub to look or play around: https://github.com/sandeep-mewara/machine-learning

Currently, it covers examples on following datasets:

Titanic Survivors
Sci-kit Iris
Sci-kit Digits
Bread Basket Bakery

Over time, I would continue building on the same repository with more samples with different algorithms.

Closure

Believe, now it’s pretty clear on how we can attack any problem for an insight using Machine learning. Try out and see for yourself.

We can apply Machine Learning to multiple problems in multiple fields. I have shared a pictorial view of sectors in the AI section that are already leveraging it’s benefit.

Keep learning!

samples GitHub Profile Readme

Sandeep Mewara Github
Sandeep Mewara Learn By Insight
Matplotlib plot samples
Sandeep Mewara Github Repositories

Machine Learning workflow Microsoft .NET5

Microsoft AI support for COVID-19

November 29, 2020November 29, 2020Sandeep Mewara Leave a comment

In January 2020, Microsoft launched AI for Health program. Goal is to use Artificial Intelligence (AI) and data to help improve the health of people worldwide. Post COVID-19 hit world wide, Microsoft took steps to support & empower researchers, nonprofits and policymakers with resources.

There have been more than 50 million confirmed cases of Covid-19 and more than 1.25 million deaths globally
World Health Organization

Research has accelerated with large-scale computing and open data.

Highlights from Grantees World wide

ai-health-covid-response — Credit: Microsoft

As shared here, few of them are:

A grassroots employee volunteer effort at Belgian biopharmaceutical company UCB.
UC Riverside researchers utilize quantum-based methods to more accurately predict the effectiveness of proposed Covid-19 inhibitors.
IHME, a global health research organization at the University of Washington School of Medicine, forecasts the Covid-19 pandemic.
Professor Amanda Randles at Duke University is conducting hundreds of millions of simulations required to help more patients have access to critical ventilators.

Partnership with Organizations

As shared here, few of them are:

Partnering with the White House Office of Science and Technology Policy’s (OSTP) High Performance Computing Consortium to support researchers and academia.
Working with Brown University’s School of Public Health and the Edmond J. Safra Center for Ethics at Harvard to visualize a common set of measures on testing and risk levels to help everyone know where we are with the pandemic and help policymakers guide their response.
Joining University of Oxford and their Government Response Tracker initiative to track and compare government and policy responses to address Covid-19 around the world.

The rapid progress means researchers can more quickly identify potential solutions to combat Covid-19 and provide timely information to policymakers for data-driven decisions that protect communities, cities and regions.
Microsoft blog

Public Information

Using publicly available information from partners, Microsoft have created:

a number of interactive visualizations to provide transparency into Covid-19 trends globally.
a unique measure called Progress to Zero to help everyone understand, progress in reducing Covid-19 cases, hospitalizations and deaths.

My Take

There are some great work being done world wide to fight Covid-19 in all possible ways. Organizations like Microsoft are making an active effort to help partners and grantees for Covid-19 response. There is much to do and supports like these will help mankind avert Pandemic.

Keep exploring!