Decode with Google 2021

Yesterday, I received an email from Google about an upcoming virtual event Decode with Google India 2021. It is a day event, planned to be held on August 26, 2021 compared to a two day event last year.

Details of the event can be found here: Decode with Google

It seems, this time the focus of event is: Contributing from India, Impacting the World

decode-with-google-2021

Join us for the virtual edition of Decode with Google India 2021 and learn about what we’re working on, the state of AI, the opportunities it presents—with a focus on the next billion internet users. You will get to hear live from the Google Tech leaders and you will also have the opportunity to ask leaders some questions and potentially have them answered live on air!

Event highlight shared by Google

If it resonates, Register to join in to learn more and ask queries about how Google engineers, researchers, scientists and product managers based in India are addressing unique Indian challenges, and finding solutions that will benefit people across the world.

Event date: August 26, 2021 12:00 AM – 1:10 PM GMT+5:30

Happy connecting!

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming
Learn Microsoft Tech via Videos LiveTV Streams

Microsoft Build 2021 Highlights

As shared in my earlier post, recently Microsoft held its Build Conference 2021 as a virtual event. It spanned across for more than two days that touched upon various topics. It was not as big as it used to be in person for last several years. There were numerous updates that they shared which I have tried to cover here.

msbuild-highlights

To empower developers, to empower you, to empower the world.

Microsoft vision

Core Themes

Microsoft unveiled a handful of news and tools to empower developers. There were multiple Innovative AI, mixed reality solutions, intelligent cloud tools solutions shared. Core themes for the conference were as depicted below:

build-theme

With so much changing in working mode, Microsoft shared how they are looking to support for Operational excellence with their developer platform. A deep dive session on how to increase developer velocity with Microsoft’s end to end platform was shared during the conference.

Further, data being the key for future transformations, they showcased how Azure Data was being built as a platform that could support AI & data governance.

ms-data-ai
dev-toolchain
ms-ai

They now support building deeper Artificial Intelligence infrastructure for everyone. This would help in easy onboard and usage of Azure services to have more proactive approach to solutions. Details about how to harness the power of data in our applications with Azure can be viewed here.

With Microsoft vision to empower developers, there was a session on how to build cloud native solutions that can run on premise, on edge and on multiple cloud. Recording of the session can be seen here.

ms-cloud-native

Covid has probably made the biggest shift in how we work today. A hybrid future work style is being imagined with new operating models to work, learn or collaborate.

Microsoft Teams is one of the Microsofts fastest growing product with 145 million daily active users to date.

With new working style, transformation seems to be an opportunity where cross device collaboration would be the key. Microsoft surely knows the same and are investing in it. Following session shares on how they are progressing on the collaborative applications journey: Build the next generation of collaborative apps for hybrid work

ms-collaborative-app

Based on the different kind of services, need of support, scalability options and various other factors puts software as a service, a much needed way. Microsoft looked focused on the same and shared on how they are working towards a cloud native SaaS apps composed on top of other clouds and components.

Microsoft also held a session showcasing how they are working towards helping build Metaverse Apps (digital and physical mixing up) incorporating digital twins, IOT, Autonomous systems, Power platforms and Mesh.

digital-twins
autonomous-systems
ms-mesh

Announcements

Like always, there were few key announcements about new initiatives and solutions as mentioned below:

ms-announcements

Real World

Through sessions, Microsoft covered few examples on how various other organizations are leveraging Microsoft solutions to provide awesome customer experience with speed and accuracy.

ms-toyota
Fusion Teams for mission critical delivery app
ms-twitter
Generate captions for live audio conversations using MS Speech service
ms-walmart
Power eCommerce transactions using MS CosmosDB
ms-servicenow
Incident response using MS Teams, Graph & Bot
ms-finastra
Application development using MS Financial Services cloud, Teams & Azure
ms-abinbev
Track bottles till distribution processes using Metaverse stack

Nadella in his keynote also shared that because of Covid, digital transformations have accelerated and has advanced by 10 years.

ms-tech-intensity
ms-gdp10
ms-dev-growth
ms-github-growth

Post pandemic, virtual world will still have a significant role in the new normal. Solutions across industries would want to be connected and continue use the benefits of digital first responders. This would further fuel the innovations and development across technology.

Windows Update

Microsoft’s Build 2021 event didn’t had much of Windows-specific news. Few that came out were:

  • 21H1 windows update was rolled out with multiple security fixes
  • Support for Linux GUI apps on Windows 10 will come later this year
  • Continues promoting ARM – Qualcomm ARM/Snapdragon Developer Kit was announced
  • Fall update (aka Sun-Valley Update) might have renewed UX

Windows10 used by 1.3 billion – work, learn, connect and play

Microsoft

.NET Update

.NET 6 is the next version of .NET, a modern, open-source development platform for building apps for any OS with the best performance and productivity.

.NET 6 completes the unification of the platform and adds new capabilities for building web, native and hybrid apps for Linux, Windows, Mac, iOS and Android with a single codebase. Details can be viewed here: .NET 6 deep dive – what’s new and what’s coming

  • .NET 6 preview 4 available, .NET 6 planned to be released during .NET Conf 2021
  • Visual Studio 2022 under works, preview to be available soon

Imagine Cup 2021

imagine-cup

Make an impact through coding, collaboration, and competition. Innovate with passion to tackle global issues and bring your idea to life in the Imagine Cup.

Imagine Cup

One Winner out of four student teams from across the world was announced. Teams brought their innovations focused on four social good categories – Earth, Education, Healthcare, and Lifestyle. With intention to solve issues in their local and global communities, multiple teams participated and final four were selected after evaluation on various parameters.

Team REWEBA from Kenya were announced as the Imagine Cup 2021 World Champions.

What’s next

Learn, connect, and explore all of the sessions and on-demand content from Microsoft Build anytime, anywhere.

Microsoft

With virtual as normal, Microsoft hinted at having multiple What’s next virtual events to share updates through out the year. For now, they have asked to save the date June 24 as event to share all about the operating system’s updates.

All the MS Build sessions are recorded and can be viewed from here.

Reference: https://mybuild.microsoft.com/home

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming
Learn Microsoft Tech via Videos LiveTV Streams

Microsoft Build 2021

Microsoft Build is an online free event planned on May 25-27, 2021. It’s suggestible to register in order to access the full learning experience, including all sessions and the Connection Zone. All the details of the event are here.

msbuild-social

At Microsoft Build, you’ll leave a better developer than when you arrived. It’s where you can solve challenges, meet the engineers behind the Microsoft platforms you use every day, and connect with a diverse group of coders who want to hone their skills.

Learn the tools and technologies that help you create solutions you can build today. Explore new developer technologies and techniques that will drive your vision and expand your toolkit. Bring your technical questions and imagination and leave with new knowledge and skills.

Event highlight shared by Microsoft

Whether you are a student or an experienced app developer, it promises technical sessions, various experts, product round-tables, One-on-One consultations and other engaging activities to learn.

Happy connecting!

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming
Learn Microsoft Tech via Videos LiveTV Streams WebView2 for WPF .NET Framework

Microsoft WebView2 – A new friend for native apps!

Now, we can embed web content (HTML, CSS, and JavaScript) in our native applications with Microsoft Edge WebView2. Earlier, it was announced only for Win32 C/C++ apps but later, it was announced available for use in .NET 5, .NET Core, and .NET Framework Windows Forms and WPF applications.

It uses the modern Microsoft Edge (Chromium) platform to host web content within native Windows applications.

edge-webview2

In the future, the Evergreen WebView2 Runtime plans to ship with future releases of Windows. Deploy the Runtime with your production app until the Runtime becomes more universally available.

Microsoft recommendation

Power of Native

With growing presence of online world, desktop applications are pushed to use more of online like capabilities. Further, web solution also helps reuse most of the code across different platforms and easy change delivery. This is leading desktop applications more and more towards hybrid approach where best of both (native and web) can be leveraged.

hybrid
Credit: Microsoft

Microsoft WebView2 comes to rescue. It helps build powerful applications with controlled access to native capabilities.

WebView2 uses the same process model as the Microsoft Edge browser. A browser process is associated with only one user data folder. A request process that specifies more than one user data folder is associated with the same number of browser processes.

browser-process-model
Credit: Microsoft

More details about browser process model can be read here.

WebView2 apps create a user data folder to store data such as cookies, credentials, permissions, and so on. After creating the folder, your app is responsible for managing the lifetime of the user data folder, including clean up when the app is uninstalled

Microsoft – Managing user data folder

Microsoft has laid here some best practices for developing secure WebView2 application.

Distribution

When distributing your WebView2 app, ensure the backing web platform, the WebView2 Runtime, is present before the app starts.

By default, WebView2 is evergreen and receives automatic updates to stay on the latest and most secure platform.

  • Evergreen Bootstrapper – a tiny installer that downloads the Evergreen Runtime matching device architecture and installs it locally.
  • Evergreen Standalone Installer – a full-blown installer that can install the Evergreen Runtime in offline environment.
  • Fixed Version – to select and package a specific version of the WebView2 Runtime with your application.

The WebView2 Runtime is a redistributable runtime and serves as the backing web platform for WebView2 apps.

webview2-exception

Download the runtime from here. Supported platforms are mentioned here.

Sample Application

I built a sample WPF application (runs on .NET Framework and not Core) to try WebView2. This was to evaluate how comparatively older .NET applications would work out.

webview2-sample

I tried to display my blog in the WPF application using a WebView2 control. Sample application had capabilities to post message to host application and back as well as hook events as per need.

public MainWindow()
{
    InitializeComponent();

    // NavigationEvents
    webView.NavigationStarting += WebView_NavigationStarting; ;
    webView.SourceChanged += WebView_SourceChanged;
    webView.ContentLoading += WebView_ContentLoading;
    webView.NavigationCompleted += WebView_NavigationCompleted;

    // Embedded at CoreWebView2 level
    InitializeOnceCoreWebView2Intialized();
}

/// <summary>
/// initialization of CoreWebView2 is asynchronous.
/// </summary>
async private void InitializeOnceCoreWebView2Intialized()
{
    await webView.EnsureCoreWebView2Async(null);

    // Hook other events
    webView.CoreWebView2.FrameNavigationStarting += CoreWebView2_FrameNavigationStarting;
    webView.CoreWebView2.HistoryChanged += CoreWebView2_HistoryChanged;

    // For communication host to webview & vice versa
    webView.CoreWebView2.WebMessageReceived += CoreWebView2_WebMessageReceived;
    await webView.CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync("window.chrome.webview.postMessage(window.document.URL);");
    await webView.CoreWebView2.AddScriptToExecuteOnDocumentCreatedAsync("window.chrome.webview.addEventListener(\'message\', event => alert(\'Message from App to WebView2 on navigation!\'));");
}

/// <summary>
/// Web content in a WebView2 control may post a message to the host 
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void CoreWebView2_WebMessageReceived(object sender, CoreWebView2WebMessageReceivedEventArgs e)
{
    // Retrieve message from Webview2
    String uri = e.TryGetWebMessageAsString();
    addressBar.Text = uri;

    // Send message to Webview2
    webView.CoreWebView2.PostWebMessageAsString(uri);
    log.Content = $"Address bar updated ({uri}) based on WebView2 message!";
}

/// <summary>
/// Execute URL
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void ButtonGo_Click(object sender, RoutedEventArgs e)
{
    try
    {
        Uri uri = new Uri(addressBar.Text);

        if (webView != null && webView.CoreWebView2 != null)
        {
            webView.CoreWebView2.Navigate(uri.OriginalString);
        }
    }
    catch (UriFormatException)
    {
        MessageBox.Show("Please enter correct format of url!");
    }
}

/// <summary>
/// Allow only HTTPS calls
/// WebView2 starts to navigate and the navigation results in a network request. 
/// The host may disallow the request during the event.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void WebView_NavigationStarting(object sender, CoreWebView2NavigationStartingEventArgs e)
{
    String uri = e.Uri;
    if (!uri.StartsWith("https://"))
    {
        e.Cancel = true;
        //MessageBox.Show("Only HTTPS allowed!");

        // Inject JavaScript code into WebView2 controls at runtime
        webView.CoreWebView2.ExecuteScriptAsync($"alert('{uri} is not safe, try an https link please.')");
    }
}

Various events run when specific asynchronous actions occur to the content displayed in a WebView2 instance.

navigation-graph
NavigationStartingWebView2 starts to navigate and the navigation results in a network request. The host may disallow the request during the event.
SourceChangedThe source of WebView2 changes to a new URL. The event may result from a navigation action that does not cause a network request such as a fragment navigation.
ContentLoadingWebView starts loading content for the new page.
HistoryChangedThe navigation causes the history of WebView2 to update.
NavigationCompletedWebView2 completes loading content on the new page.
ProcessFailedTo react to crashes and hangs in the browser and renderer processes
CloseTo safely shut down associated browser and renderer processes
Key events

Working with the sample application, I was able to display a webpage, intercept calls both ways and embed message/code to my need. It provides all the capabilities that seems to be needed for a stable web app display control.

Complete sample application can be downloaded from here: https://github.com/sandeep-mewara/WebView2WpfBrowserApp

Reference

https://developer.microsoft.com/en-us/microsoft-edge/webview2/
https://docs.microsoft.com/en-us/microsoft-edge/webview2/gettingstarted/wpf
https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/distribution

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming
Learn Microsoft Tech via Videos LiveTV Streams

Quick look into Machine Learning workflow

Before we jump into various ML concepts and algorithms, let’s have a quick look into basic workflow when we apply Machine Learning to a problem.

ml-header

A short brief about Machine Learning, it’s association with AI or Data Science world is here.

How does Machine Learning help?

Machine Learning is about having a training algorithm that helps predict an output based on the past data. This input data can keep on changing and accordingly the algorithm can fine tune to provide better output.

It has vast applications across. For example, Google is using it to predict natural disasters like floods. A very common use we hear these days in news are usage in Politics and how to attack the demography of voters.

How does Machine Learning work?

Data is the key here. More the data is, better the algorithm can learn and fine tune. For any problem output, there would be multiple factors at play. Some of them would have more affect then others. Analyzing and applying all such findings are part of a machine learning problem. Mathematically, ML converts a problem output as a function of multiple input factors.

Y = f(x)

Y = predicted output
x = multiple factors as an input

How does a typical ML workflow look?

There is a structured way to apply ML on a problem. I tried to put the workflow in a pictorial view to easily visualize and understand it:

ml-workflow

It’s goes in a cycle and once we have some insights from the published model, it goes back into the funnel as learning to make output better.

datascience-process

Roughly, Data scientists spend 60% of their time on cleaning and organizing data

Walk through an example?

Let’s use dataset of Titanic survivors found here and run through basic workflow to see how certain features like traveling class, sex, age and fare are helping us assess survival probability.

Load data from file

Data could be in various format. Easiest is to have it in a csv and then load it using pandas. More details around how to play with data is discussed here.
.

# lets load and see the info of the dataset

titanicdf = pd.read_csv("data-files/titanic.csv")
print(titanicdf.info())
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1309 entries, 0 to 1308
Data columns (total 14 columns):
 #   Column     Non-Null Count  Dtype  
---  ------     --------------  -----  
 0   pclass     1309 non-null   int64  
 1   survived   1309 non-null   int64  
 2   name       1309 non-null   object 
 3   sex        1309 non-null   object 
 4   age        1046 non-null   float64
 5   sibsp      1309 non-null   int64  
 6   parch      1309 non-null   int64  
 7   ticket     1309 non-null   object 
 8   fare       1308 non-null   float64
 9   cabin      295 non-null    object 
 10  embarked   1307 non-null   object 
 11  boat       486 non-null    object 
 12  body       121 non-null    float64
 13  home.dest  745 non-null    object 
dtypes: float64(3), int64(4), object(7)
memory usage: 143.3+ KB
None
# A quick sample view
titanicdf.head(3)
 pclasssurvivednamesexagesibspparchticketfarecabinembarkedboatbodyhome.dest
011Allen, Miss. Elisabeth Waltonfemale290024160211.3375B5S2NaNSt Louis, MO
111Allison, Master. Hudson Trevormale0.916712113781151.55C22 C26S11NaNMontreal, PQ / Chesterville, ON
210Allison, Miss. Helen Lorainefemale212113781151.55C22 C26SNaNNaNMontreal, PQ / Chesterville, ON
Data cleanup – Drop the irrelevant columns

It’s not always that the data captured is the only data you need. You will have a superset of data that has additional information which are not relevant for your problem statement. This data will work as a noise and thus it’s better to clean the dataset before starting to work on them for any ML algorithm.
.

# there seems to be handful of columns which 
# we might not need for our analysis 
# lets drop all those column that we know 
# are irrelevant
titanicdf.drop(['embarked','body','boat','name',
'cabin','home.dest','ticket', 'sibsp', 'parch'],
axis='columns', inplace=True)

titanicdf.head(2)
 pclasssurvivedsexagefare
011female29211.3375
111male0.9167151.55
Data analysis

There could be various ways to analyze data. Based on the problem statement, we would need to know the general trend of the data in discussion. Statistics Probability Distribution knowledge help here. For gaining insights to understand more around correlations and patterns, data visualization based insights help.
.

# let's see if there are any highly corelated data
# if we observe something, we will remove that 
# one of the feature to avoid bias

import seaborn as sns
sns.pairplot(titanicdf)

# looking at graphs, seems we don't have 
# anything right away to remove.
titanic-graph
Data transform – Ordinal/Nominal/Datatype, etc

In order to work through data, it’s easy to interpret once they are converted into numbers (from strings) if possible. This helps them input them into various statistics formulas to get more insights. More details on how to apply numerical modifications to data is discussed here.
.

# There seems to be 3 class of people, 
# lets represent class as numbers
# We don't have info of relative relation of 
# class so we will one-hot-encode it
titanicdf = pd.get_dummies(titanicdf,
                              columns=['pclass'])


# Lets Convert sex to a number 
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
titanicdf["sex"] = le.fit_transform(titanicdf.sex) 

titanicdf.head(2)
 survivedsexagefarepclass_1pclass_2pclass_3
01029211.3375100
1110.9167151.55100
Data imputation: Fill the missing values

There are always some missing data or an outlier. Running algorithms with missing data could lead to inconsistent results or algorithm failure. Based on the context, we can choose to remove them or fill/replace them with an appropriate value.
.

# When we saw the info, lots of age were missing
# Missing ages values filled with mean age. 
 
titanicdf.loc[ titanicdf["age"].isnull(), "age" ] = 
titanicdf["age"].mean()

# titanicdf.loc[ titanicdf["fare"].isnull(), "fare"] 
# = titanicdf["fare"].mean() 
# => can do but we will use another way 

titanicdf.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1309 entries, 0 to 1308
Data columns (total 7 columns):
 #   Column    Non-Null Count  Dtype  
---  ------    --------------  -----  
 0   survived  1309 non-null   int64  
 1   sex       1309 non-null   int64  
 2   age       1309 non-null   float64
 3   fare      1308 non-null   float64
 4   pclass_1  1309 non-null   uint8  
 5   pclass_2  1309 non-null   uint8  
 6   pclass_3  1309 non-null   uint8  
dtypes: float64(2), int64(2), uint8(3)
memory usage: 44.9 KB
# When we saw the info,
# 1 fare was missing
# Lets drop that one record
titanicdf.dropna(inplace=True)
titanicdf.info()

#<class 'pandas.core.frame.DataFrame'>
#Int64Index: 1308 entries, 0 to 1308
Normalize training data

At times, various data in context are of different scales. In such cases, if the data is not normalized, algorithm can induce bias towards the data that has higher magnitude. Eg, feature A value range is 0-10 and feature B range is 0-10000. In such case, even though a small change in magnitude of A can make a difference but if data is not normalized, feature B will influence results more (which could be not the actual case).
.

X = titanicdf
y = X['survived']
X = X.drop(['survived'], axis=1)

# Scales each column to have 0 mean and 1 std.dev
from sklearn import preprocessing
X_scaled = preprocessing.scale(X)
Split data – Train/Test dataset

It’s always best to split the dataset into two unequal parts. Bigger one to train the algorithm and then then smaller one to test the trained algorithm. This way, algorithm is not biased to just the input data and results for test data can provide better picture.
.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = 
                        train_test_split(X_scaled,y)
X_train.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 981 entries, 545 to 864
Data columns (total 6 columns):
 #   Column    Non-Null Count  Dtype  
---  ------    --------------  -----  
 0   sex       981 non-null    int64  
 1   age       981 non-null    float64
 2   fare      981 non-null    float64
 3   pclass_1  981 non-null    uint8  
 4   pclass_2  981 non-null    uint8  
 5   pclass_3  981 non-null    uint8  
dtypes: float64(2), int64(1), uint8(3)
memory usage: 33.5 KB
Run ML algorithm data

Once we have our training dataset ready as per our need, we can apply machine learning algorithms and find which model fits in best.
.

# for now, picking any one of the classifier - KN
# Ignore details or syntax for now
from sklearn.neighbors import KNeighborsClassifier

dtc = KNeighborsClassifier(n_neighbors=5)
dtc.fit(X_train,y_train)
Check the accuracy of model

In order to validate the model, we use test dataset where comparing the predicted value by model to actual data helps us know about ML model accuracy.
.

import sklearn.metrics as met

pred_knc = dtc.predict(X_test)
print( "Nearest neighbors: %.3f" 
      % (met.accuracy_score(y_test, pred_knc)))
Nearest neighbors: 0.817


Voila! With basic workflow, we have a model that can predict the survivor with more than 80% probability.

Download

Entire Jupyter notebook with more samples can be downloaded or forked from my GitHub to look or play around: https://github.com/sandeep-mewara/machine-learning

Currently, it covers examples on following datasets:

  • Titanic Survivors
  • Sci-kit Iris
  • Sci-kit Digits
  • Bread Basket Bakery

Over time, I would continue building on the same repository with more samples with different algorithms.

Closure

Believe, now it’s pretty clear on how we can attack any problem for an insight using Machine learning. Try out and see for yourself.

We can apply Machine Learning to multiple problems in multiple fields. I have shared a pictorial view of sectors in the AI section that are already leveraging it’s benefit.


Keep learning!

samples GitHub Profile Readme
Learn Python – Beginners step by step – Basics and Examples
Sandeep Mewara Github
Sandeep Mewara Learn By Insight
Matplotlib plot samples
Sandeep Mewara Github Repositories
Learn Microsoft Tech via Videos LiveTV Streams
Machine Learning workflowMicrosoft .NET5

Microsoft AI support for COVID-19

In January 2020, Microsoft launched AI for Health program. Goal is to use Artificial Intelligence (AI) and data to help improve the health of people worldwide. Post COVID-19 hit world wide, Microsoft took steps to support & empower researchers, nonprofits and policymakers with resources.

ai-health

There have been more than 50 million confirmed cases of Covid-19 and more than 1.25 million deaths globally

World Health Organization

Research has accelerated with large-scale computing and open data.

Highlights from Grantees World wide

ai-health-covid-response
Credit: Microsoft

As shared here, few of them are:

  • A grassroots employee volunteer effort at Belgian biopharmaceutical company UCB.
  • UC Riverside researchers utilize quantum-based methods to more accurately predict the effectiveness of proposed Covid-19 inhibitors.
  • IHME, a global health research organization at the University of Washington School of Medicine, forecasts the Covid-19 pandemic.
  • Professor Amanda Randles at Duke University is conducting hundreds of millions of simulations required to help more patients have access to critical ventilators.

Partnership with Organizations

As shared here, few of them are:

  • Partnering with the White House Office of Science and Technology Policy’s (OSTP) High Performance Computing Consortium to support researchers and academia.
  • Working with Brown University’s School of Public Health and the Edmond J. Safra Center for Ethics at Harvard to visualize a common set of measures on testing and risk levels to help everyone know where we are with the pandemic and help policymakers guide their response.
  • Joining University of Oxford and their Government Response Tracker initiative to track and compare government and policy responses to address Covid-19 around the world.

The rapid progress means researchers can more quickly identify potential solutions to combat Covid-19 and provide timely information to policymakers for data-driven decisions that protect communities, cities and regions.

Microsoft blog

Public Information

Using publicly available information from partners, Microsoft have created:

  • a number of interactive visualizations to provide transparency into Covid-19 trends globally.
  • a unique measure called Progress to Zero to help everyone understand, progress in reducing Covid-19 cases, hospitalizations and deaths.
india-progress-to-zero

My Take

There are some great work being done world wide to fight Covid-19 in all possible ways. Organizations like Microsoft are making an active effort to help partners and grantees for Covid-19 response. There is much to do and supports like these will help mankind avert Pandemic.


Keep exploring!

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming
Learn Microsoft Tech via Videos LiveTV Streams Microsoft .NET5

Microsoft .NET 5 – all you need to know

As announced earlier, Microsoft released .NET 5 on Nov. 10 during .NET Conf 2020. It’s a major release with many new features and improvements to .NET Core 3. Keeping cross platform support and open source development as key base, going forward it kind of merges off .NET Framework and .NET Core.

dotnet-header

Plan

Microsoft started it’s journey of cross platform and open source in 2015. .NET 5 is a major milestone of this transition as .NET Framework 4.8 was the last major version of .NET Framework. Microsoft published a blog to explain how .NET 5 relates to .NET Framework.

.NET 5.0 is the main implementation of .NET going forward and .NET Framework 4.x is still supported.

dotnet-schedule
Source: Microsoft
dotnet-unified
Source: Microsoft

.NET 5 has been called as .NET Core vNext for quite some time now. It has following key principles:

– Produce a single .NET runtime and framework that can be used everywhere and that has uniform runtime behaviors and developer experiences.

– Expand the capabilities of .NET by taking the best of .NET Core, .NET Framework, Xamarin and Mono.

– Build that product out of a single code-base that developers (Microsoft and the community) can work on and expand together and that improves all scenarios.

Microsoft

Highlights

There are many improvements in .NET 5 like:

  • Performance across many components
  • Performance in .NET Libraries
  • Language C# 9 & F# 5
  • Application deployment options
  • Platform scope (includes Windows Arm64 & WebAssembly)

Details about these enhancements are here.

dot.net and Bing.com are already running on .NET 5 for months now

References

IDE

You need Visual Studio 16.8 or later to use .NET 5.0 on Windows and the latest version of Visual Studio for Mac on macOS. Latest C# extension for Visual Studio Code already supports .NET 5.0 and C# 9.

Impacts

There are few breaking changes with upgrade to .NET 5:

  • while migrating from version 3.1 of .NET Core, ASP.NET Core, or EF Core to version 5.0 of .NET, ASP.NET Core, or EF Core are captured here.
  • in Roslyn in C# 9.0 introduced with .NET 5 are captured here.
  • for migration from .NET Framework to .NET Core are captured here.
  • obsolete features in .NET 5 are captured here.

Deprecated

.NET 5 does not have few of the known technologies:

  • ASP.NET WebForms – Microsoft’s recommendation is to move to Blazor
  • Windows Communication Foundation (WCF) – Microsoft’s recommendation is to use gRPC
  • Windows Workflow Foundation (WWF) – recommendation is to look at CoreWF, a form of WF runtime

Advantages

.NET 5 helps if you have cross-platform needs or targeting microservices or want to use Docker containers. Based on how it is setup underlying, it helps in designing high-performance and scalable systems.

.NET 5 supports side-by-side installation of different versions of the .NET 5 runtime on the same machine. This allows multiple services or applications on the same server running on their own version of .NET 5 variant.

A detailed analysis and shareout can be read here.

Wrap Up

.NET 5 is a major step of the .NET journey planned ahead. Next would be .NET 6 next year (late in 2021), which will also be a long term (multi year) supported (LTS) version.

Microsoft is working towards a defined vision – same .NET API and languages working across operating system, application types and architectures.

With active support to previous .NET versions, we have time to assess, plan and adapt the new path.


Keep exploring!

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming
Learn Microsoft Tech via Videos LiveTV Streams Microsoft .NET5

Find missing number from 1 to N?

Last week, there was a discussion in my team on the problem of finding missing number(s). We had different thoughts and approaches and thus I thought to share it across.

find-missing-number

Problem statement was something like:

– An array of size (n) has numbers from 1 to (n+1). Find the missing one number.

– An array of size (n) has numbers from 1 to (n+2). Find the missing two numbers.

First thought …

Keep track of numbers found while traversing. At the end, use it to find the missing number. So kind of brute force approach.

We can maintain a hash or a boolean array of n size and keep on updating the hash or the array index location based on number found while traversing. Use it now to find the missing number. It would cover both one as well as two missing numbers case.

This would have two traversals of n (one for filling in the structure and another to find the missing one). Thus overall, time complexity of O(n). This would need an extra space to keep track of all numbers found and thus a space complexity of O(n).

Q: Now, can we avoid extra space or two times traversal?

Second thought …

We know how to calculate the sum of n natural numbers, i.e.: n*(n+1)/2. With it, we can traverse the given array and keep a sum of all numbers. Difference of the sum from formula to sum found would give us the missing number. Nice!

# Keep track of sum
def sumOfGivenNumbers(nos, n):
    sum = 0
    # calculate sum
    for i in range(0, n):
        sum += nos[i]
    return sum

# Input
numbers = [4, 2, 1, 6, 5, 7] 

# number range 
n = len(numbers) + 1
expectedSum = n*(n+1)/2
numbersSum = sumOfGivenNumbers(numbers,len(numbers))

print('Missing number:', expectedSum - numbersSum)

# Output
# Missing number: 3.0

This would help is solve one missing number in single traversal, thus time complexity of O(n). No extra space was used and thus space complexity of O(1).

Q: Can we extend this to two missing numbers now?

Yes, we can extend it. Along with sum, we can also use the product of n natural number as an expression. With it, we will have two equations and two numbers to find:

Missing1 = x1
Missing2 = x2
Sum of provided numbers = N1
Sum of n Natural numbers = N
Product of provided numbers = P1
Product of n Natural numbers = P

x1 + x2 + N1 = N
x1 * x2 * P1 = P

We can solve it to find the two missing numbers. It does have the quadratic flavor associated though. It maintains the time complexity as O(n) and space complexity as O(1). Nice!

Q: Does the solution help with large integers? Think of possible overflow?

Third thought …

Let’s look at possible way for 1 missing number first.

We will traverse through all the numbers of the array. While doing so, maintain a number that would be sum of all numbers traversed so far reduced by sum of all the indexes traversed (+1 if index starts from 0). It is still making use of n natural numbers (in form of indexes) to keep a check on sum to a defined limit.

# Keep track of sum
def getMissingNumber(nos, n):
    sum = 0
    # calculate sum
    for i in range(0, n):
        sum += (i+1)
        sum -= nos[i]

    # last number to add from n+1 natural nos.
    return sum+n+1

# Input
numbers = [4, 2, 1, 6, 5, 7] 

missingNumber =getMissingNumber(numbers,len(numbers))

print('Missing number:', missingNumber)

# Output
# Missing number: 3.0

This looks good and we maintain the same complexities along with solving for overflow.

We can probably try a similar thing for two missing numbers where we keep on multiple and divide the traversed number by index but it still could have overflow issues in worst case. Further, there could be round off issues.

Fourth thought …

Looking more, it seems we can make use of XOR operation to find the missing numbers. We can make use of XOR’s property to nullify the duplicate pair. We will take XOR of provided numbers and XOR of natural numbers. Combining both again with XOR will leave with missing numbers XOR output.

For one missing number, this would be easy and covers all the hurdles discussed earlier keeping same performance.

# Keep track of XOR data
def getMissingNumber(nos, n):
    x1 = nos[0]
    xn = 1

    # start from second
    for i in range(1, n):
        x1 = x1 ^ nos[i]
        xn = xn ^ (i+1)
    
    # last number to XOR
    xn = xn ^ (n+1)

    # find the missing number
    return x1 ^ xn

# Input
numbers = [4, 2, 1, 6, 5, 7] 

missingNumber =getMissingNumber(numbers,len(numbers))

print('Missing number:', missingNumber)

# Output
# Missing number: 3.0

For two missing numbers, using a similar logic of XOR above, we will have an output of XOR value of both missing numbers. Now, given the XOR value will not be zero, the XOR corresponding valid bit in missing1 and missing2 must be different to make it “1”.

# Keep track of XOR data
def getTwoMissingNumber(nos, n):
    x1 = nos[0]
    xn = 1

    # start from second
    for i in range(1, n-2):
        x1 = x1 ^ nos[i]
        xn = xn ^ (i+1)
    
    # last numbers to XOR
    xn = xn ^ (n-1) ^ (n)

    # XOR of two missing numbers
    # Any set bit in it must be 
    # set in one missing and 
    # unset in other missing number 
    XOR = x1 ^ xn

    # Get a rightmost set bit of XOR  
    set_bit_no = XOR & ~(XOR-1) 
  
    # Divide elements in two sets 
    # by comparing rightmost set bit of XOR 
    # with bit at same position in each element. 
    x = 0
    y = 0 
    for i in range(0,n-2): 
        if nos[i] & set_bit_no:    
            # XOR of first set in nos[]  
            x = x ^ nos[i]   
        else: 
            # XOR of second set in nos[]  
            y = y ^ nos[i]   

    for i in range(1,n+1): 
        if i & set_bit_no: 
            # XOR of first set in nos[]  
            x = x ^ i        
        else: 
            # XOR of second set in nos[]  
            y = y ^ i
    
    print ("Missing Numbers: %d %d"%(x,y)) 
    return

# Input
numbers = [4, 2, 1, 6, 7, 5] 

# total length will be provided count+2 missing ones
getTwoMissingNumber(numbers, len(numbers) + 2)

# Output
# Missing Numbers: 3 8

This overcomes the overflow issue and was easier to solve (compared to solving a quadratic equation). Though it took more than one traversal, overall it maintains the time complexity as O(n) and space complexity as O(1). Nice!

Closure …

There could be multiple ways to solve for one or more missing numbers. One can look at it based on ease and need.


Keep solving!

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming
Learn Microsoft Tech via Videos LiveTV Streams

POC Guide for Developers by Microsoft

Few weeks back, Microsoft Azure published a white paper about a Proof Of Concept (POC) guide for developers. Intent of the paper is to share what POC is and how it would help minimize risk and reduce cost while exploring new technology or an idea.

proof-of-concept

Guide uses Azure application as an example on how to create and execute a POC.

POC guide can be downloaded from here.

Brief Overview

Guide touch-bases and share details around:

  • POC Introduction
  • Steps of a POC
  • Example 1: Sample Azure web app
  • Example 2: Sample Azure chatbot

Additionally, it covers overview of Azure and where to find interactive learning path for beginners. It also shares additional resources that can help develop an Azure based application.

What is Proof of Concept?

Proof of concept (POC) is validating an idea or a concept about it’s feasibility, capability and provides helpful insights. They are scoped to limited features and are generally done in a quick and dirty manner to acquire some metrics.

A proof of concept is an important first step in cultivating business innovations.

How to prepare for and start your POC?

We can get a good POC following a defined set of basic steps:

  1. Defining a goal with success criteria
  2. Setting up a timeline and cost boundaries
  3. Scoping the feature(s) for the POC
  4. Designing, Implementing & Testing
  5. Measure metrics to deduce insights
  6. Take decisions based on the insights

Closing Thoughts

POCs are great way to validate an idea or a technology or a concept. With it, we can take a data back decision for a project that would reduce risk, provide a higher success probability and a conscious cost decision. It also help gauge probable next steps needed once adopted as a mainstream solution.

Provided document is a nice compile and provides a systematic approach for building a POC. With couple of examples, it demonstrates the steps in action. Definitely worth a read.

Don’t ever shy away from building and using a POC to explore and learn.


Keep exploring!

Sandeep Mewara Github
News Update
Tech Explore
Data Explore
samples GitHub Profile Readme
Learn Machine Learning with Examples
Machine Learning workflow
What is Data Science
Word Ladder solution
What is Dynamic Programming
Learn Microsoft Tech via Videos LiveTV Streams

Learn AI ML with Netflix’s Fei Fei!

Off late, Microsoft has been working on providing more and more learning materials. This is as per their Global Skills Initiative aimed at helping 25 million people worldwide acquire new digital skills.

global-initiative

Our goal is to ignite the passion to solve important problems relevant to their lives, families and communities.

Microsoft

Last week, they partnered with Netflix to release a new learning experience featuring a young female hero Fei Fei, who has a passion for science and explores space.

Newly launched

Microsoft has launched three new modules under Explore Space with “Over the Moon” learning path. These modules will help learn basic concepts of data science, artificial intelligence and machine learning:

  1. Plan a Moon Mission using the Python Pandas Library
  2. Predict Meteor Showers using Python and VC Code
  3. Use AI to Recognize Objects in Images using Azure Custom Vision
fei-fei-explore

The movie’s story takes place in a beautifully animated universe and tackles problems real-life space engineers face.

How is it?

They cover the basic workflow of a machine learning problem similar to what I shared in my previous article here.

Exercises also provide a professional experience as they are built on Visual Studio Code and use Azure Cognitive Services.

Looking at it, seems a fun way to learn and explore the data world. Microsoft is really trying to make things simple and available to all. An effort worth to try out. Would recommend and ask to give it a shot.


Keep learning!

samples GitHub Profile Readme
Learn Python – Beginners step by step – Basics and Examples
Sandeep Mewara Github
Sandeep Mewara Learn By Insight
Matplotlib plot samples
Sandeep Mewara Github Repositories
Learn Microsoft Tech via Videos LiveTV Streams
Machine Learning workflow