How to disable UI debugging tools for Xaml in Microsoft Visual Studio

This trips me up because I don’t do it regularly enough to remember it; how to disable the XAML debugging tools widget in UWP apps:

UWP UI Thing

Kind of ironic given what my next post will be.  How to disable:

  1. From main menu: Tools > Options
  2. In Dialog navigation (left pane): Debugging > General
  3. In content pane (right hand side):
    1. Scroll to the bottom
    2. Untick “Enable UI Debugging Tools for XAML”

Thanks, Senthil Kumar B: http://developerpublish.com/disable-ui-debugging-tools-for-xaml-in-visual-studio-2015/

 

Advertisement

The 7 Deadly Sins of Developer Experience with Cristiano Betta #APIDAYSAU

One of the sessions I enjoyed the most at API Days 2018 was Cristiano Betta’s talk on Developer Experience (DX), i.e. how to more effectively engage with developers who are consuming your API’s.  The learnings go beyond developer onboarding specifically, and are applicable to product development in general – which is partially why it was so cool.

Slides here: https://betta.io/blog/2017/11/10/the-seven-sins-of-developer-experience

I also caught-up briefly with Cristiano afterwards where he expanded on a couple of points, as the talk he gave was a slightly shorter version of a longer talk.

An overarching theme was reducing cognitive load through the use of fundamental design principles.  The deadly sins he covered were mainly around information:

  1. Too much
  2. Too soon
  3. Too little
  4. Unstructured
  5. Unsupported
  6. Incomplete

…with “no control” over tools as #7.

There was a variety of points of interest that I noted down, which I’ll briefly cover, but the things that really grabbed my interest were:

  • “Too little / too late”, which is effectively about taking a holistic approach.
  • The idea of measuring and responding to developer friction.

Note – the focus of Cristiano’s talk is around the developer experience in terms of on-boarding rather than the API design itself – for more info on that (developer experience in terms of API design) you might want to check out something like APIs You Won’t Hate.

Too Little, Too Late

This is partially about documentation – but not in the sense of manuals, it’s more about providing enough information when it is needed.  Case in point: resolving errors.

The example Chistiano gave was when a developer is making a call to your API (probably the for the first time) and they encounter an error – e.g. related to input.  Let us say they call your API which provides this response:

{
  "error": "000123 - Invalid input"
}

What you want to avoid is the situation where the developer needs to resort to internet searching.  Sure, you might have it covered in your help documentation:

Developer Guide – Error Codes – Page 421

Error 00123 – Invalid input.  Occurs when you use a boolean on a Friday, on Friday you must use an int: 0 = false and 1 = true.

Your problem is that developers will already have formed familiar techniques for dealing with issues like this, probably using online resources – resources they are familiar with, and which through habit present them with a relatively low cognitive load.

There are many reasons why this is bad: you have no control over the experience, how long it will take and how frustrated they will get – not to forget “OMG, I can’t believe they don’t just say that” and “why is this so unnecessarily hard” comments all over  StackOverflow.com.

What’s The solution?

A better way to do it is to include useful information in the responses error message itself:

{
  "error": "000123 - Invalid input. 
    Occurs when you use a boolean on a Friday, on Friday 
    you must use an int: 0 = false and 1 = true."
}

Yes, you can also have this information in your developer guide.  The trick is including the relevant information when it’s needed; not too much, not too little, and just at the right time.  This leads on nicely to another cool concept…

Developer Friction

Adrian Trenaman’s QCon NY 2017 presentation on Developer Experience included the idea of minimising “the distance between ‘hello, world’ and production”.  In that context he was discussing development in a holistic sense (tooling, environment, and so on) where you are employing developers, but as Cristiano explained to me, you can also look at “developer friction” in the context of developer adoption of your APIs.

In this context, developer friction is effectively the amount of time between (a) making an API call that errors and (b) the first successful call to the same API – or some meaningful variation along those lines, such as the time between developer registration and their first successful API call.

So, imagine that you have 10 developers a day signing up to your API and making their first ‘hello world’ call.  Let’s say 50% of them get an error the very first time they make a an API call, and on average 90% of those developers are able to make a successful call within 2 minutes.  Now compare that to a situation where 80% get an error the first time, and of those 90% take on average 2 hours to make a successful call.  Clearly the second situation has much higher developer friction.

According to Cristiano, some organisations use techniques like this to monitor adoption of their APIs and specifically to help them identify areas where their overall developer experience may need improvement.

Other Gems

I won’t go into these concepts in much detail, and hopefully you should be at least aware of them already – if not I kindly (and strongly) suggest you check them out.  Cristiano’s slide-deck is a great place to start.  It covers a lot more than what I have included here.

Cognitive Load, Overload and Progressive Disclosure

Cognitive load refers to the effort being used in the working memory; cognitive overload is where (for example) a learner is unable to simultaneously process a certain amount of information or tasks.  Solutions to this include:

  • Chunking information up, e.g. into lists of about 8 items, with a useful heading.
  • Applying the 80/20 rule, e.g. call out the small number (~20%) of items that developers are most likely going to be seeking, especially if they are new to your platform, and leave the other 80% accessible but through other navigational means.

That second point is an example of Progressive Disclosure, a great technique for managing cognitive load, covered in detail in the book “Universal Principles of Design”.

Another really interesting pitfall around cognitive load was around asking people question, like on sign-up forms:

sins-062-98658263

As Cristiano explained, this may look simple but it raises a lot of questions in peoples heads – questions which might not seem a big deal to you but can be problematic for others (especially if it’s mandatory):

  1. Who will see this?
  2. Can I change it later?
  3. What do you need it for?

These 3 simple question really resonated with me, and they provide a simple checklist you should consider when reviewing questions you ask your customers.  I know from firsthand experience that questions like this, in some circumstances, often force me to stop and think way more than should be necessary.

Tools Out of Control

This is where community tools and SDK’s are more obvious than yours.  Unfortunately Cristiano didn’t have time to go into this in a lot of depth in terms of solutions, but clearly SDK’s and other tools are a integral part of your offering, and a critical part of DX; therefore it’s critical to have a plan in place for managing these as part of your product.

This is most likely going to include monitoring the community – where they are; understanding what tools they want; staying engaged.

Using Structure

Another nice 3 point list was around structure – i.e. allowing people to navigate through the information you provide them:

  1. Where am I?
  2. Where can I go?
  3. Where did I come from?

Telling a Story

Whilst having information in inherently useful structures is good, you can augment this in key situations (such as developer onboarding) with Story Telling – another technique covered in the Universal Principles of Design.

Cristiano cited Pusher as an example of doing this well – the “hello world” make your first app story.  Here’s the screenshots, as you can see the path from account creation to “hello world” has been streamlined, and users can easily opt out of this if they want.

sins-120-c2bdc012

sins-121-02b36e95

sins-122-84eca6c6

References

#ApiDaysAU

Real World Machine Learning with Susie Sheldrick #APIDAYSAU

I’m at the API Days conference, and one of the first sessions of note was Deep Learning: Real World Applications with Susie Sheldrick, which explored some of the practical real-world challenges related machine learning, based on experience.  I also caught up with her after the session where we expanded on some of curlier questions.

Quick Context: 30 Second Intro to Machine Learning

Susie kicked off with a simple diagram that sums up what machine learning is in comparison to traditional applications:

DeepLearningSusanSheldrick

Machine learning partially turns this model on its head: the solution is able to “learn” its own rules (through training its internal rules model) at much greater scale than some person/team coding them by hand.  So, rather than feeding data and manually created rules into an solution, simply train the solution to produce its own rules.

The Chaser

This nice intro kicked off a mental train of thought for me: in practice the more complete solution probably looks something like this:

DeepLearningAdrianKearns

The end goal is still to build an solution that provides the answers users were seeking, we’re simply using machine learning to help out with the rules.

Devil in the Detail

That all sounds wonderful on paper – or in ivory-tower pixels – but, as should be no surprise, the real world is not so straightforward.

Of critical importance:

  • Understanding the problem you’re trying to solve.
  • Gathering the right data to train the model.

This is much easier said than done, it transpires that:

  • It’s all too easy to inadvertently train bias into the rules model.
  • Tracing exactly how the AI made a specific decision actually turns out to be really hard.

Whilst the second point has obvious implications for developers and testers, both points combined have massive implications for your legal teams, anyone who considers themselves ethical (like you, right?), product owners and anyone at the receiving end of a machine determined decision.

Bias

Susie gave some examples of unexpected and undesirable bias ending up in rule models, such as one experiment that determined prisoners eligibility for parole.  It turns out that the model significantly favored granting parole to white prisoners and was relatively much less favourable to prisoners of colour.   In contrast, in terms of parolees reoffending – the actual results were the exact opposite of the bias.

It turns out that the information used to train the model was “correct” but only in the sense that it faithfully transposed the bias already inherent in the legal system, against people of colour.

True Representation

A related issue isn’t so much of bias in the data, but of bias stemming from an absence of data.  Once more issues of race come to the fore; this time it was a passport application solution that told an Asian gentleman his submitted photo “did not meet our standards” because he was “asleep”.  As you might be able to guess, the model had obviously not been sufficiently trained with data that faithfully represented the entire user base, and therefore could not correctly handle non-european facial features.

Just to be crystal clear, the technology is more than capable of correctly handling a wide range of cases, nuances and subtlety – including racially based facial features.  The actual issue is the correct training of the model – meaning it’s critical to gather the right data, data that covers the entire spectrum of cases.  Not to mention testing and monitoring the behaviour of the solution.

Building an AI Solution: Custom or OOTB?

If you’re about to embark on a project that involves machine learning, one of the practical questions you’ll come up against is whether or not you can use an Out-Of-The-Box (OOTB) solution, or need to custom build something.  Susie’s discussion here was mostly in reference to the rule models specifically.  If you want a model capable of identifying cats in pictures online for you meme generator – you’re in luck, but if you need to correctly identify something more obscure, or more specific, you may have to build this model yourself.  Which is why the stuff above about bias is so important, because you’re going to have to navigate that minefield yourself.

Further Questions

Our chat after the session was very stimulating; a couple of the more curly questions that our conversation provoked were:

How to identify, and test for, unexpected bias?

The obvious ethical reaction to all of this is “great, let’s ensure we keep unwanted bias out of the model and our solution”.  What is much less obvious is how to do that.

Were the team behind the parole example conscious of the bias in that solution?  Let us assume they weren’t aware of it – in such a situation how would they (or you) identify that bias, and in addition, having established an operational solution how would you ensure none was introduced?

This is where, for me, machine learning is like a lens that amplifies human behaviours and bias.  It has the potential to help expose them, but how clearly, how soon, and at what cost?

How will your model react in the event of change over time?  I.e. if there is a fundamental shift in the (data) foundations on which the model was originally conceived and trained?

For example, Google is looking at moving back into the Chinese market, despite pulling out some years ago due to human rights concerns.  Hypothetical example: let’s assume that they have machine learning models built up, based on the data they currently have access to – i.e. does not include China’s current population of 1.3 billion.

What would happen if 1.3 billion Chinese people suddenly have access to a Google solution that is backed by a rules model that was not trained with them in mind?  Sure, Google’s data should be a fair representation of their current global user base, which will include Chinese – but wouldn’t adding 1.3 billion people potentially shift the model?  How will it react?  Will the responses it provides be biased against the new user population because hitherto they were not expected by the model?  Will the model be able to adapt over time, and if so how long will that be?

 

Please note that this post is based on rapidly scrawled notes in session and my recollection of subsequent discussions – my accuracy should be reasonable but may not be perfect.

References: