Prototype Windows 10 app eliminates the napkin by turning sketches into code

Quickly capturing an idea on a paper napkin can lead to lucrative results, but what if those doodles could turn into lines of code? That is what a group of Microsoft Garage Project interns set out to create: A Windows 10 app that enables app designers to draw wireframe sketches and export them into Visual Studio. Simply called Ink to Code, Xamarin funded the project and joined by five additional interns from the Massachusetts Institute of Technology to join and help develop the first prototype app. 

“The sponsoring team and interns were both motivated by a desire to modernize the brainstorming and prototyping process from using napkin and whiteboard sketches, to an experience that is more automated and cohesive with the Visual Studio suite,” Microsoft ‎Product Marketing Manager Lainie Huston said. 

The app is based on Windows 10’s Smart Ink feature injected into the Universal Windows Platform for Microsoft’s Creators Update, which relies on artificial intelligence to improve ink recognition. Of course, that means you need a Windows 10 device that supports Windows Ink and touch-based input, but with this setup, you won’t need to sift through 30 sketches to compile your final code-based design. 

But Ink to Code isn’t meant to completely generate your app from scratch. It merely accelerates the initial layout process by recognizing common design symbols to get you started. According to Huston, the app currently supports buttons, images, labels, text fields, and text paragraphs. It’s a digital canvas where designers and programmers can merge to hammer out ideas and speed up the overall development process. 

“Ink to Code can be used as a more productive canvas in brainstorm meetings, or even more significantly, as a tool that can bridge the gap between collaborators with different levels of design or technical knowledge,” Huston adds. 

As stated, Ink to Code is just a prototype for now, and presumably only visible to Windows 10 devices with touch-based screens; the Microsoft Store didn’t display the linked app on our non-touch laptop. But as the demo video shows, you can use a supported stylus to roughly draw specific elements of the app, such as its rectangular border, an image placement box, a text input field, and so on. The app turns all hand-drawn shapes into perfect lines and circles for a clean look without any manual coding involved. 

Microsoft first introduced Smart Ink in May 2017. Part of the platform is Ink Analysis that recognizes shapes and understands composition, such as text residing inside a roughly drawn rectangle. Ink Presenter collects all the strokes, and Direct Ink deals with rendering the strokes with digital ink. 

“Our goal is to understand user intent and empower developers to turn it into rich digital constructs, as well as to leverage understanding from all parts of the system,” the company says. 

Microsoft introduced similar technology to Sticky Notes with the release of Anniversary Update (Redstone 1) in August 2016.