Under the Hood with Windows Universal Projects

One of the main themes coming out of the keynotes and content at this year’s //Build 2014 conference was the continued convergence of the Windows Store and Windows Phone development platforms.  One exciting result of the fact that both these platforms now target very similar Windows Runtimes (WinRT and WinPRT, respectively) is the support for the new “Universal Apps” in Visual Studio 2013 Update 2.  (Another exciting result is that the “Programming the Windows Runtime by Example” book that Jeremy Likness and I wrote is even MORE relevant!) Projects created using the Universal App templates in Visual Studio include primary projects for each targeted platform as well as a shared project whose content “magically” gets included into each of the head projects.  Currently, only Windows 8.1 and Windows Phone 8.1 primary projects are supported, though apps targeting the Xbox One will one day be included, and Xamarin has indicated they’d be joining the fun soon. image
A Universal Apps Solution, showing Windows, Windows Phone, and Shared projects

I won’t spend a ton of time discussing the “how-to” aspect of Universal Apps projects – Jeff Prosise has published a series of blog posts that present the basics of working with those projects:

I want to focus on the mechanisms working behind-the-scenes that enables these things to work.  How does the Shared content make its way into the app projects in the solution, and why does it behave differently from other sharing mechanisms?  I find that understanding the inner workings of things leads to better decision making when employing them…perhaps that’s why I enjoy reading Jeff Richter’s books.

Where did they put the hood latch on this model?

So what is going on under the hood on these projects?  To understand that we need to get inside of the projects themselves.  Remember that most of the Visual Studio project files you work with (csproj, vbproj, etc) are actually MSBuild XML files.  You can examine this yourself by either opening the project files directly in a text editor (enhanced editors like Notepad++ or Sublime Text offer a better visual experience than good-old Notepad, but in a pinch, Notepad does work.)  You can also examine the inner content of a project file from Visual Studio itself with the following steps (or see here):

  • Right click on the project you want to work with in the Solution Explorer and select Unload Project
  • Right click on the project again and click Edit <ProjectName>

Regardless of what editor you choose to use, the content of your project files will follow the MSBuild Project File XML schema that is described here.  One of the jobs of the Visual Studio IDE is to properly maintain the various MSBuild schema elements as you add, remove, or rearrange items within your projects, but nothing prevents you from doing that manually, and there are actually a few MSBuild “tricks” that you can take advantage of that aren’t provided by the Visual Studio IDE.  Keith Rome has a blog post covering Wildcard Inclusions in C# Project Files that takes specific advantage of this capability.

Pay no attention to that man behind the curtain!

With one of the application project files open for XML editing, you can start to explore the magic of the Shared project.  Scroll down to the bottom of the file and locate the Import element that includes the attribute ‘Label = “Shared”’”:

<Import Project="..\UnderTheHood.Shared\UnderTheHood.Shared.projitems" 
Label="Shared" />

The purpose of the Import Element is to bring the contents of one project file into another project file, and a quick check of the MSDN docs suggests that it’s been around since at least .NET 2.0!  In this case, it is bringing in the contents of a file with a projitems extension from the shared project’s directory.  Note however that this file is NOT the shproj file that defines the Shared Project! (What gives?!) image
The Shared project extension is actually NOT what is referenced in the project files’ Import Element!

NOTE: You may also want to notice that there’s a Root Namespace property.  Setting this value determines the namespace that is given by default to new code files created within this shared project.

The projitems file is what the shared project creates/maintains.  If you open a File Explorer window to the Shared project file location, you will find this file.  Opening the file in a text editor (you can’t open it in Visual Studio while the shproj file is loaded, since it is already being referenced) will show that it is just another MSBuild file that includes the files that have been added to it  (if your text editor supports change notifications, add a new file to the Shared project while the projitems file is opened in the text editor and notice that when you save the project, the projitems file will be updated.)

So as you add, remove, or manipulate items in the shared project, the contents of the projitems file is updated, and this file’s contents in turn are simply incorporated into your app project files via the Import elements in their MSBuild XML.  This is why you can use #if declarations that are scoped to one project or another to differentiate behavior.

But wait, there’s (partially) more!

Remember that there’s more that you can do to affect your shared files’ integration behavior than simply throwing a bunch of #if and #else blocks throughout the shared code (ew!).  Don’t forget about partial classes and their often-overlooked cousins, partial methods. Partial methods first came on to the scene as a tool to allow developers to introduce code that strategically extended designer-generated files without having to worry about designer regeneration obliterating their code without having to create subclasses and override virtual methods.  When a partial class includes a partial method, private methods can call into that partial method, and partial extensions can choose whether or not to provide an implementation of the partial method. Wait…what?  Perhaps an example. I’ve added a new class to the shared project:

namespace UnderTheHood
{
    public partial class PartialClass
    {
        public void DoSomethingInteresting()
        {
            DoSomethingPlatformSpecific();
        }

        partial void DoSomethingPlatformSpecific();
    }
}

Note the lack of implementation for the DoSomethingPlatformSpecific method. Now in the Windows 8.1 project I’ve provided an implementation of the DoSomethingPlatformSpecific method:

namespace UnderTheHood
{
    public partial class PartialClass
    {
        partial void DoSomethingPlatformSpecific()
        {
            Debug.WriteLine("I am a Windows 8.1 implementation.");
        }
    }
}

And a slightly different implementation in the Windows Phone 8.1 project:

namespace UnderTheHood
{
    public partial class PartialClass
    {
        partial void DoSomethingPlatformSpecific()
        {
            Debug.WriteLine("I am a Windows Phone 8.1 implementation.");
        }
    }
}

Now calling the public DoSomethingInteresting method will call into whatever platform-specific DoSomethingPlatformSpecific implementation is in the current project, allowing for platform-specific customizations to occur in the platform-specific projects, rather than mixing them in the shared projects with #if and #else statements.  If new platforms are added to the Universal Project “universe” (sorry), you don’t have to worry about the #if statements becoming these big messes that include a lot of code that is greyed out by Visual Studio depending on which project it thinks is currently in scope. Bear in mind that partial methods differ from abstract methods in that they do not have to be overridden/implemented.  If no implementation is provided, the call into the partial method won’t fail or cause any negative side effects…it is simply a no-op.  As a consequence, there are a few rules to follow when working with partial methods:

  • They must return void (if they had return types, an implementation would be required)
  • That can have ref parameters, but not out parameters (if no implementation is provided, the ref parameter is unmodified)
  • Partial methods are inherently private in scope.
  • They can be static, and can be generic.
  • Delegates can only refer to implementations of static methods.

The bottom line

So hopefully now you have a better understanding of how these magical new shared projects work in Visual Studio.  Much of the technology they use has been in the DNA of our project files for a while.  Visual Studio 2013 Update 2 has been enhanced to include some nice new IDE updates in order to surface these features in a way that makes them very useful for writing reusable code.  This new compile-time reusability combines quite nicely with the post-compile reusability that is offered by Portable Class Libraries. If you want to learn more about them, Jeff Prosise will be delivering a session on universal apps at the Software Design & Development conference in London next month. The talk will delve into what universal apps are and how they’re structured, and he promises to have plenty of samples to share as he goes over the ins and outs of writing apps that target separate but similar run-times.  You might also benefit from watching Jeff’s video entitled Introduction to WinRT on WintellectNOW. If you’re not already a subscriber, you can sign up with the promo code Garland-13 and enjoy two weeks of access for free. Also, did I happen to already mention that Jeremy Likness and I had written a book about using the Windows Runtime?

Adding Intelligence to the IntelliSense for Windows Azure Mobile Services

Some of the members of the Visual Studio Windows Azure Tools & Mobile Services teams recently published instructions for “Enabling IntelliSense for Mobile Services JavaScript in Visual Studio.”  Since working with the Windows Azure Mobile Services (WAMS) server-side JavaScript files can be a little bit of a blind experience – debugging happens primarily through strategic placement of console.log statements and frequent examination of the Mobile Services server script reference on MSDN – even the littlest help goes a long way…though I am NOT saying that his IntelliSense support only offers “the littlest help.”

To get started, the JavaScript IntelliSense definition file can be downloaded from this link.  Additionally, there’s a ReadMe document that provides some supporting information available at this link.  The ReadMe file points out that the IntelliSense definition – in its current incarnation – is somewhat limited in its coverage (hey, it’s a start!)

Placing and Referencing the IntelliSense Definition File

The original article references two mechanisms for actually directing Visual Studio to make use of the reference file: adding an explicit reference to the precise path of the definition file on the local file system, and adding a reference to the definition file through the Tools/Options/Text Editor/JavaScript/IntelliSense/References panel in Visual Studio (under “Implicit (Web)”.)  Pros and cons of each approach were outlined, including:

  • Providing the path to the file within the script files themselves is a bit more work, since every script file needs to be modified;
  • Adding the Visual Studio reference results in a “pollution” of the JavaScript IntelliSense since Visual Studio doesn’t discriminate as to whether or not you’re working on Mobile Services scripts in order to choose what definition files to consult.

Now that the stage is set, I’d like to offer some tips for how to extract a little bit more from these IntelliSense files.

A Slight Change to Definition File References

There’s a small alteration that can be made to the list above to enable referencing the definition file.  My concern with the presentation of the approaches listed above is that they require the definition file to be in the same path location on every machine that is used to edit the JavaScript contents.  In my experience, this is a pain in the neck to achieve – even simply spanning my own development machines.  However, the JavaScript IntelliSense tooling does support relative paths.  With that in mind, I’d suggest putting the IntelliSense file into the WAMS script Shared directory and uploading it through Git (for more information on using the Microsoft Visual Studio Tools for Git to manage WAMS script files, see my previous article here), then adding relative references to “../shared/mobileservices.intellisense.js” either in the script files or in the Visual Studio settings dialog.

The screenshots below show this in use through the Visual Studio settings:

image

image

(* a note about working with the Visual Studio settings – if you’ve added your file path using absolute or relative paths and find that IntelliSense still is not coming up, it may be due to inserting the definition file reference in the wrong “Reference Group”, such as Implicit (Windows) – I call this out because it bit me MULTIPLE times.  Reference Groups are discussed in the MSDN documentation here.)

Note that since the file path is relative, when opening/working on JavaScript files in an HTML Web project, unless there’s a matching “mobileservices.intellisense.js” file within a folder titled “shared” relative to the file being worked on in that project, the WAMS IntelliSense won’t be included.

Likewise, including the relative file at the beginning of the WAMS JavaScript file itself has the same results:

image

While the upside to this approach is that other developers/other development machines acquire the IntelliSense simply by synchronizing the client and server repositories, there is a downside to this approach.  Interacting with scripts opened through the Windows Azure Mobile Services node in the Server Explorer will not include IntelliSense.  When these files are opened in Visual Studio, they are downloaded and opened from a temporary location on the local file system, so the Shared files are not placed in the location specified by the relative path.  I’m OK with this, as I’ve grown to prefer to use the source-control managed file editing and retrieval, and the Server Explorer doesn’t yet provide access to API Scripts or Shared Script content (though I have no doubt that it will soon…but I still like the “safety net” offered by working through Source control and the ease with which I can by using the integrated Microsoft Visual Studio Tools for Git.)

Providing Context

One of the problems with the IntelliSense solution is that there’s no information providing Visual Studio with insight as to how to handle the parameters in the script signatures.  The good news is that is not hard to do, and since these objects are frequently referenced within the WAMS scripts, being able to allow them to join in the “IntelliSense party” provides a lot of help.

Having Visual Studio include IntelliSense for these parameters is simply a matter of adding a JavaScript XML Documentation Comment for the method signature.  The IntelliSense comments for each of the table script methods follow. (Custom API scripts will be discussed in a second.)

Read:

function read(query, user, request) { /// <param name="query" type="Query"></param> /// <param name="user" type="User"></param> /// <param name="request" type="Request"></param>

Insert (Update uses the same XML comments):

function insert(item, user, request) { /// <param name="item" type="Object"></param> /// <param name="user" type="User"></param> /// <param name="request" type="Request"></param>

Delete:

function del(id, user, request) { /// <param name="id" type="Number"></param> /// <param name="user" type="User"></param> /// <param name="request" type="Request"></param>

With the comments in place that describe the types of the incoming parameters, the server-side scripts now get IntelliSense within Visual Studio:

Before:

image

After:

image

Working with IntelliSense in Custom API Scripts

The emphasis of this first JavaScript IntelliSense definition file was to provide support for Table operation scripts.  However, with a little help, Custom API scripts can also join in the fun.

Custom API scripts receive two parameters – request and response.  However, it is important to note that the request object used in Custom API scripts is not the same request object that is used in the table scripts…in fact, this object provides access to several objects that are directly used as parameters in the Table operation scripts.  This distinction is covered in the MSDN documentation for the request object.

Because I now maintain and reference a copy of the mobileservices.intellisense.js file within my source-control-managed “shared” folder, I can augment and/or make changes to the file with some confidence…when an update to the IntelliSense file is published, I can simply drop the file into place, and before committing and synchronizing the changes I can use a differencing tool to see what has changed.  Ideally, I will notice if the new file is about to lay-waste to the changes I’ve put in…so let’s change the definition file.

I’ve basically added some content to the definition file to provide information about the request object for Custom APIs.  The IntelliSense content I’ have added comes from a combination of the MSDN documentation and the Express.js library documentation, which is where these request and response objects originated.  My content is not comprehensive – I’ve only documented the items that were immediately relevant to me, but there’s nothing to stop others from adding more content – the Express.js documentation for the request object can be found here and the documentation for the response object can be found here.

The content I added to the mobileservices.intellisense.js file is:

/* Custom API Augmentations */ service = { ///<field name="push" type="push">Returns an object used to access the the various push notification proxies.</field> push: push, ///<field name="tables" type="tables">Returns an object used to access the the tables in this instance.</field> tables: tables, ///<field name="mssql" type="mssql">Returns an object used to access the mssql instance.</field> mssql: mssql, } ApiRequest = function ApiRequest() { ///<summary>Faux type used only for providing IntelliSense in Custom API scripts. DO NOT CREATE INSTANCES.</summary> } ApiRequest.prototype = { ///<field name="headers" type="Object">Returns a collection of all the message headers, as a JSON object. Individual headers are obtained by calling the header function.</field> headers: headers, ///<field name="query" type="Object">Used to access the parsed query-string.</field> query: query, ///<field name="service" type="service">Provides access to mobile service-specific resources.</field> service: service, ///<field name="user" type="User">Returns the user object which contains information about the client sending the request.</field> user: user, header: function (headerValue) { /// <summary>Returns a JSON representation of the named header-value from the HTTP request headers sent in a custom API request.</summary> /// <param name="headerValue" type="string">The name of the header to locate.</param> } } ApiResponse = function ApiResponse() { ///<summary>Faux type used only for providing IntelliSense in Custom API scripts. DO NOT CREATE INSTANCES.</summary> } ApiResponse.prototype = { send: function (status, body) { ///<signature> ///<summary>Sends a response.</summary> ///<param name="body" type="Buffer">When a Buffer is given the Content-Type is set to 'application/octet-stream' unless previously defined with the set command.</param> ///</signature> ///<signature> ///<summary>Sends a response.</summary> ///<param name="body" type="String">When a String is given the Content-Type is set defaulted to 'text/html'.</param> ///</signature> ///<signature> ///<summary>Sends a response.</summary> ///<param name="body" type="Array">When an Array or Object is given the response will include the JSON representation.</param> ///</signature> ///<signature> ///<summary>Sends a response.</summary> ///<param name="body" type="Object">When an Array or Object is given the response will include the JSON representation.</param> ///</signature> ///<signature> ///<summary>Sends a response.</summary> ///<param name="status" type="Number">The HTTP response code. When a Number is given without any of the previously mentioned bodies, then a response body string is assigned for you. For example 200 will respond will the text 'OK', and 404 'Not Found' and so on.</param> ///</signature> ///<signature> ///<summary>Sends a response.</summary> ///<param name="status" type="Number">The HTTP response code.</param> ///<param name="body">A body element as defined in the other signatures.</param> ///</signature> }, set: function (field, value) { ///<signature> ///<summary>Set header field to value, or pass an object to set multiple fields at once.</summary> ///<param name="field" type="String">The header field name.</param> ///<param name="value" type="String">The value to set for the header field.</param> ///</signature> ///<signature> ///<summary>Set header field to value, or pass an object to set multiple fields at once.</summary> ///<param name="values" type="Object">A JSON Object consisting of header field/value pairs.</param> ///</signature> } }

With that in place, it is still necessary to include some “helper” XML comments in order for Visual Studio to “connect the dots” between the Custom API function signature and the new IntelliSense:

exports.post = function (request, response) { /// <param name="request" type="ApiRequest"></param> /// <param name="response" type="ApiResponse"></param>

And with that, we now have IntelliSense in our Custom API script:

image

Wrapping Up

I have it on good authority that the Mobile Services team is continuing to work on ways to enhance the script editing experience, so it is likely that some of these steps will only be temporary.  But in the interim, the inclusion of these IntelliSense aids should make editing WAMS script files in Visual Studio considerably simpler.

Managing Windows Azure Mobile Services (WAMS) Server Scripts with Microsoft Visual Studio Tools for Git

Windows Azure Mobile Services (WAMS) has come a long way in a few short months.  At the 2013 Build conference in San Francisco, Microsoft announced more than the service’s General Availability date – it also showed that the service had been integrated into the Visual Studio 2013 Preview in several key places.  These included a “Connected Services Manager” that will augment an existing project to connect with a new or preexisting WAMS instance, an entry for WAMS in the Server Explorer panel that allows viewing and editing the server-side data table business logic scripts (with access to the other WAMS script elements expected to be added over time), and a Push Notification Wizard that facilitates connecting an existing Windows Store app to a WAMS instance.  These features are detailed in this post on the Visual Studio Blog.  The expectation is that these integration points are expected to be enhanced as Visual Studio 2013 is updated to its RTM version.

One of the more recent features added to WAMS itself has been the integration of Git-based source control for the server scripts (as of this writing, this feature is currently in Preview.)  This integration provides several advantages for managing a WAMS service’s scripts.  First, it allows for scripts to be edited locally /offline using desktop file editing tools, including Visual Studio.  Changes can be managed through the local Git repository until they are ready to be synchronized to the server.  What’s more, this arrangement also allows access to some of the WAMS script-related content that is not (presently) available through the WAMS management portal (though access to these files can also be obtained from the command-line management tools.)

To mostly round out the collection of the tools will be discussed here, the Team Foundation Server Power Tools team at Microsoft has released the Visual Studio Tools for Git, which is an extension that allows Visual Studio’s Team Explorer to integrate with Git.  This integration includes using the Team Explorer GUI to manage and connect to local repositories, as well as to track changed elements in a selected repo, and managing branches, commits, and syncs in that repo.  As of this writing, the extension is at version 0.9.5.

This post will explore how the new integrated Visual Studio tools for Git can be used to provide GUI-based source-controlled management for client-side editing of the scripts belonging to a Windows Azure Mobile Services instance.

Enabling Source Control in a WAMS Instance

The first step in the process is to enable Git Source Control for the account which owns the WAMS instance, if it has not already been set up for that account.  This process is documented here.  Basically, the steps entail:

  • – If this is the first time a Mobile Service Git repository is being set up for the current Azure subscription:
    • – From the WAMS Management Portal “Dashboard” page for the desired service, select “Set up source control”
    • – Provide a username and password to be used for access to mobile service Git repositories within the subscription.  These credentials can be changed later by selecting the “Reset your source control credentials” link on a WAMS Dashboard page for any Mobile Service attached to the same subscription.
  • – Copy the Git URL from the service instance’s Configure tab.

image

Connect to the Git Repository from Visual Studio

The next step is to connect to the server-side repository that was just created and clone it to the local machine.  This (and many of the remaining steps) can be accomplished via several Git tools, including the command line, but the focus here is on using Visual Studio (for the most part, I prefer to leave the command line to build scripts and for hacking into the Matrix.)  Once the Visual Studio Tools for Git extension has been downloaded and installed, open Visual Studio 2013 and select the Team Explorer pane (if necessary, it can be found in the View menu.) 

Select Connect (the icon looks like a the male end of a 2-prong extension cord)

image

In the ensuing Connect dialog, select Clone to do the initial setup of the local repo as a clone of the server repository.  This will show a couple of text boxes – the top one is used to provide the URL to the remote repository – paste the URL that was copied earlier from the WAMS Management Portal.  In the second one, type in or browse to the local directory path into which the repository should be cloned.

image

When you press the Clone button, you will be prompted for your server-side Git repository credentials.

image

…and after entering these credentials, you will have a local clone of the WAMS repository.

Examining the WAMS File Structure and Script Files

If you right-click on the new repository entry in the Team Explorer Connection Panel, you will see context menu options which include one for opening the repository in File Explorer.

image

 

The folder you open will contain a file and single subfolder titled “service”.  Within the service folder are the folders that contain the WAMS script content.  These folders and their purpose are listed below:

  • api: contains 2 files per Custom API defined in the WAMS portal.  The first file is a JavaScript (api name.js) file containing the script itself and the second is a JSON metadata file (api name.json) that describes the permission level required for each custom api script action.
  • scheduler: contains a JavaScript file (scheduled job name.js) for each defined scheduler script.
  • shared: contains scripts and/or Node.js modules that can be reused by other scripts in the service instance.  Note that there is no browser in the portal (at present) for files in this folder.
  • tables: contains between 1-5 files per defined table in the WAMS instance.  These files minimally include a JSON metadata file (table name.json)  For any modified table action script (read, insert, update, delete), there is also a corresponding JavaScript file (table name.operation.js)

Note #1: Each folder also includes a text md file that describes the intended contents of the folder and any layout constraints for each file. 

Note #2: For the most part, with the exception of the “shared” folder, the files should be “seeded” by creating the desired named item server-side and then syncing the repository.  WAMS often has to spin up other infrastructure elements in addition to just the scripts, such as database tables and entries for the scheduler. 

For Table scripts, the table can be created in the Visual Studio Server Explorer, but the individual table operation script files won’t be brought down by a Sync operation until the corresponding scripts is modified in some way – the change can be as simple as adding a comment.  Once the file has been altered and saved to the server, using Sync will fetch the file to the local repository.  The WAMS instance context menu (with the Create Table option) in the Server Explorer is shown below, along with a script that has been modified with nothing but a comment.

image              image

The screenshot below shows after the file above has been saved to the server from within the Server explorer, after doing a Sync of the Git repositories, the local folder has been updated with the table’s corresponding read script file.

image

Note that generating files this way is probably best left only for initially “seeding” the files, thus leaving day-to-day editing to an operation that is covered by source control and includes change tracking, etc.

Working with the Visual Studio Git Tools

From the Connections panel, opening one of the repositories (double-click or right-click and select Open) displays the Home panel for the selected repository.

image

Once the necessary files have been synced to the local repository, the files within the folders in the repository can be edited on the local machine and the changes will be tracked by Git.  These changes can be seen in Visual Studio by bringing up the Changes Panel.  In the illustration below, the “read” script file for the “blogdemo” table has been modified locally (in fact, it was opened in Visual Studio and edited there to take advantage of Visual Studio’s JavaScript editing tools.  Any text editor could have been used, however.)

image

Note that there’s also an entry for “demosharedcontent.js” in the Untracked Files listing.  To include the file, simply drag it (or its containing folder) up and drop it in the “Included Changes” section:

image

Once the collection of changed files that is to be committed to the local repository is ready, a commit message (comment) can be entered and the Commit button can be pressed.  Pressing the Commits link from this panel, returning to the home panel and selecting the Commits button, or selecting the Commits pulldown at the top of the TFS explorer will show the current set of outgoing and incoming Commits available for Syncing with the Server.

image

Pressing the Sync button once again synchronizes the script changes with the WAMS service.  Examining the table script in the WAMS Management Portal shows that the change in fact has been uploaded to the server:

image

If the Sync results in conflicts that Git cannot resolve, (the same word was changed both in the client and the server, for example) the Sync will indicate so, and a list of conflicts will be displayed:

image

The conflicts can be examined in detail and adjusted using the 3-way merge tool that comes with Visual Studio (with apologies for my typos):

image

Once the Merge is committed, it can be synced to the server.

The final working panel that hasn’t been shown yet is the Branches panel, which allows management of branches in the selected local repository.

image

Wrapping Up

While Windows Azure Mobile Services currently benefits from several points of integration into the Visual Studio 2013 IDE, the addition of the Visual Studio Tools for Git offers an additional tool to help with the management of server-side scripts.  For what it’s worth, the Visual Studio Tools for Git also works with Visual Studio 2012, and supports the same kinds of source control management as was shown here in Visual Studio 2013 Preview, except that Visual Studio 2012 lacks the additional WAMS integration points such as the entry in the Server Explorer.  Nonetheless, look for the integration of these tools to become even richer as WAMS, Visual Studio 2013 , and even the Visual Studio Tools for Git are enhanced over the next few months.