MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

Update project templates to the latest NuGet packages

We noticed a question on StackOverflow that proved we weren't the only ones finding it a little sub-optimal having to update NuGet packages right after creating a new project.

Most of us are likely to use the default project templates that come with Visual Studio or an SDK. Let's take the example of the MVC4 project template for C#, using Razor syntax.

MVC4 C# Web application template using Razor syntax

 

This project template is consuming quite a few NuGet packages by default. jQuery is one of them. The whole point is that these NuGet packages can be updated more frequently and independent from any pending SDK update or other product release. This is a good thing!

As a direct consequence, this also means that the default templates become "outdated". Outdated is a strong word, as the template itself isn't really outdated, but rather the packages list it wants to consume from NuGet. jQuery is one of those packages that gets very frequent updates. There's an easy way to update all packages in a solution all at once. Use the Package Manager Console, type

Update-Package

and hit ENTER. Done!

But why not avoid this step (or at least partially) and change the defaults?

Note: We'd recommend you to create your own project template so you can always revert back to the default one in case you, or someone else, is going to mess things up :)

All Visual Studio (2012) project templates can be found here:

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\ProjectTemplates\

If you want to create some custom project templates, I'd recommend you to create them here:

%USERPROFILE%\Documents\Visual Studio 2012\Templates\ProjectTemplates

In this post, we'll show you how you can change the defaults for the MVC4 CSHTML project template. You can find it here:

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\ProjectTemplates\CSharp\Web\1033\MvcWebApplicationProjectTemplatev4.0.cshtml\

To modify the files, you'll have to edit them as Administrator (you know the drill, right-click Notepad++ or Sumblime and click Run As Administrator).

The file you'll want to take a look at is the .vstemplate file. It's an XML file containing template instructions for Visual Studio. Look for a section called packages. It should look something like this:

<WizardData> 
<packages repository="registry" keyName="AspNetMvc4VS11" isPreunzipped="true"> 
<package id="EntityFramework" version="5.0.0" skipAssemblyReferences="true" /> 
<package id="jQuery" version="1.7.1.1" />
...

Let's take jQuery as an example again: we want to upgrade the dependency to version 1.8.2 by default.

To do so, you modify the above snippet to look like this:

<WizardData>
<packages repository="registry" keyName="AspNetMvc4VS11" isPreunzipped="true">
<package id="EntityFramework" version="5.0.0" skipAssemblyReferences="true" />
<package id="jQuery" version="1.8.2" />
...

Easy huh?

Now you found the candy, you can change the default installed package versions, or even add or remove the packages you want. Whatever you do, make sure you don't break the template so proceed with caution. If you remove a package dependency, make sure you remove any dependent configuration or references in the project template's files. If you update a package to a newer version, make sure those dependent configurations are updated as well.

Take a look at the project template's Scripts folder. You see that little _references.js file?

This is a harmless example of things that can be left behind and out-of-sync with the package edits you make. Open the file (run as administrator) and update those references accordingly. The jQuery reference should now be the following:

/// <reference path="jquery-1.8.2.js" />

Ever wondered why you didn't have to be online to be able to create a new MVC project and consume all those packages? Then check this folder and be amazed:

C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET MVC 4\Packages

Obviously, the packages you want to support in your default project templates should be available there as well, so download those NuGet packages and extract them here. You can download the NuGet package after logging in to NuGet.org: look for the package you want, select the version you need, and you'll notice a download link on the left side.

Download a NuGet packages from the Gallery

Once downloaded, unblock the package (right-click, properties, unblock), copy it to the packages directory (C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET MVC 4\Packages). Next, unzip it, and remove all garbage. The relevant content is selected on the following screenshot. Ensure you rename the .nuspec file by adding the version in front of it, e.g. jquery.1.8.2.nuspec.

Relevant package contents

From now on, all newly created default MVC4 CSHTML Web projects will already contain the updated jQuery dependency.

How we push GoogleAnalyticsTracker to NuGet

If you check Maarten’s blog post Tracking API usage with Google Analytics, you’ll see that a small open-source component evolved from MyGet. This component, GoogleAnalyticsTracker, lives on GitHub and NuGet and has since evolved into something that supports Windows Phone and Windows RT as well. Here’s his guest post:


It’s funny how things evolve. GoogleAnalyticsTracker started as a small component inside MyGet, and since a couple of weeks it uses MyGet to publish itself to NuGet. Say what? In this blog post, I’ll elaborate a bit on the development tools used on this tiny component.

Source code

Source code for GoogleAnalyticsTracker can be found on GitHub. This is the main entry point to all activity around this “project”. If you have a nice addition, feel free to fork it and send me a pull request.

Staging NuGet packages

Whenever I update the source code, I want to automatically build it and publish NuGet packages for it. Not directly to NuGet: I want to keep the regular version, the WinRT and WP version more or less in sync regarding version numbers. Also, I sometimes miss something which I fix again 5 minutes after. In the meanwhile, I like to have the generated package on some sort of “staging” feed, at MyGet. It’s even public, check http://www.myget.org/F/githubmaarten if you want to use my development artifacts.

When I decide it’s time for these packages to move to the “official NuGet package repository” at NuGet.org, I simply click the “push” button in my MyGet feed. Yes, that’s a manual step but I wanted to have some “gate” in the middle where I should explicitly do something. Here’s what happens after clicking “push”:

Push to NuGet

That’s right! You can use MyGet as a staging feed and from there push your packages onwards to any other feed out there. MyGet takes care of the uploading.

Building the package

There’s one thing which I forgot… How do I build these packages? Well… I don’t. I let MyGet Build Services.do the heavy lifting. On your feed, you can simply click the “Add GitHub project” button and a list of all your GitHub repos will be shown:

Build GitHub project

Tick a box and you’re ready to roll. And if you look carefully, you’ll see a “Build hook URL” being shown:

MyGet build hook

Back on GitHub, there’s this concept of “service hooks”, basically small utilities that you can fire whenever a new commit occurs on your repository. Wouldn’t it be awesome to trigger package creation on MyGet whenever I check in code to GitHub? Guess what…

GitHub build hook

That’s right! And MyGet even runs unit tests. Some sort of a continuous integration where I have the choice to promote packages to NuGet whenever I think they are stable.

MyGet Build Services - Public Beta

We’re happy to announce that MyGet Build Services is in Public Beta since, well, now. MyGet Build Services enable you to add packages to your feed by just giving us your Git, Mercurial or Subversion repo. We build it, we package it, we publish it. And starting today, every MyGet user can use it.

More information on MyGet Build Services can be found in a previous blog post, although some things have been updated. You’ve provided us feedback through our UserVoice, resulting in the following changes:

  • We now support .NET Framework 4.5. New features available in .NET Framework 4.5 are described at http://msdn.microsoft.com/en-us/library/vstudio/ms171868.aspx.
  • Build hooks! The list of your build definitions now shows a build hook URL which you can link to your GitHub or BitBucket repository. Calling this build hook (HTTP POST) will trigger a new build. Or in other words: a commit will trigger a build.
  • Our build server now checks for a file called build.bat first, then MyGet.sln, then your .sln and then .csproj/.vbproj files (UV)
  • We now support Git, Mercurial and Subversion repositories (UV)
  • You’ll now have some tools available in the environment variables (UV)
  • There’s more information in the build logs (UV)

Keep the feedback coming! Build Services are now enabled on every MyGet account. Do note MyGet Build Services are still a beta, and as Willy Wonka would say… “Little surprises around every corner, but nothing dangerous.”

Happy packaging!

PS: Why not deploy your freshly built apps using Octopus Deploy?

MyGet Build Services - Join the private beta!

Good news! Over the past 4 weeks we’ve been sending out tweets about our secret project MyGet project “wonka”. Today is the day Wonka shows his great stuff to the world… In short: MyGet Build Services enable you to add packages to your feed by just giving us your GitHub repo. We build it, we package it, we publish it.

Our build server searches for a file called MyGet.sln and builds that. No problem if it's not there: we'll try and build other projects then. We'll run unit tests (NUnit, XUnit, MSTest and some more) and fail when those fail. We'll search for packages generated by your solution and if none are generated, we take a wild guess and create them for you.

To make it more visual, here are some screenshots. First, you have to add a build source, for example a GitHub repository (in fact, GitHub is all we currently support):

MyGet Add build source

After that, you simply click “Build”. A couple of seconds or minutes later, your fresh package is available on your feed:

MyGet build package

MyGet package result

If you want to see what happened, the build log is available for review as well:

MyGet build log

Enroll now!

Starting today, you can enroll for our private beta. You’ll get on a waiting list and as we improve build capacity, you will be granted access to the beta. If you’re in, tell us how it behaves. What works, what doesn’t, what would you like to see improved. Enroll for this private beta now via http://www.myget.org/buildservices. Limited seats!

Do note it’s still a beta, and as Willy Wonka would say… “Little surprises around every corner, but nothing dangerous.”

Happy packaging!

PS: Why not deploy your freshly built apps using Octopus Deploy?

Feature highlight: MyGet Gallery

It's been less than two months since we introduced the public MyGet Gallery, and we are very happy to have some awesome projects onboard, often shipping development builds or using MyGet as a staging environment before pushing packages upstream. These feeds deserve more attention and people should know these feeds exist.

Did you know you can get the latest development builds of SignalR and MVC Contrib delivered through MyGet?

If you think your project's development builds deserve a place in our Gallery, you can easily opt-in using your feed's Gallery Settings.

Gallery feed readme

Our latest v1.3 release now allows feed owners to add a feed readme for publication in the MyGet Gallery. This way, feed owners now can easily add some instructions or disclaimers to the gallery feed details page using the well-known markdown format. Don't forget to add a link to your project page and/or sources as well.

You might recognize the pagedown markdown editor there, which is also used on the StackExchange Web sites, so you are probably already familiar with it. In return for this excellent control, we also added the StackOverflow identity provider to our login page. People who prefer to login using their StackOverflow account can now do so, or you can link it to your other identities within your MyGet account as well.

Layout improvements

While at it and based on the invaluable feedback of our users, we also improved the look-n-feel of the packages and feed listings by removing clutter and putting focus on content.

You can now not only see which packages are listed, and what version they're at, but you can also track when this version has become available. A simple click on the icon next to the package will trigger a package download in your browser, whilst the icons on the top right bring you to the SymbolSource symbols repository of this feed (if available) or give access to the RSS feed associated with this package repository.

What's next?

The screenshot below (taken from our latest development builds) is giving you a glimpse at what's coming next: user profiles and activity streams! That's right: you'll be able to get insights into the activity of a public feed or track the history of a given package. Coming soon...

We hope you like it! Keep those suggestions and feedback coming! Who knows, maybe one day you'll see your wish granted :)

Happy packaging!

MyGet Update: version 1.3.0

We are happy to announce a new version of the MyGet Web site containing a set of improvements, fixes and updates.

The following work items have been addressed in this release:

Product

  • number of plans has been reduced. We now have Free, Starter (formerly Small plan), Professional (formerly Large plan) and a new Enterprise plan
  • introduction of the new Enterprise plan focused on companies that require more performance, integration and security

Usability

  • improved layout and readability on smaller screen resolutions
  • auto-lower cased routing for feed URL (helps avoiding issues caused by incorrect casing of feed identifier)
Performance
  • upgraded to MVC4
Features
  • new identity provider: StackExchange
  • upgraded SymbolSource integration to take benefit from their improved and freshly released API
  • Gallery feeds now support a README in Markdown format, available in the feed's Gallery Settings
  • Enterprise plan: upgraded administrative & quota management dashboard
More details about the new features will be covered in follow-up blog posts soon.
 
Happy packaging!

Site issues on July 2nd, 2012

What a day! July 2nd seemed to go from sunshine to rain in a couple of hours. The good news is: we’re back. Let’s first go through what happened…

What happened?

imageThis morning, you've probably received an e-mail stating that "Your Free subscription at MyGet has expired". Free? Expired? Even we were puzzled! We’ve quickly sent out an e-mail stating free subscriptions don’t expire and that we were investigating the issue. In our rush to find out what happened, we even forgot to remove some template text from that e-mail. Professional of us, really.

A couple of hours later, the website started acting funny: one moment it was up, one moment it appeared down. The strange thing was: we did not see this happening in our server monitoring page. The brand new Windows Azure portal was all shiny and bright, all systems running. A short glitch? Probably.

And then 4:00 PM (GMT+1) came around: we started to receive SMS messages from Pingdom. UP, DOWN, UP, DOWN. Some sweaty hours later, we’ve found the root cause of all this mess and shortly thereafter, deployed a hotfix to the cloud. What a day!

All-in-all, we’ve spammed you in the morning, had some small glitches during the day and went bezerk between 4:00 PM (GMT+1) and 7:00 PM (GMT+1).

What was wrong?

During the entire day, we were suspecting two of our latest changes. First of all, we’ve moved storage accounts in Windows Azure to make use of some of the new features in there with regards to mirroring packages. The code isn’t done yet, but we did already switch the storage accounts. Next to that, we’ve recently deployed our site using ASP.NET MVC 4 (RC). Was any of this the cause? We’ve been searching… But no.

Elmah revealed some interesting details. We were getting throttled on those new storage accounts. Was this due to the fact that we enabled storage analytics? We’ve disabled them and found that we were back up in full glory! For a moment, because this wasn’t the reason we were being throttled…

We process several operations asynchronously, such as updating our caches. It seemed that there were 16.000 commands in there, all telling the system to update the cache for one specific feed. One of our newest MyGet members managed to break a couple of gears there! Something we don’t judge. In contrary: there’s no better stress test or user test than real users.

A rookie mistake

Now why were there 16.000 messages unprocessed and why were some of them on the dead-letter queue? And why were our machines looking fine (all green!) while the website did not respond? The answer lies in the following piece of code:

1 [DebuggerDisplay("{Title} Version={Version}")] 2 public class FeedPackage 3 : TableServiceEntity 4 { 5 public string Id { get; set; } 6 public string Version { get; set; } 7 8 // ... 9 10 public override bool Equals(object obj) 11 { 12 if (ReferenceEquals(null, obj)) return false; 13 if (ReferenceEquals(this, obj)) return true; 14 if (obj.GetType() != typeof(FeedPackage)) return false; 15 return GetHashCode() == obj.GetHashCode(); 16 } 17 18 public override int GetHashCode() 19 { 20 int hashCode = 7; 21 22 if (PartitionKey != null) hashCode ^= PartitionKey.GetHashCode(); 23 if (RowKey != null) hashCode ^= RowKey.GetHashCode(); 24 25 return hashCode; 26 } 27 }

Did you spot it? We’ve gone with a complete rookie implementation of Equals and GetHashCode there…

The devil is in the fact that we’re using the hashcode of package A and compare it with package B. That’s a serious problem, as those hash codes are based on the hashcodes of their package identifier and package version. Guess what MSDN tells us

If two string objects are equal, the GetHashCode method returns identical values. However, there is not a unique hash code value for each unique string value. Different strings can return the same hash code.

Our newest member was in the position where he had a lot of packages uploaded to MyGet and by coincidence, two packages appeared to be “equal” using our faulty code above.

Now why did the website choke on that?

Windows Azure table storage, which we use to store all your data, In the .NET space, the WIndows Azure storage client uses WCF Data Service to fetch data. Deep in that framework, the Equals'() method is called on every entity to check whether a specific entity is already being tracked by the data context in use. And since we had a duplicate result for Equals(), an unhandled exception was thrown there.

Wait a minute: you guys don’t catch unhandled exceptions? Yes we do. And when such exception occurs, we retry some operations for up to five times. Retry operations? Well, rather than failing immediately, a good pattern is to retry a failing operation to check that the error wasn’t just a minor glitch like a network fault. On Windows Azure, it is recommended to do this for calls to any external system, like a webserver calling storage. Check David Aiken’s blog for some info on this.

So, retries… Yes. Retry an I/O operation with minor latency for five times. Have a user who’s experiencing an issue with the website F5 a couple of times. And have that queue with 16.000 operations pending hammer this storage account again. What will happen? Throttling. Windows Azure tells our system to retry again after a second. And our retry logic does exactly that. This retry logic causes some threads to sleep, all the queue processing and F5-ing causes some more thread to sleep and what happens? The server comes to a halt while it still looks okay to our external monitoring.

Are we running just one server? No. We’re running multiple, in a round-robin load-balanced farm. This means that all this refreshing impacted all of our servers, making the site as slow as a snail. And recover again. And go almost down again…. and come up again. Exactly what we’ve been seeing today.

What are we doing to prevent this from happening in the future?

The truths of offering a cloud service are: hardware fails, software has bugs and people make mistakes. Hardware failures are mitigated by the Windows Azure system. Software bugs? It seems we’re proficient at that. And yes, that was a giant mistake of us. Our job is to mitigate all of these and provide a reliable, robust service to you. We’ll be using the lessons learned today to improve our service, your service.

First of all, we’ve fixed our rookie mistake. This should have never happened and we’ll make sure that our code base is checked and repaired for faulty Equals / GetHashCode implementations.

Next, we’ll be reviewing our retry logic. Blindly retrying on any exception type isn’t the smartest thing to do, so it seems. We’ll be making sure the retry logic only fires on transient exceptions and not blindly on any exception that occurs.

We’ll also be investigating additional monitoring. We’re not sure yet about possible tools or services there, we do want to know when something is wrong and our CPU is heating the entire datacenter at 100%.

We're sorry!

Happy packaging!

The MyGet team

Pushing packages from MyGet to NuGet (or another feed)

Imagine you have a package repository hosted on MyGet. Every time a team member of your open source or enterprise project commits source code changes, your build server pushes an updated release to this package repository in the form of a prerelease NuGet package. Now what happens if a release to the official NuGet package source has to be created? Typically, you will either create a fresh package which will be the package to release, or download a package from your build server, change the version and upload that one to NuGet.org (or another repository). No need for such overloaded process anymore: MyGet will perform the push for you.

Setting up a continuous integration (CI) feed

First of all, you will need a CI feed to which your build server can push every NuGet package related to your project. Simply create your feed on MyGet, a three-second process. After creating your feed, MyGet will present you the feed URL as well as an API key. Optionally, you can make it a private feed and ensure only people who were granted the correct privileges can access your feed.

Next, configure your build server to push the NuGet artifacts to MyGet. Using TeamCity, for example, this can be done by adding a NuGet Push build step:

MyGet feed TeamCity

Pushing packages from MyGet to NuGet (or another feed)

The first time you want to push a package to another NuGet feed, you’ll probably have to configure the other feed’s URL and API key to use when pushing there. The push package feature is based on the package source proxy feature released earlier. Navigate to your feed and on the left hand side, click the “Package Sources” item and either add a new package source or edit the existing, default NuGet package source. In order to be able to push a package to another feed, the API key has to be specified. You can enter your NuGet.org (or another feed) API key and click the Save button.

MyGet package source selection

Pushing packages from MyGet to NuGet or any other feed is easy. From the moment a package source has been configured, using the “Push” button will enable you to push packages to another feed.

image

After clicking the “Push” button, MyGet will present you with an overview of the package which will be pushed to another feed. Select the feed to which you want to push and verify the other fields. When pushing a prerelease package to a stable package, simply make the Prerelease tag field blank. You can also modify the prerelease tag if wanted. If you do intend to push a prerelease version, simply continue clicking the “Push” button.

Push a NuGet package

Told you it was easy. MyGet will now push the package to the selected feed and inform you by e-mail should anything go wrong. Happy packaging!

Pro NuGet is finally there!

Short version: Install-Package ProNuget or http://amzn.to/pronuget

Pro NuGet - Continuous integration Package RestoreIt’s been a while since I wrote my first book. After I’ve been telling that writing a book is horrendous (try writing a chapter per week after your office hours…) and that I would never write on again, my partner-in-crime Xavier Decoster and I had the same idea at the same time: what about a book on NuGet? So here it is: Pro NuGet is fresh off the presses (or on Kindle).

Special thanks go out to Scott Hanselman and Phil Haack for writing our foreword. Also big kudos to all who’ve helped us out now and then and did some small reviews. Yes Rob, Paul, David, Phil, Hadi: that’s you guys.

Why a book on NuGet?

Why not? At the time we decided we would start writing a book (september 2011), NuGet was out there for a while already. Yet, most users then (and still today) were using NuGet only as a means of installing packages, some creating packages. But NuGet is much more! And that’s what we wanted to write about. We did not want to create a reference guide on what NuGet command were available. We wanted to focus on best practices we’ve learned over the past few months using NuGet.

Some scenarios covered in our book:

  • What’s the big picture on package management?
  • Flashback last week: NuGet.org was down. How do you keep your team working if you depend on that external resource?
  • Is it a good idea to auto-update NuGet packages in a continous integration process?
  • Use the PowerShell console in VS2010/11. How do I write my own NuGet PowerShell Cmdlets? What can I do in there?
  • Why would you host your own NuGet repository?
  • Using NuGet for continuous delivery
  • More!

I feel we’ve managed to cover a lot of concepts that go beyond “how to use NuGet vX” and instead have given as much guidance as possible. Questions, suggestions, remarks, … are all welcome. And a click on “Add to cart” is also a good idea ;-)

Introducing MyGet package source proxy (beta)

My blog already has quite the number of blog posts around MyGet, our NuGet-as-a-Service solution which my colleague Xavier and I are running. There are a lot of reasons to host your own personal NuGet feed (such as protecting your intellectual property or only adding approved packages to the feed, but there’s many more as you can <plug>read in our book</plug>). We’ve added support for another scenario: MyGet now supports proxying remote feeds.

Up until now, MyGet required you to upload your own NuGet packages and to include packages from the NuGet feed. The problem with this is that you either required your team to register multiple NuGet feeds in Visual Studio (which still is a good option) or to register just your MyGet feed and add all packages your team is using to it. Which, again, is also a good option.

With our package source proxy in place, we now provide a third option: MyGet can proxy upstream NuGet feeds. Let’s start with a quick diagram and afterwards walk you through a scenario elaborating on this:

MyGet Feed Proxy Aggregate Feed Connector

You are seeing this correctly: you can now register just your MyGet feed in Visual Studio and we’ll add upstream packages to your feed automatically, optionally filtered as well.

Enabling MyGet package source proxy

Enabling the MyGet package source proxy is very straightforward. Navigate to your feed of choice (or create a new one) and click the Package Sources item. This will present you with a screen similar to this:

MyGet hosted package source

From there, you can add external (or MyGet) feeds to your personal feed and add packages directly from them using the Add package dialog. More on that in Xavier’s blog post. What’s more: with the tick of a checkbox, these external feeds can also be aggregated with your feed in Visual Studio’s search results. Here’s the magical add dialog and the proxy checkbox:

Add package source proxy

As you may see, we also offer the option to filter upstream packages. For example, the filter string substringof('wp7', Tags) eq true that we used will filter all upstream packages where the tags contain “wp7”.

What will Visual Studio display us? Well, just the Windows Phone 7 packages from NuGet, served through our single-endpoint MyGet feed.

Conclusion

Instead of working with a number of NuGet feeds, your development team will just work with one feed that is aggregating packages from both MyGet and other package sources out there (NuGet, Orchard Gallery, Chocolatey, …). This centralizes managing external packages and makes it easier for your team members to find the packages they can use in your projects.

Do let us know what you think of this feature! Our UserVoice is there for you, and in fact, that’s where we got the idea for this feature from in the first place. Your voice is heard!