MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

Manage NuGet Package Sources using PowerShell

If you regularly need to manage your NuGet package sources in Visual Studio, and got familiar to using the NuGet Package Manager Console for all kinds of operations and automation, you might find the following approach appealing.

This approach will save you a few clicks and is based on a very cool tools package called ManagePackageSource. This NuGet package installs a few extra PowerShell commands into your NuGet PowerShell profile. That’s right, NuGet supports its own PowerShell profile! So instead of installing these tools packages over and over again every single time you create a new solution, let’s simply install these commands once and for all, so they are always available.

To do so, simply run the following command:

Install-Package ManagePackageSource

You should see the following output in the NuGet Package Manager Console

2012-05-30_2306

 

After that, you can simply uninstall the package again from your project to undo the changes in your working directory. The commands have been installed into your NuGet PowerShell profile and the profile has been reloaded, so they will remain there even without ever needing to install this package again.

If you happened to have another instance of Visual Studio open before installing this package, you can reload the profile as well in this other instance by running the following command in the NuGet Package Manager Console:

. $profile

In case you’re wondering where we changed stuff on your computer, you can find the NuGet_powershell.ps1 profile in the C:\Users\username\My Documents\WindowsPowerShell\ directory, and the new ManagePackageSource module can be found under the Modules directory inside that one.

To register a new NuGet feed into your Visual Studio, for instance a MyGet feed, you can run the following command now:

Add-PackageSource "My MyGet feed" "http://www.myget.org/F/myfeedname/"

If you're interested into the sources, you can obviously unzip the NuGet package or browse the code on our GitHub repository. Feel free to contribute your own extensions and improvements!

NuGet $version$ token explained

Many questions that often come to mind when building NuGet packages are related to versioning. There's one question in particular I'd like to post here because it's one of the easier to answer. The question is: How do I use the $version$ token in the NuGet manifest (nuspec) file? Where does it get the version number from?

If you look at the NuGet docs explaining the nuspec replacement tokens, it states that the following - if it doesn't, my pull request got accepted :) - The assembly version as specified by the assembly's AssemblyVersionAttribute.

That is not entirely accurate (got a pull request accepted on the docs, so this should be fixed soonish). Consider the following AssemblyInfo.cs file contents for example.

[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]
[assembly: AssemblyInformationalVersion("1.0.2.0")]

This is where most people get confused. I won’t get into the details of which version attribute you should or should not use, as there can be good reasons to use either one of them in different scenarios. I’ll focus on how nuget is using this information to provide a version number to the $version$ replacement token in the nuspec file.

Building a NuGet package using a tokenized nuspec file that relies on assembly information can be achieved in various ways, for instance:

  1. nuget spec <csproj> to generate the tokenized nuspec, followed by nuget pack <csproj>
  2. nuget spec –a <assemblyPath> inside the csproj folder to generate a non-tokenized nuspec (so with the metadata already filled in), followed by nuget pack <nuspec>

Basically, it comes down to this:

  • If the AssemblyInformationalVersion attribute is available, then that one is used.
  • If the AssemblyInformationalVersion attribute is not available, then the AssemblyVersion attribute is used.
  • If none of the above are specified, your assembly will have a version number of 0.0.0.0, as well as the resulting package.
  • NuGet totally ignores the AssemblyFileVersion attribute.

Note that this behavior is the same when skipping the nuspec at all and building a nuget package directly from an assembly, using using nuget pack <assembly.dll>.

How to help yourself when NuGet goes down

Today will be remembered as the day that NuGet.org went down and broke quite some builds. While many people would love to see the NuGet team wearing a pink sombrero, there is something to say about wearing it yourself if you did not manage to workaround this. Let me explain…

First of all, just as with the Azure downtime on a leap day, whenever you rely on an external system and make it mission critical, you should design for failure. You need to anticipate downtime. I’m sure the NuGet team does everything within its power to fix this and is going to inform us whenever they can, but give them some credit please: we’re all human beings making mistakes, that’s how we learn.

pinksombrero

How do I know it’s not just me?

Here’s what you can do

The best thing one can and should do, again, is to anticipate. Have a backup repository, or mirror packages on a MyGet.org feed. Looking at my twitter streams, it is striking to see how many did not think about it.

If you have your packages in source control, impact is limited to not being able to upgrade your packages or install new packages. If you don’t have your packages in your VCS, and you did not anticipate, you might get lucky enough and fix your builds by targeting the local NuGet cache.

Simply register the following path as a NuGet package source and target this one:

%Localappdata%\NuGet\Cache

Avoid spending your day fingers crossed!

MyGet tops Vanilla NuGet feeds with a Chocolatey flavor

We recently deployed an all new version of MyGet.org, which contains quite a lot of optimizations and some new features as well. If you didn’t notice, go check it out!

My personal favorite is in fact the underlying architecture that allows us to aggregate feeds and link package sources. These package source presets are configurable on Feed level through the new Package Sources tab available in the Feed management interface.

managePackageSources

To add a package from these package source onto your own feed (either referenced or mirrored, with or without its dependencies), navigate to the Add Package dialog and select the From Feed tab (previously called ‘From NuGet.org’).

addChocolateyPackage

The default setting is still NuGet.org obviously, but you might notice the dropdown containing another feed: Chocolatey.org! That’s right, why not add Chocolatey to your package sources and build a feed containing your favorite tools?

Build your own favorite Chocolatey tools feed

Building such feed is very straightforward. Choose a feed name (which will be represented in your URL) and provide a meaningful description.

myChocolateyFeedDetails

All that is left for a basic MyGet feed is to choose one of the predefined feed templates, as shown below. Obviously you can modify and tweak these settings further to meet your needs afterwards. We’ve updated our FAQ with a full explanation of MyGet’s security model and how you can assign or revoke user rights on a feed.

myChocolateyFeedTemplate

Once the feed is created, you can start pushing packages to it. We support various ways of doing so, including:

  • Uploading packages through the website (and optionally mirror any dependencies found on the configured package source)
  • Referencing or mirroring packages from another feed (any package source, such as NuGet.org, Chocolatey.org, …)
  • Uploading a packages.config file, targeting any package source

Let’s add some packages from the Chocolatey Gallery shall we? Simply select the Chocolatey Gallery package source and type any package name (or any other search criteria using our other search options). Autocomplete will kick in after you typed a character or two and show you a list of possible matches. Select the one you want, verify whether you want to only reference it (copy the metadata onto your feed and keep the real package in the package source) or mirror it (deep copy). If you perform a deep copy, you might prefer to opt-in and check the include dependencies checkbox.

addChocolateyPackageGitExtensions

After clicking the upload button, the operation is queued into the background for processing. It might take up to a minute until the package (and its dependencies if requested) appear on your feed. Note that we also cache the packages list on the website, but nevertheless, once processed they will appear instantly onto your feed when queried from within Visual Studio for instance.

For this demo, I only added GitExtensions to the feed but I mirrored it and included dependencies. After processing, my feed packages list contains the following packages.

myFavoriteToolsPackagesList

Now it’s a piece of cake to add those other tools I’m using. You might be wondering…

Why do you want to do that?

Very simple: I don’t want to mess around with a script or packages.config file on multiple computers. If I repaved my system, that file is not on there. If I want to put it on that repaved machine, I need to find it back (that’s usually the real issue). And if I somehow manage to do so, it’s usually out-of-date.

What if there was a central (read: cloudy) location where I could keep track of this list? A single place to manage the tools list, and always the same location to refer to. Something that would reduce the installation of all my favorite tools to a one-liner. When drafting the Chocolatey chapter in our Pro NuGet book, I learned about the convenient –all command line option support by many of Chocolatey’s commands, including the update (cup) and list (clist) commands . That’s when it struck me that this switch was missing for the install (cinst) command, so I bothered Rob Reynolds (@ferventcoder) with it :) Rob was so nice to help us out and made sure the cinst –all –source [feedUrl] scenario would be supported in the near future. Guess what, it is! Thanks again Rob!

Knowing this, it’s pretty straightforward to repave a system and get all your favorite tools installed in no time. It suffices to run the following command (replace my feed with your feed):

cinst -all -source http://www.myget.org/F/myfavoritetools

Not only can you use it after repaving your system, you could use it as well every time you work for a new customer and need to set up your development environment (maybe have multiple feeds for various scenarios?), or why not share the feed with your team members and make sure everyone benefits from these awesome tools out there.

If ever you have a question about MyGet or need further assistance to get you started, please refer to our blog, reach out on twitter (@MyGetTeam) or use the Support form to contact us. We’ll be happy to help!

Continuous Package Integration: NuGet vs Source Control

Update (August 17, 2011):

David Fowler created an awesome NuGetPowerTools package that streamlines this process further. Also check out David Ebbo's post for more info.

One of the questions I receive regularly when talking about enterprise approaches for using NuGet, is the following one:

"Why don't you put the NuGet packages in source control as well?"

In my opinion a very valid question to ask, and I reached the point where I realized it's better to write a blogpost on it once and use it for future reference :-)

Please note: what I will discuss here is my take on it, and not the holy grail!

Storing NuGet packages in source control?

As David Ebbo already explained, there are approaches to not store the NuGet packages in source control. Your question is: why?

One could say that disk space is cheap! That's a valid approach if you have no issues repeating your dependencies in source control every single time. I'd like to see however how that works out for you if you have let's say 500 line of business apps in your SCM, all having the same dependency to your favorite logging library of around 0.5Mb. That's 250Mb worth of storage! This also slows down the time needed to perform a backup, or the disk space required to store your backups, thus increasing your costs in storage and power consumption. Now extrapolate this reasoning to all your dependencies..

Add on top of this that some SCM's don't manage binary files that well (TFS anyone?): ask your devs how much time they lost already fighting with a system that is unable to do binary diffing? You ever experienced upgrading an assembly in TFS from version X to version Y? Forgot to explicitly check out the file, replaced it on disk, to find out the system tells you it found no changes to commit? Don't get me wrong, TFS is a great tool, but its source control system could be better.

For some, this is a reason not to use TFS, but I'd say that any Source Control system is meant to store sources, not binaries. Use a document management system if you need document versioning and take benefit from its search features for instance. Assemblies explicitly tell you their AssemblyVersion, so why would you want history on those files?

What you really want to is to have history on your dependencies! You want to be able to say: version A of my app depends on version X of a dependency, version B of my app depends on version Y of that same dependency, we upgraded X to Y on that date when person P was working on feature F... That's the interesting information, not the actual binary.

Now people argue: Yeah, but I want my builds to be reproducable, a given changeset (pardon my wording: revision) should always produce the same output. Agreed! But does that mean that the dependencies over which you don't have any control need to be in source control? You know, those libraries that already are versioned, the information already compiled and baked into the binary file? Nope, just reference them, you'll only find one binary that matches that specific version of the library out there (I'm coming back on this point later in this post). Now that's exactly the kind of information you'll find in the repositories.config and packages.config files that NuGet uses to keep track of your package dependencies.

A benefit of storing the packages in source control is that its consumers don't need to rely on NuGet. This is one of the main principles behind NuGet, as outlined by Phill Haack in the announcement of NuGet. That is nice indeed, and it is cool that you can do that, but should you? There are cases where you should, but I find them rare. The main reason for me to store those packages in source control would be because you don't want to wait for that choice to be made :-) Team A kicking off a new project could start using it, while Team B has no time because it's dealing with featuritis. In such situations you could benefit from storing the packages to avoid impacting other teams. 

In my opinion, using NuGet or supporting teams that want to use it, should preferably be a company-wide strategic choice. For the same reason, you probably won't mix IDEs (Visual Studio, SharpDevelop, Notepad, VIM, other? :-)) amongst your team members, or mix source control systems (TFS, Git, SVN, ...), build servers, ...

So here is my statement:

  • don't store the binaries in source control: keep NuGet packages out
  • only store the metadata in source control: keep track of repositories.config and packages.config files
  • have a strategy: standardising the way you embrace the power of NuGet inside the organisation will benefit you all and avoid headaches

Continuous (Package) Integration

Now, as stated earlier: every build of a given changeset/revision of your project should always produce the same output. How does this work when you depend on files which are not in source control?

Given the fact you do store what you depend on (metadata), all you need to do is fetch those dependencies in a pre-build step. This is what I call Continuous Package Integration.

Note: I'm speaking about CI builds here by example. I use the exact same approach for any other type of build: QA builds, Release builds, ...

This setup implies you have a corporate NuGet server running, preferably even with multiple feeds. There are multiple ways to accomplish this, but if you want a quick start and play with this setup, I suggest you try out MyGet (NuGet-as-a-Service) and focus on your process first. MyGet can also help in ensuring a specific package version is always available by mirroring packages from the official NuGet feed.

Optimization

Now, it might seem cumbersome to continuously fetch those packages every single time you do a build, but what prevents you of setting up a caching mechanism on your build agents? Now its my turn to tell you disk space is cheap :-) (with the addition that it's used as a local cache: no need for backup).

This could be very simply accomplished with a few PowerShell scripts hooking into the build steps, that check the local cache first, and if not present fetch it. This mechanism will be self-organizing and become faster as we use it (don't you like that?).

The same counts for your development machines: have a script and local cache for the packages you use. Run a pre-build that checks if any repositories.config or packages.config file has changed and updates your local environment.

Please, feel free to comment and share your thoughts on this matter!

3 simple steps to publish a nupkg to MyGet using NuGet Package Explorer 1.6

Today, Luan Nguyen (@dotnetjunky) announced the release of NuGet Package Explorer (NPE) version 1.6.

Those of you who installed/updated yet will have noticed he did a great job in improving the look-and-feel of the app, but I wanted to do a shout out here and point you to the feature I'm most excited about.

As you can read in the release notes, NPE now supports publishing packages to a custom source!

Now how convenient is that for.. let's say.. MyGet ?! :-)

Here's a little tutorial for you.

How to publish a package to MyGet using Nuget Package Explorer 1.6

  1. Grab your MyGet feed details (API key & URL)

    (Don't try this one, the 'Change API key' link does work :-P)

  2. Open up or create your favorite NuGet package in NuGet Package Explorer 1.6

  3. Click on File > Publish... and follow the instructions using your feed details

And that's it! Notice the "package published successfully" message and go check your MyGet feed. The next time you want to publish a package to this feed, Package Explorer will remember your API-key so you can just pick the feed URL from the dropdown. Nice and easy!

Note that package details might be cached for a minute on the site, but the feed is updated immediately.

MyGet now compatible with NuGet Package Explorer v1.5

Most of you will agree that Package Explorer is a major part within the NuGet ecosystem. In preparation for the latest version 1.5 release, Luan Nguyen (aka dotNetJunky) pointed us to an incompatibility issue with MyGet (thanks again for that!).

A new package property IsLatestVersion was added and Package Explorer depends on it for the improved Select package dialog as explained here.

I'm glad to announce that MyGet is now using this property as well, with a slight switch to it.

In the NuGet Gallery, a package with IsLatestVersion=true simply means: it is the latest one... duh! :-) for the record: the latest official one!

For MyGet, we use this property within the scope of the specific MyGet feed: this means, it is the latest version available within that specific feed. Actually, this is exactly the same behavior as within the NuGet Gallery, but the meaning is different because it concerns a private feed on MyGet. The MyGet feed is (currently) unaware of new versions that might get published on the official one. So unless you upload or add a newer package to your MyGet feed, the latest version within your private feed might get out-of-sync with the latest version within the NuGet Gallery.

Let me illustrate this with an example: let's say SomeAwesomePackage shipped a first version 1.0 in the NuGet Gallery which you added to your MyGet feed. This will be flagged with IsLatestVersion = true (it's the only version too). You later add a newer version 1.1 of the package, which in turn will be flagged as the latest version, and will unflag the 1.0 version. Now, your happy with this version and didn't upgrade for a couple of months, while version 1.2 has been made available in the NuGet Gallery. At this point in time, your latest version in the MyGet feed (v 1.1) is not the latest version from the NuGet Gallery (v 1.2). Just trying to point out the difference here :-)

To query your feed, just copy/paste your MyGet feed url into the dialog as shown below.

The little checkbox at the bottom saying "Only show latest version of each package Id." is interacting with the IsLatestVersion property of the packages.

Enjoy!

Adding NuGet packages from the official feed to your MyGet feed: some improvements

One of the things we want to improve on MyGet is the add-package functionality from the official NuGet feed. We felt this user experience could be better, so here's a first step!

First of all, the default search behavior has changed (and we hope improved as well!):

  • the term you enter in the search box is used now to scan the NuGet package ID and Title only
  • the default search method is StartsWith (self-explanatory I hope?)
  • uppercasing or lowercasing doesn't matter (we do a ToLower behind the scenes anyway)
  • by default, we now only search through the latest versions
You'll notice there are a bit more options in the UI as well, so you can adjust the behavior to your needs.

Some of the search settings are now optional:

  • search through the package Summary field
  • search through the package Description field
  • search through all versions of all packages
The moment you type at least two characters, an autocomplete box will display with your matching results, as shown below:
In a second phase, I hope to add some more useful functionality to this feature, such as search by Author, OSS license type, ...
Feel free to suggest the ones you feel are really missing.

Get your local NuGet repository online in a private MyGet feed

One of the MyGet features I've been working on lately should make it easier to populate your private feed with the NuGet packages you want.

One of the specific use cases for MyGet is to be able to quickly set up a private feed with your own packages, which you might have in a local repository on your computer.

Before MyGet, you could make this feed available to other computers by sharing your repository for instance on a network drive, or by hosting your own NuGet server.

This required you to do some plumbing to get a server running, or to manage folder security settings, and above all, if a computer trying to consume the feed was not in the network, it could not get any packages from you repository.

MyGet provides you with a service (NuGet-as-a-Service) that allows you to easily host your private NuGet feeds and have it always accessible from any computer connected to the internet, without that setup hassle.

Since we launched, you were already able to upload a single package at a time to your feed. Maarten extended this with a simple checkbox to define whether or not you want to include its dependencies. Note that this dependency resolution only works for dependencies to packages on the official NuGet feed at the moment.

To facilitate the upload progress, I've now extended it to allow you to upload multiple packages at once.

multi-package-upload

This saves you again a couple of clicks and redirects!

When the packages are successfully uploaded to your feed, you'll get a nice notification telling you which packages have been added.

multi-package-upload success

Noticed that the above upload of 2 packages results in 3 packages being added? (it resolved a dependency to elmah.corelibrary 1.2 for elmah 1.2.0.1)