MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

MyGet Year in Review 2013

2013 was a good year. We had tons of fun developing MyGet and judging from the feedback we get from you we can tell you are having fun with MyGet as well. And that is awesome! Thank you so much for sharing your feedback and your continued support!

Speaking of feedback: what do you think our 2014 roadmap should look like? Let us know through this survey!

Looking back at 2013 also involves some numbers, so here goes:

  • Average response time went from 3 081 ms back in January to 538 ms in December. That's almost a factor 6 improvement! Now let's scale those improvements to various other components of MyGet.
  • Average uptime in 2013 was 99.83%. We had 6 out of 12 months with 100% uptime. That's not too bad knowing that we combine various components with a 99.99% service availability each. Full history can be found on our status page.
  • Despite a 5min glitch on October 10, we have to go back to September 12 to find our last service issue caused by a circular package dependency, which in turn caused our package retention rules engine to choke on it. Measures have been taken to prevent this from happening on MyGet and a full root cause analysis is available on our blog.
  • We are now hosting 2394 feeds (excluding the Enterprise plan feeds), good for 85.000 unique packages and 40 GB of storage (plus 3 months worth of backups, that's 600 GB in total). Bandwidth is currently just below 1 TB a month.
  • Build services has ran 4.800 builds this year, and we're seeing quite some growth there over the last months.

Even though we deploy continuously, we tagged the following releases in 2013:

  • MyGet 1.5 (January 5th): new profile page, activity streams, feed cloning, package retention policies, CodePlex and BitBucket integration in MyGet Build Services.
  • MyGet 1.6 (February 25th): feed setting for SemVer validation, download feed as ZIP, package sources out-of-beta, lots of improvements to build services (versioning, SDKs and tools, logs, ...)
  • MyGet 1.7 (May 17th): new documentation site, search and filtering enhancements, NuGet 2.5 compatibility, Package Source Discovery, auto-publish symbols feed-setting, build service enhancements (assemblyinfo-patching, links to produced packages, support for build.ps1/.cmd/.bat, build parameters)
  • MyGet 1.8 (September 10th): NuGet 2.7 compatibility (new package restore!), auto-update metadata from upstream packages, package pinning, source labeling when pushing packages upstream, automatic package mirroring for feed-proxies, build services enhancements (build labeling compatible with GitHub releases, support for MyGet.ps1/.cmd/.bat).
  • And the obvious bug fixes (and occasionally new bugs) along the way...

We also shipped our second book: Pro NuGet, 2nd Edition (October 7th), packed with tons of new stuff and a dedicated chapter of recipes. Thanks again to Apress and the NuGet team for the high-quality reviews and helping us ship what is probably the most complete NuGet guide out there.

Let's make 2014 another great year for MyGet, NuGet and dependency management in general!

Happy New Year! And happy packaging!

PS: Let us know your feedback on what you want from MyGet in 2014 through this survey!

Publishing packages to NuGet.org during build

Ever wanted to push a package to NuGet.org or another feed during a build on MyGet Build Services? Affraid of checking in the API key to source control just to be able to do that? Well here’s a little trick that will help you do that without spilling secrets.

When we implemented support for NuGet Package Restore, we’ve also added support for transfering package source credentials to the build server in a safe way. From the Package Sources tab on your feed, you can use the Add package source button to specify all details about a feed that should be available during build, both for consuming and pushing packages. You can add any feed you want: a Chocolatey feed, a TeamCity feed or another MyGet feed.

NuGet Push in MyGet build

After specifying an API key through MyGet, you can simply push packages during build, from your build.bat. Let’s push all packages in the build’s release folder to NuGet.org:

nuget push release/*

Prefer pushing MyPackage 1.0 to another feed? Add it as a package source in MyGet, specify the API key and push from build.bat:

nuget push MyPackage.1.0.nupkg -Source http://other-feed

Note that the packages generated during build will also be added to your MyGet feed.

Happy packaging!

GitHub Releases and MyGet Build Services

During the summer, GitHub added a great feature for many projects: releases. Projects on GitHub can create a release and label sources, add binaries and release notes. Following the conventions of many Git projects, releases are tied to Git tags. You can use an existing tag, or let releases create the tag when it's published.

Since MyGet Build Services supports labeling and we can label sources when pushing packages upstream, GitHub Releases make a great combination with MyGet! Let’s see.

Labeling Sources from MyGet

When enabled in the build source configuration on MyGet, source code can be labeled with the build number. This can be done for successful builds only (recommended) as well as for failed builds.

Label Sources during Build and make it a GitHub release

It is also possible to base tags for GitHub Releases on having a fresh package pushed to, say, NuGet.org. This can be done while pushing packages upstream or “promoting” them. A dialog will provide you with additional options, e.g. configure the package version to be used upstream, as well as the option to label the sources for packages originating from MyGet Build Services.

GitHub release when pushing to NuGet.org

Releasing through GitHub

Regardless of the approach taken for creating a label, all that is left now is creating the release on GitHub. Since either the build process or the package promotion process have created the label already, all that GitHub requires is a set of release notes and a description for the release.

Integrating GitHub Releases and MyGet

Pretty sweet, no? Let us know your thoughts through the comments below or in our forums!

Happy packaging and shipping!

Labeling Sources when Pushing to NuGet.org

When adding package sources through your feed’s settings, a very nice scenario becomes available: the package promotion workflow. In other words: pushing a package from one feed to another. Or in other words: publishing nightlies to MyGet and promoting specific package versions to NuGet.org.

With the newly introduced labeling feature, it is now possible to label sources when pushing a package upstream. When enabled, MyGet will find the build from which the package originated and will add a label to the source control revision it was built from. Note that the build must originate from MyGet Build Services for this to work.

Choose the package you want to promote and with a click of a button you can push it upstream. A dialog will provide you with additional options, e.g. configure the package version to be used upstream.

Label sources when pushing upstream

An important note: If you want to make use of labeling, you will have to specify credentials to connect to the remote repository, or remove and add the build source again. Labeling will fail if this is neglected.

Let us know what you think about this feature through the comments below or in our forums!

Happy packaging!

Using additional tools in the build process

Out of the box, MyGet Build Services supports building projects that depend on various SDK's and/or test runners. A full list of supported project types and SDK's at your disposal is available on our docs.

However, sometimes you run into a missing tool or SDK you absolutely need to compile your projects. Or perhaps you need a different version than the one available by default. If that's the case, you have two options:

  • Download the tools you need and commit them to your own repository
  • Use the brand new MyGet Build Tools repository on GitHub as a submodule to your repository

The first option is likely what you're doing today and allows you to cherry-pick the required tools without polluting your build with tools you don't need, making the build process faster.

The second option will bring down the tools from this repository during a build while they are technically not in your own GitHub repository. It currently contains the following tools:

  • NuGet 2.5, 2.6, 2.7, 2.7.1
  • NUnit 2.6.2
  • XUnit 1.9.6
  • curl

To add a submodule, run the following from your repository root (make sure you use the https endpoint):

git submodule add https://github.com/myget/BuildTools.git myget

From then on, you can use the tools from the myget/tools folder in your builds. For example, NuGet 2.5 can be used by calling myget/tools/nuget/2.5/nuget.exe.

Happy Packaging!

Making your life easier with multiple access tokens

What if you are using your API key on your TeamCity server and several other locations and for some reason you have to reset that API key? Major headache? Not anymore: from now on, MyGet supports multiple access tokens.

Since MyGet day one, we’ve had support for two credentials linked to your account. The primary API key could be used when publishing packages with NuGet.exe or NuGet Package Explorer while your username and password were useful when consuming private feeds from Visual Studio or a build server.

Additional access tokens can be generated from your profile page. The primary API key can be regenerated and new tokens can be easily created or revoked. Access tokens can be given a short description: this will help keeping track of where you used the access token and revoke it if necessary.

Access tokens can be used for all authentication purposes. They can be used when pushing to your MyGet feed or as an alternate password when authenticating against a private feed or SymbolSource.org.

Go ahead and separate the credentials you use personally, on the build server or in other software. Access tokens can be created and revoked at any time.

image

Happy packaging!

Labeling Sources after Build

When enabled in the build source configuration on MyGet, source code can be labeled with the build number. This can be done for successful builds only (recommended) as well as for failed builds.Labeling Source Code from MyGet

The label originating from MyGet will always be named in the form vX.Y.Z, vX.Y.Z.P or v.X.Y.Z-pre. The description for the label will always be the label name (the version number), followed by "- MyGet Build Services". This labeling scheme is compatible with GitHub releases and can link a given NuGet package version number to a GitHub release.

An important note: If you enable build labeling on build configurations that have been created before 2013-09-11, you will have to specify credentials to connect to the remote repository, or remove and add the build source again. Builds with labeling enabled will fail if this is neglected.

Let us know what you think about this feature through the comments below or in our forums!

Happy packaging!

GitHub Commit Status API now supported

A while ago, GitHub launched their Commit Status API which allows integrating the status of a build with commits and pull requests on GitHub. When a MyGet build source is linked to a GitHub repository and has credentials specified, we have started reporting status messages back to GitHub.

GitHub Commit Status API MyGet

When a build succeeds or fails, you will see a status message posted to GitHub, linking to the build log on MyGet.

MyGet reports build status to GitHub

To enable GitHub Commit Status messages on your builds, make sure the build configuration has credentials specified. Specifying credentials can be done by removing and adding the build configuration again, a method which doesn’t require you to enter your password. You can also specify credentials manually by editing the build source.

Happy packaging!

NuGet Package Restore and MyGet Build Services

With the release of NuGet 2.7, a new way of doing package restore came to life. Package restore allows you to keep only your source code and packages.config under source control and just download and install NuGet dependencies during build.

What does this mean for MyGet Build Services? Let’s say we’re making it easier for you! When building from a Visual Studio solution or project, there is nothing you should do: we will run package restore automatically. Let’s dive into the package restore conventions we have in place. Full details on the build process are available from our documentation.

MyGet Build Services

Package Restore Conventions

MyGet Build Services runs NuGet Package Restore as part of every build of solution or project files even if it's not enabled for the solution. Note that package restore is not run for builds making use of batch or PowerShell scripts. In those cases, you are the responsible for running package restore.

In order of precedence, the following package restore commands are run. When one succeeds, package other commands will be skipped.

  • nuget restore MyGet.sln -NoCache -NonInteractive -ConfigFile MyGet.NuGet.config
  • nuget restore MyGet.sln -NoCache -NonInteractive -ConfigFile NuGet.config
  • nuget restore <your solution file> -NoCache -NonInteractive -ConfigFile MyGet.NuGet.config
  • nuget restore <your solution file> -NoCache -NonInteractive -ConfigFile NuGet.config
  • nuget restore packages.config -NoCache -NonInteractive -ConfigFile MyGet.NuGet.config
  • nuget restore packages.config -NoCache -NonInteractive -ConfigFile NuGet.config
  • nuget restore MyGet.sln -NoCache -NonInteractive
  • nuget restore <your solution file> -NoCache -NonInteractive
  • nuget restore packages.config -NoCache -NonInteractive

If you are working with other feeds than the default NuGet.org feed, there are some options available.

Restoring from other Package Sources

If you want MyGet Build Services to restore packages from a specific feed, there are several options available to do this. The easiest way of making a package source available to the build process is by adding it through the UI.Making package source available during build

Using the Add package source button, you can add any feed that you want to make available during build: a Chocolatey feed, a TeamCity feed or another MyGet feed. It is even possible to store credentials so an authenticated feed can be used during build.

If you prefer to have all feed information in your source repository, that’s an option too. Adding a MyGet.NuGet.config file to your repository is the key to success. See the NuGet docs for more information on how such file can be created. The following is a sample registering a custom NuGet feed for package restore.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="Chuck Norris feed" value="https://www.myget.org/F/chucknorris/api/v2/" />
  </packageSources>
  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>
</configuration>

 

A small remark: if you have an authenticated feed but do not wish to add credentials to source control, credentials can be added to the feed's package source. These credentials will be available during build and allow you to consume a protected feed with ease. Authentication of feeds

Summary

Using NuGet Package Restore in MyGet Build Services is a breeze. We follow standard NuGet conventions to do your builds and allow adding package sources which will be made available during build through the UI. If you do need to customize things, there are several hooks available too. Full details on the build process are available from our documentation.

Let us know what you think about this feature through the comments below or in our forums!

Happy packaging!

Retention Rules and Package Pinning

When using MyGet feeds for package archiving or nightly builds, a lot of packages show up on the feed very quickly. By default, we keep all package versions available on your feed. If you would like to do some automated housekeeping, retention rules can be added per feed. Whenever a package is added to your feed, we'll make sure these retention rules are respected.

Package retention rules for a feed

Our latest release added some additional options for package retention. You can now:

  • Specify the number of stable versions to keep
  • Specify the number of prerelease versions to keep
  • Specify if we have to keep depended packages or not

That last one is different from what we did in the past. We used to blindly delete packages that were older than version X, even if there were packages depending on it. From now one, we will keep packages that are depended on unless specified through the settings.

Pinning Package Versions

Certain packages should never be deleted automatically. For every package, we now support pinning specific versions – a pinned package will never be deleted by the retention rules. From the package details, this can be done using the pin/unpin buttons.

Pinning packages - ignore retention rules

Let us know what you think about this feature through the comments below or in our forums!

Happy packaging!