MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

Implementing custom package retention using webhooks

Earlier this week, we got the question if custom retention policies could be enforced on MyGet. More specific, the request was to be able to keep the latest 5 versions of a minor version (e.g. keep 1.0.6 through 1.0.2, but delete 1.0.1 and 1.0.0). We’ve introduced webhooks on MyGet to enable exactly this sort of scenarios!

Our own retention policies run whenever a package is added to a feed, so let’s see if we can implement a webhook handler that does exactly what was asked… The code for this blog post can be found in our GitHub organization.

Building the webhook handler

We’ll first need something that can run our custom logic whenever a webhook event is raised. This can be an ASP.NET MVC, Web API, NancyFx or even a PHP application. In this case, let’s go with an ASP.NET Web API controller. We want to be triggered on POST when a package added event is raised.

// POST /api/retention public async Task<HttpResponseMessage> Post([FromBody]WebHookEvent payload) { // The logic in this method will do the following: // 1) Find all packages with the same identifier as the package that was added to the originating feed // 2) Enforce the following policy: only the 5 latest (stable) packages matching the same minor version may remain on the feed. Others should be removed. string feedUrl = payload.Payload.FeedUrl; // Note: the following modifies NuGet's client so that we authenticate every request using the API key. // If credentials (e.g. username/password) are preferred, set the NuGet.HttpClient.DefaultCredentialProvider instead. PackageRepositoryFactory.Default.HttpClientFactory = uri => { var client = new NuGet.HttpClient(uri); client.SendingRequest += (sender, args) => { args.Request.Headers.Add("X-NuGet-ApiKey", ConfigurationManager.AppSettings["Retention:NuGetFeedApiKey"]); }; return client; }; // Prepare HttpClient (non-NuGet) var httpClient = new HttpClient(); httpClient.DefaultRequestHeaders.Add("X-NuGet-ApiKey", ConfigurationManager.AppSettings["Retention:NuGetFeedApiKey"]); // Fetch packages and group them (note: only doing this for stable packages, ignoring prerelease) var packageRepository = PackageRepositoryFactory.Default.CreateRepository(feedUrl); var packages = packageRepository.GetPackages().Where(p => p.Id == payload.Payload.PackageIdentifier).ToList(); foreach (var packageGroup in packages.Where(p => p.IsReleaseVersion()) .GroupBy(p => p.Version.Version.Major + "." + p.Version.Version.Minor)) { foreach (var package in packageGroup.OrderByDescending(p => p.Version).Skip(5)) { await httpClient.DeleteAsync(string.Format("{0}api/v2/package/{1}/{2}?hardDelete=true", feedUrl, package.Id, package.Version)); } } return new HttpResponseMessage(HttpStatusCode.OK) { ReasonPhrase = "Custom retention policy applied." }; }

Once we have this in place and are hosting it somewhere, we can configure the webhook on our MyGet feed.

Configuring the webhook

On our MyGet feed, we can create a new webhook. It should send application/json for the package added event to the URL where we deployed the above code.

Configure web hook

When this hook now triggers, we will be retaining just the 5 latest minor versions of a package (ignoring prereleases).

That’s it. Using nothing but webhooks, we can run our own retention policies (or other logic) when something happens on our feed (like strong-name signing packages, for example). There are a number of events that we can subscribe to!

Happy packaging!

User-defined environment variables in MyGet builds

Sometimes you may want to pass in a value to the build scripts without hard-coding it into the build script. MyGet now supports setting additional environment variables (that can be used in custom build scripts as well as plain MSBuild). From the Build Source configuration, you can add up to 15 environment variables that will be made available during build.

Edit environment variables

The open/closed eye icon next to the environment variable can be used to show/hide the environment variable value in the build log. Sometimes it is not desirable to have things like passwords or API keys shown in the build log, and by making the environment variable hidden we’ll hide it in the build log.

Shown in build log

Happy packaging!

Notifications let you know when a package is updated

Call it human nature, but whenever an update to a software package is available we want to know about it and install it immediately. We do this with our computer, phone and even our NuGet packages. For good reason! The packages we depend upon may be updated because of bug or security fixes and even new features we want to make use of.

MyGet already supported automatically updating NuGet packages from upstream package sources, however we felt that this might not always be the best solution for all scenarios. We’ve now made it possible to receive e-mail notifications about package updates instead, allowing you to decide manually if you want to upgrade or not.

Update notifications per e-mail

E-mail notifications can be enabled from your feed’s Package Sources settings, editing the upstream package source configuring which actions should be performed and when.

NuGet update notifications

We will send you an e-mail (or perform an automatic update, or both) at the chosen interval. This helps you keep up-to-date and allows for an easy update of the package: after clicking a link in the e-mail, MyGet will show what the latest available version is and offers a one-click install.

Latest available NuGet package version

Happy packaging!

Introducing MyGet webhooks

One of the most requested features for MyGet so far is support for webhooks. We’re happy to announce MyGet webhooks (preview) is available today. Every MyGet feed provides the option to communicate with external services, such as a web server, whenever a specific action occurs on the feed.

Webhooks for MyGet

Only feed owners and co-owners can manage webhooks for a feed. Each webhook can be triggered for one or more event types, depending on the implementation. Webhook deliveries can be inspected, including full logs, as well as redelivered in case this is needed.

Delivery log

Give webhooks at try! For more information, check our documentation website. We’re open for any feedback or feature requests you may have.

Happy packaging!

MyGet now offered through the Microsoft Azure Store

Microsoft Azure

We are happy to announce that MyGet partnered with Microsoft to be included in the Microsoft Azure Store. This is our second partnership with them, since we also integrate with Visual Studio Online. The Microsoft Azure Cloud is widely used to build and deliver applications to end-users. Developing these applications could greatly benefit from using a package management solution to streamline development, and that’s where MyGet comes in.

Having MyGet available in the Microsoft Azure Store allows developers and team leads to create MyGet accounts for themselves or their team members, whether free or one of the paid plans. There are no separate bills to be paid: MyGet will be part of the monthly account statement or Enterprise Agreement.

imageIf you are a Microsoft Azure customer, here’s how to create a new MyGet account:

  1. Go to the Microsoft Azure Management Portal
  2. In the bottom toolbar, click New and select Store
  3. In the Choose an Add-on dialog, select MyGet and click next
  4. In the Personalize Add-on dialog select the MyGet plan you want to sign up for.
  5. If you want to create one (or more) paid subscriptions, you can use the promotional code STORELAUNCH, which will give you a 25% discount during the first 6 months of the subscription (code valid until end of August, 2014).
  6. Once the subscription is created, click Manage in the bottom toolbar and pick the username and password you wish to use for your MyGet subscription.

If you are an existing MyGet user and would like to make use of the Microsoft Azure Store integrated billing, contact support and we will make it happen!

Happy packaging!

Promoting packages generated during build

We’ve supported the “Push upstream” workflow for quite a while now. This workflow allows you to promote packages from one feed to another, ideal when you are pushing prerelease packages on one feed and pushing them as stable packages to another feed after testing them.’

So far, it has been only possible to push individual packages upstream, or all “latest” packages. We realized this was painful for one scenario: if you’re usign Build Services, it may be handy to be able to push just the packages generated by a specific build. And that is exactly what we’ve added now!

Promoting packages generated duing build

When expanding a build’s packages, a new menu entry Push upstream…is now available to push the packages generated by a build to another feed. This should greatly improve usability for this scenario.

Happy packaging!

Package mirroring is now enabled by default

Any service can experience a brief moment of downtime. This is also true for any upstream package source you configure in MyGet. That's why we have the package mirroring feature: when uploading or adding a package to your feed, the added package (and its dependencies) are stored into MyGet's own storage system and they remain available in the event of an upstream service outage.

This package mirroring checkbox used to be disabled by default. Why? Because we wanted to save on storage costs. You might think we were being cheap here, but today we are hosting over 50Gb of packages and serving many of them multiple times a day! In the early days of MyGet, it made sense to simply reference the packages from eg. nuget.org and redirect you to the package URL upstream. This has saved us quite some storage, bandwidth and backup costs whilst bootstrapping (you know the Free plan isn't free for us, right?).

However, as we are maturing, we also gain new insights in how you are using MyGet so we can more effectively reduce friction in the way you work with packages. We measured that 95% of our users are mirroring their packages, so we are now reversing the logic for it. As of today, the checkbox will be checked by default!

A popular question we get is: why aren't you hosting an entire mirror of nuget.org? Answer: this is another one of those backlog items that has been sitting around for years, and we believe NuGet API v3 will greatly facilitate such scenario. One day :)

Happy packaging!

NuGet Dependency Management with Drone Delivery

Using MyGet just became easier. We are proud to announce a new feature (in preview) which brings a better and bolder way of consuming NuGet packages from your feed! Next to using Visual Studio or the NuGet command line tool to have packages delivered to your project, it is now possible to have packages delivered by drones using the new Drone Delivery feature.

Many established companies, as well as startups, are experimenting with drones for their services. MyGet will be the first to offer dependency management using this approach. And for good reasons: the Drone Delivery feature will make package restore a breeze even if you lose your Internet connection.

Here’s an overview of how Drone Delivery works:

How drone delivery works

The new Drone Delivery feature will surface in many places throughout the MyGet website, for example on package details pages and in the MyGet Gallery. It is also possible to consume all packages from a feed using Drone Delivery:

Drone Delivery of an entire feed

We're really excited about this feature and will be adding additional capabilities in the future. We are thinking about Google Glass apps and Oculus VR support to enable tracking package delivery in real time.

More information on this new feature can be found in our documentation. If you want the preview of Drone Delivery enabled for your account, let us know.

Happy packaging!

Setting default package sources during build

MyGet gives you the option to specify one or more package sources for a feed. Package sources for a feed are also available during every build on MyGet Build Services. This can be really useful!

  • An additional package source is needed during build. MyGet will make the package source available during build if it has been added to the feed's package sources.
  • If you have an authenticated feed but do not wish to add credentials to source control, credentials can be added to the feed's package source. These credentials will be available during build and allow you to consume a protected feed with ease.
  • The API key for a package source is also transferred to the build server. This means during a build, you can call into nuget.exe push and push packages to configured package sources.
  • You want to make use of nuget.exe push in a build script without having to specify the -Source parameter.
Setting default package sources during build

The NuGet.config on our build machines is configured using NuGet's defaults, enriched with all package sources configured for a feed. Based on these defaults, the following conventions are active:

  • The default package source is set to (Aggregate Source), meaning all feeds will be queried for packages in the order defined in the feed's package sources.
  • The default push source (when using nuget push without the -Source parameter) is NuGet.org.

Both of these conventions can be overridden by editing the build source configuration:

Setting package sources used during a build with NuGet

Happy packaging!

MyGet Documentation site redesigned

When we first launched the MyGet Documentation site, we decided to fork the NuGet documentation site and apply our own colors and content to it. After our website redesign a few months ago, we felt it was time to work on our documentation site’s design, too.

Documentation on how to use MyGet

The front page looks completely different. We decided to put a search engine central, as well as some popular articles that can help you get started.

One of the things we want to encourage everyone to do is comment on documentation: explain how you did something, ask questions and get help. If we see there are some things that are not completely clear from these comments, we’ll work on additional documentation there. Therefore, every article now gets a section where you can add your comments.

Add comments to MyGet documentation

Not that we are lazy, but if you feel you can do a better job at an article, spot a typo or want to add something, every article features a direct link to our GitHub repository where you can send us a pull request with changes. And that’s not work you’re doing for free: for every accepted Pull Request, you get a free one month extension of your current subscription.

image

Happy packaging!