MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

Enhancements to MyGet Gallery for Enterprise subscriptions

All MyGet Enterprise subscriptions feature a complete “copy” of MyGet, tailored to your organization’s development team. A custom URL provides access to private repositories used by your team. But what if you want to make a specific feed available to the general audience? And how can private feeds within the organization be easily discovered by team members?

The answer to these two questions? The MyGet Gallery. It serves as the Golden Pages for your team’s NuGet feeds. Through the feed settings, a feed can be listed in the gallery, complete with a readme and icon.

Adding a feed to the gallery

Only public and community feeds will be shown in the public MyGet Gallery. Feeds that are public within your team will only be shown in the MyGet Gallery for authenticated users. This makes the MyGet Gallery a great place to discover feeds! Your customers can browse the MyGet Gallery and see which feeds they can use. Team members will see feeds that are only available within the organization and can have a look, too.

MyGet Gallery

And if you want to disable the MyGet Gallery? Simply use the administration dashboard to do so.

Disable gallery

Happy packaging!

We Love Our Customers!

Whether you are into Valentine's Day or not, we love you! That's why we are offering a 50% discount to all new subscriptions (Starter, Professional) as well as to all subscription renewals (Starter, Professional) purchased on Valentine's Day and the whole weekend after! 

We want you to spend time with the ones you love and care about, and not worry about missing this limited offer. As we are offering our services worldwide, we also don't want to annoy anyone with timezone differences. 

That's why this offer is valid from Friday February 14th (UTC-12) until Sunday February 16th (UTC+12). No matter what timezone you're in, you have the whole 4 days :-)

Checkout the subscription plans and share the love!

Happy packaging!

Default feed endpoint issues 7 February 2013 - Post mortem

On February 7, 2013, we've experienced a partial downtime. The default endpoint for consuming NuGet feeds was slow to respond (+/- 2 minutes for a simple request) and caused issues with a lot of our users and their development teams and processes. We share your pain: we use MyGet for developing MyGet and we hate when things like this happen as much as you do. Our sincerest apologies for the inconvenience caused!

In this post, we'll have a look at what happened and the reason this happened. But before we dive in: we've had 3 hours of partial downtime on the default feed endpoint which most users have configured. We want to apologize by extending all paid subscriptions (Starter, Professional and Enterprise) with one additional month. No action is required on your end, your subscription has already been extended.

What happened?

At around 3:15 PM (CET), our monitoring alerted us about latency on the default feed endpoint going up. After a quick investigation and some additional time, latency went back to normal. One hour later, around 4:15 PM (CET), we saw the same happening again, as well as a number of support e-mails coming in.

MyGet has the following feed endpoints available (see documentation):

  • /F/<your_feed_name> - the NuGet v2 API endpoint
  • /F/<your_feed_name>/api/v2 - the NuGet v2 API endpoint for consuming packages
  • /F/<your_feed_name>/api/v2/package - the NuGet v2 API endpoint for pushing packages
  • /F/<your_feed_name>/api/v1 - the NuGet v1 API endpoint for consuming and pushing packages (still in use by Orchard CMS and some others)

In this case, the default endpoint we offer our users was experiencing issues, the other endpoints were not. We decided to reply all support requests with the advice to switch to the v2 feed endpoint to make sure you could continue work. This requires some reconfiguration, but we figured it's better than having very slow access and timeouts connecting to your feeds.

We checked logs, diagnostics, monitoring reports, read and re-read our code, but could not pinpoint any reason for the default feed endpoint having such high latency. At 5:00 PM (CET) we decided to fail-over to our secondary datacenter. Being a separate environment, we wanted to see if the issue would haoppen there as well. It didn't! So we flipped the DR switch and took our primary datacenter out of the loop to be able to investigate the servers without causing additional issues for our users. This fail-over solved the issue for most of our users, who were able to work with MyGet's default feed endpoint from around 6:00 PM (CET), mind some DNS propagation for some users.

Digging through more logs, we discovered the issue that caused the default feed endpoint to experience this latency (we'll elaborate in a minute). At 8:00 PM (CET), we rolled out a hotfix to our secondary datacenter to make sure the issue would not resurface there either. We worked on a final fix for the next few hours and at 11:30 PM (CET), we flipped the switch back to our primary datacenter.

Why was the default feed endpoint so slow?

As you know, MyGet supports adding upstream feeds and allows to proxy them. This is a very useful feature as one feed can essentially bundle several others, so a developer team only has to configure one and still get packages from multiple.

MyGet treats all upstream package sources as external feeds, even if an upstream feed is a MyGet feed. The reason for that is security contexts can be diferent and we want to have all security aspects applied at any time. This does mean that a MyGet feed having an upstream MyGet feed as a package source is effectively using the default feed endpoint of this second feed, going oer our internal network through the load balancer. Next to security, this also allows us to fan out queries going to multiple package sources over multiple machines.

One of our users configured upstream package sources that were referencing themselves: feed A referencing feed B referencing feed A. We prohibit self-referencing feeds, but the scenario this user tried to achieve was unsupported and resulted in a reference loop on the default feed endpoint. The default feed endpoint was bringing itself down by fanning out queries to itself.

Previous incidents teached us to partition as many things as possible, and luckily we partition all alternative feed endpoints. This allowed us to direct support requests towards the other feed endpoints so users could keep using our service.

Solution

As a quick fix, we disabled the faulting package source so our service could be resumed in a stable fashion. Once that was done, we worked on a permanent solution to this issue. Do we want to block self-referencing feeds? Absolutely, as that makes no sense. Do we want to block adding other MyGet feeds as upstream package sources? No.

We want to keep supporting having upstream package sources that can be MyGet feeds, so that complex hierarchies of feeds and privileges can be configured. Quite a number of feeds do this today, having one feed proxying a series of upstream feeds.

How do we prevent the same problem from happening again? We are now actively tracking refering feeds and detecting loops in these references. Doing this, we keep supporting all scenarios that were possible before while protecting ourselves from shooting ourselves in the foot. Feed A can reference feed B, which can reference feed A. Users can make use of feed A as the entry pont, as well as feed B, and get all expected packages on the feed. We will make sure reference loops are removed from the feed hierarchy.

When a MyGet feed references a non-MyGet feed, we will also be adding an HTTP header to all upstream requests, X-NuGet-Feedchain, which contains the full list of refering feeds. Other NuGet feed implementations can make use of this header to detect reference loops on their end as well.

While we replied to all support requests in under 5 minutes with a workaround to use one of the other endpoints, we will be working on extending our status page at http://status.myget.org, showing details about all endpoints. We also want to be able to post status messages and tips & tricks like switching to the secondary feed endpoints on there.

Hardware fails, software has bugs and people make mistakes. We strive to mitigate as many of these factors and maintain a high quality service with fast, good support. We'll keep doing that to provide you with the best experience possible.

Again, we do apologize for the inconvenience caused.

Happy packaging!

Package not found during package restore

When working with your own feed, whether private or public, chances are you want to consume more than just that feed. We see many people using their MyGet feed and the NuGet.org feed simultaneously. Sometimes, an interesting error occurs during package restore.

Unable to find version xxxx of package yyyy

That’s not funny! You know that the package is on the feed as you’ve installed it from there in the first place! The reason this happens is because the NuGet command line, the NuGet Visual Studio Extension and the NuGet PowerShell Console all have a configuration option specifying which package source to install from. When this setting is changed to one specific feed, other feeds will be ignored and the error above will be shown during package restore.

The solution is very simple: you can set the active package source to aggregate in Visual Studio, or simply configure NuGet to always use the aggregate package source for the current project. NuGet has an inheritance system for NuGet.config files, where the NuGet.config file closest to the solution file gets the last say. So in short, if you add the following NuGet.config file next to the solution file for your project, you should be fine:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>
</configuration>

We can take this one step further and instead of configuring your MyGet feed globally for your system (and requiring other devs on your team to do the same), why not distribute a NuGet.config along with the sources?

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>
  <packageSources>
    <add key="nuget.org" value="https://www.nuget.org/api/v2/" />
    <add key="MyGet" value="https://www.myget.org/F/chcuknorris/" />
  </packageSources>
  <disabledPackageSources />
  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>
</configuration>

This really makes working with multiple feeds a breeze. But we can go even further and use only MyGet, proxying packages from NuGet.org along the way. For more info on how that works, check the documentation on upstream package sources.

Happy packaging!

A very useful MyGet PowerShell suite

We recently shed a light on how you can easily use additional build tools on MyGet Build Services. However, there is more. A lot more!

We always say how much we love our users and this blog post is yet another illustration why we do. One of our users, Peter Rekdal Sunde, created an awesome PowerShell utility pack to make it even easier to customize your MyGet Build Services experience. The result is a complete build suite for creating NuGet packages and interacting with the MyGet Build Services environment. The scripts not only work on MyGet but also on your local development computer (you do need to have msysgit installed though). The entire code base was generously opensourced (MIT license) and is available on GitHub: https://github.com/peters/myget.

How does it work?

Simply include the myget.include.ps1 script in your build.ps1 on MyGet and use the provided functions.

Where do I begin?

To illustrate its purpose, we provide you a glimpse at some of the functionality provided by these scripts:

Build agent communication

  • MyGet-Write-Diagnostic - writes a diagnostic message to the standard output
  • MyGet-Build-Success - report build success
  • MyGet-Die - report build failure

NuGet utility functions

  • MyGet-NuGetExe-Path - path to NuGet.exe
  • MyGet-NuGet-Get-PackagesPath - returns the value of the repositoryPath attribute in nuget.config for a given project folder

Build steps

  • MyGet-Build-Bootstrap - starts a build (including NuGet package restore)
  • MyGet-Build-Solution - starts a build of a solution file
  • MyGet-Build-Project - starts a build of a project file
  • MyGet-Build-Nupkg - creates a NuGet package based on a specified .nuspec file. The .nuspec can contain additional replacement tokens, taking benefit from some of the variables provided by default by MyGet Build Services. More information at https://github.com/peters/myget#nuspec-substitutions.

Test runners

  • MyGet-TestRunner-Nunit - invoke NUnit
  • MyGet-TestRunner-Xunit - invoke XUnit

We recommend you to check out the readme and the samples for a detailed view of what's available though. Especially the test runner support is really nice, just check the below example!

Thank you Peter!

Happy packaging!

Build Status Badges

With MyGet Build Services, you can embed a status image for a build into any web page out there, including your project’s README file or documentation. Your users will be immediately updated about the status of the last build performed. Here’s an example badge for a successful build:

MyGet Build Services Status Badge

Badges will be shown for pending builds (queued or building) as well as successful and failed builds.

The URL for a build badge can be obtained through the Build Services configuration:


It can then be used in HTML, for example with a hyperlink to your feed on the MyGet Gallery:

<a href="https://www.myget.org/gallery/googleanalyticstracker"><img alt="GoogleAnalyticsTracker Nightly Build Status" src="https://www.myget.org/BuildSource/Badge/googleanalyticstracker?identifier=479ff619-28f2-47c0-9574-2774ed0cd855" /></a>

You can do the same in Markdown:

[![GoogleAnalyticsTracker Nightly Build Status](https://www.myget.org/BuildSource/Badge/googleanalyticstracker?identifier=479ff619-28f2-47c0-9574-2774ed0cd855)](https://www.myget.org/gallery/googleanalyticstracker)

Of course, you can also use it in any other markup language that supports embedding images.

Happy packaging!

MyGet Year in Review 2013

2013 was a good year. We had tons of fun developing MyGet and judging from the feedback we get from you we can tell you are having fun with MyGet as well. And that is awesome! Thank you so much for sharing your feedback and your continued support!

Speaking of feedback: what do you think our 2014 roadmap should look like? Let us know through this survey!

Looking back at 2013 also involves some numbers, so here goes:

  • Average response time went from 3 081 ms back in January to 538 ms in December. That's almost a factor 6 improvement! Now let's scale those improvements to various other components of MyGet.
  • Average uptime in 2013 was 99.83%. We had 6 out of 12 months with 100% uptime. That's not too bad knowing that we combine various components with a 99.99% service availability each. Full history can be found on our status page.
  • Despite a 5min glitch on October 10, we have to go back to September 12 to find our last service issue caused by a circular package dependency, which in turn caused our package retention rules engine to choke on it. Measures have been taken to prevent this from happening on MyGet and a full root cause analysis is available on our blog.
  • We are now hosting 2394 feeds (excluding the Enterprise plan feeds), good for 85.000 unique packages and 40 GB of storage (plus 3 months worth of backups, that's 600 GB in total). Bandwidth is currently just below 1 TB a month.
  • Build services has ran 4.800 builds this year, and we're seeing quite some growth there over the last months.

Even though we deploy continuously, we tagged the following releases in 2013:

  • MyGet 1.5 (January 5th): new profile page, activity streams, feed cloning, package retention policies, CodePlex and BitBucket integration in MyGet Build Services.
  • MyGet 1.6 (February 25th): feed setting for SemVer validation, download feed as ZIP, package sources out-of-beta, lots of improvements to build services (versioning, SDKs and tools, logs, ...)
  • MyGet 1.7 (May 17th): new documentation site, search and filtering enhancements, NuGet 2.5 compatibility, Package Source Discovery, auto-publish symbols feed-setting, build service enhancements (assemblyinfo-patching, links to produced packages, support for build.ps1/.cmd/.bat, build parameters)
  • MyGet 1.8 (September 10th): NuGet 2.7 compatibility (new package restore!), auto-update metadata from upstream packages, package pinning, source labeling when pushing packages upstream, automatic package mirroring for feed-proxies, build services enhancements (build labeling compatible with GitHub releases, support for MyGet.ps1/.cmd/.bat).
  • And the obvious bug fixes (and occasionally new bugs) along the way...

We also shipped our second book: Pro NuGet, 2nd Edition (October 7th), packed with tons of new stuff and a dedicated chapter of recipes. Thanks again to Apress and the NuGet team for the high-quality reviews and helping us ship what is probably the most complete NuGet guide out there.

Let's make 2014 another great year for MyGet, NuGet and dependency management in general!

Happy New Year! And happy packaging!

PS: Let us know your feedback on what you want from MyGet in 2014 through this survey!

Publishing packages to NuGet.org during build

Ever wanted to push a package to NuGet.org or another feed during a build on MyGet Build Services? Affraid of checking in the API key to source control just to be able to do that? Well here’s a little trick that will help you do that without spilling secrets.

When we implemented support for NuGet Package Restore, we’ve also added support for transfering package source credentials to the build server in a safe way. From the Package Sources tab on your feed, you can use the Add package source button to specify all details about a feed that should be available during build, both for consuming and pushing packages. You can add any feed you want: a Chocolatey feed, a TeamCity feed or another MyGet feed.

NuGet Push in MyGet build

After specifying an API key through MyGet, you can simply push packages during build, from your build.bat. Let’s push all packages in the build’s release folder to NuGet.org:

nuget push release/*

Prefer pushing MyPackage 1.0 to another feed? Add it as a package source in MyGet, specify the API key and push from build.bat:

nuget push MyPackage.1.0.nupkg -Source http://other-feed

Note that the packages generated during build will also be added to your MyGet feed.

Happy packaging!

GitHub Releases and MyGet Build Services

During the summer, GitHub added a great feature for many projects: releases. Projects on GitHub can create a release and label sources, add binaries and release notes. Following the conventions of many Git projects, releases are tied to Git tags. You can use an existing tag, or let releases create the tag when it's published.

Since MyGet Build Services supports labeling and we can label sources when pushing packages upstream, GitHub Releases make a great combination with MyGet! Let’s see.

Labeling Sources from MyGet

When enabled in the build source configuration on MyGet, source code can be labeled with the build number. This can be done for successful builds only (recommended) as well as for failed builds.

Label Sources during Build and make it a GitHub release

It is also possible to base tags for GitHub Releases on having a fresh package pushed to, say, NuGet.org. This can be done while pushing packages upstream or “promoting” them. A dialog will provide you with additional options, e.g. configure the package version to be used upstream, as well as the option to label the sources for packages originating from MyGet Build Services.

GitHub release when pushing to NuGet.org

Releasing through GitHub

Regardless of the approach taken for creating a label, all that is left now is creating the release on GitHub. Since either the build process or the package promotion process have created the label already, all that GitHub requires is a set of release notes and a description for the release.

Integrating GitHub Releases and MyGet

Pretty sweet, no? Let us know your thoughts through the comments below or in our forums!

Happy packaging and shipping!

Labeling Sources when Pushing to NuGet.org

When adding package sources through your feed’s settings, a very nice scenario becomes available: the package promotion workflow. In other words: pushing a package from one feed to another. Or in other words: publishing nightlies to MyGet and promoting specific package versions to NuGet.org.

With the newly introduced labeling feature, it is now possible to label sources when pushing a package upstream. When enabled, MyGet will find the build from which the package originated and will add a label to the source control revision it was built from. Note that the build must originate from MyGet Build Services for this to work.

Choose the package you want to promote and with a click of a button you can push it upstream. A dialog will provide you with additional options, e.g. configure the package version to be used upstream.

Label sources when pushing upstream

An important note: If you want to make use of labeling, you will have to specify credentials to connect to the remote repository, or remove and add the build source again. Labeling will fail if this is neglected.

Let us know what you think about this feature through the comments below or in our forums!

Happy packaging!