MyGet Blog

Package management made easier

NAVIGATION - SEARCH

NuGet Dependency Management with Drone Delivery

Using MyGet just became easier. We are proud to announce a new feature (in preview) which brings a better and bolder way of consuming NuGet packages from your feed! Next to using Visual Studio or the NuGet command line tool to have packages delivered to your project, it is now possible to have packages delivered by drones using the new Drone Delivery feature.

Many established companies, as well as startups, are experimenting with drones for their services. MyGet will be the first to offer dependency management using this approach. And for good reasons: the Drone Delivery feature will make package restore a breeze even if you lose your Internet connection.

Here’s an overview of how Drone Delivery works:

How drone delivery works

The new Drone Delivery feature will surface in many places throughout the MyGet website, for example on package details pages and in the MyGet Gallery. It is also possible to consume all packages from a feed using Drone Delivery:

Drone Delivery of an entire feed

We're really excited about this feature and will be adding additional capabilities in the future. We are thinking about Google Glass apps and Oculus VR support to enable tracking package delivery in real time.

More information on this new feature can be found in our documentation. If you want the preview of Drone Delivery enabled for your account, let us know.

Happy packaging!

Setting default package sources during build

MyGet gives you the option to specify one or more package sources for a feed. Package sources for a feed are also available during every build on MyGet Build Services. This can be really useful!

  • An additional package source is needed during build. MyGet will make the package source available during build if it has been added to the feed's package sources.
  • If you have an authenticated feed but do not wish to add credentials to source control, credentials can be added to the feed's package source. These credentials will be available during build and allow you to consume a protected feed with ease.
  • The API key for a package source is also transferred to the build server. This means during a build, you can call into nuget.exe push and push packages to configured package sources.
  • You want to make use of nuget.exe push in a build script without having to specify the -Source parameter.
Setting default package sources during build

The NuGet.config on our build machines is configured using NuGet's defaults, enriched with all package sources configured for a feed. Based on these defaults, the following conventions are active:

  • The default package source is set to (Aggregate Source), meaning all feeds will be queried for packages in the order defined in the feed's package sources.
  • The default push source (when using nuget push without the -Source parameter) is NuGet.org.

Both of these conventions can be overridden by editing the build source configuration:

Setting package sources used during a build with NuGet

Happy packaging!

MyGet Documentation site redesigned

When we first launched the MyGet Documentation site, we decided to fork the NuGet documentation site and apply our own colors and content to it. After our website redesign a few months ago, we felt it was time to work on our documentation site’s design, too.

Documentation on how to use MyGet

The front page looks completely different. We decided to put a search engine central, as well as some popular articles that can help you get started.

One of the things we want to encourage everyone to do is comment on documentation: explain how you did something, ask questions and get help. If we see there are some things that are not completely clear from these comments, we’ll work on additional documentation there. Therefore, every article now gets a section where you can add your comments.

Add comments to MyGet documentation

Not that we are lazy, but if you feel you can do a better job at an article, spot a typo or want to add something, every article features a direct link to our GitHub repository where you can send us a pull request with changes. And that’s not work you’re doing for free: for every accepted Pull Request, you get a free one month extension of your current subscription.

image

Happy packaging!

Release notes for MyGet 1.9

MyGet 1.9 was released on February 27, 2014. We will be blogging about new features in the next days and weeks.

Features

MyGet

MyGet Enterprise

MyGet Build Services

Bug Fixes

  • Packages downloaded through the browser now have a .nupkg file extension
  • NuGet Package Explorer 2.8 publishing works again
  • Package restore with proxied feeds now works on feeds larger than 100 packages
  • Load time of activity feeds has been improved
  • Push upstream now works with private feeds

Next to all these, we have done a tremendous effort on our back-end: upgrade to the latest Windows Azure SDK and switch to JSON-based traffic to our storage accounts, a new queuing framework which increases back-end messaging throughput, ...

Happy packaging!

A very useful MyGet PowerShell suite

We recently shed a light on how you can easily use additional build tools on MyGet Build Services. However, there is more. A lot more!

We always say how much we love our users and this blog post is yet another illustration why we do. One of our users, Peter Rekdal Sunde, created an awesome PowerShell utility pack to make it even easier to customize your MyGet Build Services experience. The result is a complete build suite for creating NuGet packages and interacting with the MyGet Build Services environment. The scripts not only work on MyGet but also on your local development computer (you do need to have msysgit installed though). The entire code base was generously opensourced (MIT license) and is available on GitHub: https://github.com/peters/myget.

How does it work?

Simply include the myget.include.ps1 script in your build.ps1 on MyGet and use the provided functions.

Where do I begin?

To illustrate its purpose, we provide you a glimpse at some of the functionality provided by these scripts:

Build agent communication

  • MyGet-Write-Diagnostic - writes a diagnostic message to the standard output
  • MyGet-Build-Success - report build success
  • MyGet-Die - report build failure

NuGet utility functions

  • MyGet-NuGetExe-Path - path to NuGet.exe
  • MyGet-NuGet-Get-PackagesPath - returns the value of the repositoryPath attribute in nuget.config for a given project folder

Build steps

  • MyGet-Build-Bootstrap - starts a build (including NuGet package restore)
  • MyGet-Build-Solution - starts a build of a solution file
  • MyGet-Build-Project - starts a build of a project file
  • MyGet-Build-Nupkg - creates a NuGet package based on a specified .nuspec file. The .nuspec can contain additional replacement tokens, taking benefit from some of the variables provided by default by MyGet Build Services. More information at https://github.com/peters/myget#nuspec-substitutions.

Test runners

  • MyGet-TestRunner-Nunit - invoke NUnit
  • MyGet-TestRunner-Xunit - invoke XUnit

We recommend you to check out the readme and the samples for a detailed view of what's available though. Especially the test runner support is really nice, just check the below example!

Thank you Peter!

Happy packaging!

Build Status Badges

With MyGet Build Services, you can embed a status image for a build into any web page out there, including your project’s README file or documentation. Your users will be immediately updated about the status of the last build performed. Here’s an example badge for a successful build:

MyGet Build Services Status Badge

Badges will be shown for pending builds (queued or building) as well as successful and failed builds.

The URL for a build badge can be obtained through the Build Services configuration:


It can then be used in HTML, for example with a hyperlink to your feed on the MyGet Gallery:

<a href="https://www.myget.org/gallery/googleanalyticstracker"><img alt="GoogleAnalyticsTracker Nightly Build Status" src="https://www.myget.org/BuildSource/Badge/googleanalyticstracker?identifier=479ff619-28f2-47c0-9574-2774ed0cd855" /></a>

You can do the same in Markdown:

[![GoogleAnalyticsTracker Nightly Build Status](https://www.myget.org/BuildSource/Badge/googleanalyticstracker?identifier=479ff619-28f2-47c0-9574-2774ed0cd855)](https://www.myget.org/gallery/googleanalyticstracker)

Of course, you can also use it in any other markup language that supports embedding images.

Happy packaging!

Publishing packages to NuGet.org during build

Ever wanted to push a package to NuGet.org or another feed during a build on MyGet Build Services? Affraid of checking in the API key to source control just to be able to do that? Well here’s a little trick that will help you do that without spilling secrets.

When we implemented support for NuGet Package Restore, we’ve also added support for transfering package source credentials to the build server in a safe way. From the Package Sources tab on your feed, you can use the Add package source button to specify all details about a feed that should be available during build, both for consuming and pushing packages. You can add any feed you want: a Chocolatey feed, a TeamCity feed or another MyGet feed.

NuGet Push in MyGet build

After specifying an API key through MyGet, you can simply push packages during build, from your build.bat. Let’s push all packages in the build’s release folder to NuGet.org:

nuget push release/*

Prefer pushing MyPackage 1.0 to another feed? Add it as a package source in MyGet, specify the API key and push from build.bat:

nuget push MyPackage.1.0.nupkg -Source http://other-feed

Note that the packages generated during build will also be added to your MyGet feed.

Happy packaging!

Release notes for MyGet 1.8

MyGet 1.8 was released on September 10, 2013. We will blog about new features in the next days and weeks.

Features

MyGet

  • Support for NuGet 2.7
  • Metadata for packages is auto-updated from upstream feeds
  • Retention policies: pin packages so they don't get deleted
  • Retention policies: packages that are depended on will no longer be deleted (unless explicitly enabled)
  • Push upstream: package source code repositories can be labeled when pushing packages upstream
  • Send e-mail when feed permissions change
  • Users can revoke their own access from a feed
  • Automatic mirroring for packages from upstream feeds when feed proxy is enabled

MyGet Enterprise

  • Administrators can now join a feed. Feed owners are notified of this action.

MyGet Build Services

  • Repositories from GitHub organizations are now shown
  • The latest build version is shown in the UI
  • Package sources added at the feed level are available on the build server
  • Automatic support for NuGet package restore even if it's not enabled for the solution
  • Support for NuGet 2.7 package restore - see http://docs.myget.org/docs/reference/build-services#Package_Restore
  • Package sources added at the feed level are available on the build server for package restore
  • Build labeling: on succesful or failed builds, a label can be added to the sources. This is compatible with GitHub Releases. - see http://docs.myget.org/docs/reference/build-services#Sourcelabeling(tagging)
  • Support for MyGet.cmd, MyGet.bat, MyGet.ps1

Bug Fixes

  • Fixed an issue with copy to clipboard
  • Fixed an issue when pushing packages upstream to authenticated feeds
  • Support page is no longer behind an authentication wall
  • Fixed an issue when build services used @ in the username
  • Various performance improvements

Happy packaging!

Downtime September 11 and 12, 2013 - Root cause

On September 11 and 12, 2013, we have experienced some downtime. The website and back-end services have crashed a number of times across all instances, bringing our system to a halt a couple of times. We hate downtime as much as you do and want to apologize for the trouble this may have caused you. Let’s have a look at the symptoms and root cause.

Symptoms

Twitter and e-mail notified us of this a good 2 minutes before our monitoring kicked in. When we had a look at our status dashboards, we saw high CPU usage on all machines, inexplicable crashes of machines and an error message in the Windows event logs right before the machines died and restarted.

In our back-end queue, where all work is added and waits to be processed, we found 25 messages that seemed to be retried over and over again, having been dequeued over 150 times.

We have manually removed these messages from the queue and inserted them again after the website was running stable again. After monitoring the situation for a couple of hours, all systems seemed to be running stable again. Seemed…

A number of hours later, we received some additional monitoring events. The problem was back! This time we decided not to fire fight but dig deeper into the issue we were facing. We took the site offline and connected to the machines to analyze event logs and so on. Except for the error message in the Windows event log, nothing was going on. Next, hooked a debugger to it, waiting for anything to happen.

The issue

Perhaps a root cause analysis is not the best moment to blog about our new features and updates in our latest deployment, but we have to in this case. Last Tuesday, we deployed our 1.8 release which brings an update to retention policies. These allow MyGet to automatically delete packages from a feed after a certain number of packages has been added, an automatic feed cleanup as it were.

The updated retention policies feature now tries not to break a dependency chain. Before, packages that were depended on by other packages but were subject to the retention policy settings would be deleted. Now, the retention policy handler respects the entire dependency chain for a package and will no longer remove depended packages. This, of course, requires us to parse the entire dependency tree for a package.

imageNow what happens if package A depends on package B, B depends on C and C depends on B? Right: that’s a potential infinite loop. Guess what was happening: a StackOverflowException, crashing our entire machine without giving it the chance to write output to our error logs.

Because of the crash, the message in our backend was requeued over and over. We typically move a message to a dead letter queue after a number of retries (typically 32). However because of the crash, that logic couldn’t kick in and move the message out, resulting in the message being requeued over and over again, triggering the same behavior again.

While we test our application thoroughly before going to production, this was an edge case. Who would create a package with circular dependencies anyway? We found that on one feed, a circular package dependency did exist.

Solution

We have made a fix to our algorithm which now supports this scenario and will stop following the circular dependency.

After confirming our algorithm, we have deployed it at 6 AM (CET) this morning, resolving the issue. The messages that were stuck in our queue (and triggered the initial issue) have been requeued to confirm correct working and  were all processed successfully.

Our status page can always be found at http://status.myget.org, showing uptime of our last months as well as the outage of the past hours.

Again, we do apologize for the inconvenience caused.

Happy packaging!

MyGet is running over HTTPS only

A couple of weeks ago we've informed you that from July 1st, 2013, MyGet would be switching to HTTPS traffic only. We have just deployed these changes and from now on, your feed URL should be prefixed with https:// instead of http://.

Haven't made the switch yet? No problem: we will be redirecting most traffic to the new HTTPS endpoint. However this may add a little latency to your requests. Hence it's best to switch your URL now if you haven't done so.

To help in updating feed URLs on developer machines, you can make use of package source discovery. http://docs.myget.org/docs/reference/package-source-discovery
In short, every developer can issue the following commands in his/her Visual Studio Package Manager Console to update feed URLs:

Install-Package DiscoverPackageSources
Discover-PackageSources -Url https://www.myget.org/Discovery/Feed/ -OverwriteExisting

Protecting the security and privacy of our users is one of our most important tasks at MyGet. The fact that you can safely store your intellectual property on our servers is the best proof of that. We are confident this one-time change will make the entire MyGet experience even more secure.

Do not hesitate to contact us if you have additional questions through the comments below.

Happy packaging!

PS: If you are a MyGet Enterprise customer and are using your own domain name, no action isrequired on your side.