Join the Global Windows Azure Bootcamp 2014

Global Windows Azure BootcampIt’s no secret that MyGet is running on top of the Windows Azure cloud platform. Because we love it and want to get as many people to learn about it, we’re participating again in this year’s Global Windows Azure Bootcamp.

In April of 2013 the first Global Windows Azure Bootcamp was held at more than 90 locations around the globe. This year’s bootcamp will again offer a one day deep dive class to help thousands of people get up to speed on developing Cloud Computing Applications for Windows Azure. In addition to this great learning opportunity there will be a hands on lab in which everyone can participate. A huge global compute farm will be created by attendees to perform diabetes research!

If you want to learn about Windows Azure or help in diabetes research, find a location near you and join this massive event on Saturday, March 29, 2014. MyGet is giving away a free 2-month Starter subscription to every attendee, and other sponsors are offering swag and licenses as well!

Happy packaging in the cloud!

Author: Maarten Balliauw on 06 Mar 2014

MyGet Documentation site redesigned

When we first launched the MyGet Documentation site, we decided to fork the NuGet documentation site and apply our own colors and content to it. After our website redesign a few months ago, we felt it was time to work on our documentation site’s design, too.

Documentation on how to use MyGet

The front page looks completely different. We decided to put a search engine central, as well as some popular articles that can help you get started.

One of the things we want to encourage everyone to do is comment on documentation: explain how you did something, ask questions and get help. If we see there are some things that are not completely clear from these comments, we’ll work on additional documentation there. Therefore, every article now gets a section where you can add your comments.

Add comments to MyGet documentation

Not that we are lazy, but if you feel you can do a better job at an article, spot a typo or want to add something, every article features a direct link to our GitHub repository where you can send us a pull request with changes. And that’s not work you’re doing for free: for every accepted Pull Request, you get a free one month extension of your current subscription.

image

Happy packaging!

Author: Maarten Balliauw on 03 Mar 2014

Release notes for MyGet 1.9

MyGet 1.9 was released on February 27, 2014. We will be blogging about new features in the next days and weeks.

Features

MyGet

Author: Maarten Balliauw on 28 Feb 2014

Where does this package come from?

Ever wondered where a package comes from, or if it exists on any of your package sources? Our latest deployment features a tiny little gem on the package details page which gives us that information:

Package found on

MyGet will query all configured package sources and check if the package exists on there. If it does, a link to it will be displayed in the package details page.

Happy packaging!

Author: Maarten Balliauw on 27 Feb 2014

Which packages are added to a feed during build?

With MyGet Build Services, it is very easy to create NuGet packages from source control. Link a GitHub, BitBucket or CodePlex project to your MyGet feed and we’ll take care of building it and publishing generated packages to that feed. But which packages are added to your feed?

By default, MyGet will add all NuGet packages generated during build to your feed, as long as they are created in a folder named other than packages. The reason for this is that the packages folder is reserved by NuGet itself and may contain packages that were used during the build process and are not necessarily to be added to your feed. When creating a batch-based build, make sure to generate packages in a folder not named packages. A good example folder name could be output.

How to be selective about this? Is it possible to specify which packages are added to your feed? Well yes! To override the default behaviour, a series of wildcard matches can be specified in the build source configuration. When omitted, all packages generated during build will be pushed to your feed. When specified, only packages matching any of the specified package names or wildcards will be pushed to your feed.

image

In the above example, all package names matching Google*.nupkg or Newtonsoft* will be added to your feed.

Happy packaging!

Author: Maarten Balliauw on 24 Feb 2014

Enhancements to MyGet Gallery for Enterprise subscriptions

All MyGet Enterprise subscriptions feature a complete “copy” of MyGet, tailored to your organization’s development team. A custom URL provides access to private repositories used by your team. But what if you want to make a specific feed available to the general audience? And how can private feeds within the organization be easily discovered by team members?

The answer to these two questions? The MyGet Gallery. It serves as the Golden Pages for your team’s NuGet feeds. Through the feed settings, a feed can be listed in the gallery, complete with a readme and icon.

Adding a feed to the gallery

Only public and community feeds will be shown in the public MyGet Gallery. Feeds that are public within your team will only be shown in the MyGet Gallery for authenticated users. This makes the MyGet Gallery a great place to discover feeds! Your customers can browse the MyGet Gallery and see which feeds they can use. Team members will see feeds that are only available within the organization and can have a look, too.

MyGet Gallery

And if you want to disable the MyGet Gallery? Simply use the administration dashboard to do so.

Disable gallery

Happy packaging!

Author: Maarten Balliauw on 17 Feb 2014

We Love Our Customers!

Whether you are into Valentine’s Day or not, we love you! That’s why we are offering a 50% discount to all new subscriptions (Starter, Professional) as well as to all subscription renewals (Starter, Professional) purchased on Valentine’s Day and the whole weekend after! <div>
</div><div>We want you to spend time with the ones you love and care about, and not worry about missing this limited offer. As we are offering our services worldwide, we also don’t want to annoy anyone with timezone differences. <div>
</div><div>That’s why this offer is valid from Friday February 14th (UTC-12) until Sunday February 16th (UTC+12). No matter what timezone you’re in, you have the whole 4 days :-)
<div>
</div><div>Checkout the subscription plans and share the love! <iframe title="Twitter Tweet Button" class="twitter-share-button twitter-tweet-button twitter-count-horizontal" id="twitter-widget-0" src="http://platform.twitter.com/widgets/tweet_button.1392079123.html#_=1392199030249&count=horizontal&id=twitter-widget-0&lang=en&original_referer=http%3A%2F%2Fblog.myget.org%2Fadmin%2Feditor%2Fpost.cshtml%3Fid%3D6da850db-b991-4f2c-a9f1-f26913483f8d&size=m&text=50%25%20off%20for%20Valentine's%20Day!%20Share%20the%20love%20at%20www.myget.org%20%23nuget&url=http%3A%2F%2Fwww.myget.org" frameborder="0" scrolling="no" style="width: 107px; height: 20px;" allowtransparency="true" data-twttr-rendered="true"></iframe> </div><div>
</div><div>Happy packaging!
<div>
</div></div></div></div>

Author: Maarten Balliauw on 14 Feb 2014

Default feed endpoint issues 7 February 2013 - Post mortem

On February 7, 2013, we've experienced a partial downtime. The default endpoint for consuming NuGet feeds was slow to respond (+/- 2 minutes for a simple request) and caused issues with a lot of our users and their development teams and processes. We share your pain: we use MyGet for developing MyGet and we hate when things like this happen as much as you do. Our sincerest apologies for the inconvenience caused!

In this post, we'll have a look at what happened and the reason this happened. But before we dive in: we've had 3 hours of partial downtime on the default feed endpoint which most users have configured. We want to apologize by extending all paid subscriptions (Starter, Professional and Enterprise) with one additional month. No action is required on your end, your subscription has already been extended.

What happened?

At around 3:15 PM (CET), our monitoring alerted us about latency on the default feed endpoint going up. After a quick investigation and some additional time, latency went back to normal. One hour later, around 4:15 PM (CET), we saw the same happening again, as well as a number of support e-mails coming in.

MyGet has the following feed endpoints available (see documentation):

  • /F/<your_feed_name> - the NuGet v2 API endpoint
  • /F/<your_feed_name>/api/v2 - the NuGet v2 API endpoint for consuming packages
  • /F/<your_feed_name>/api/v2/package - the NuGet v2 API endpoint for pushing packages
  • /F/<your_feed_name>/api/v1 - the NuGet v1 API endpoint for consuming and pushing packages (still in use by Orchard CMS and some others)
  • </ul>

    In this case, the default endpoint we offer our users was experiencing issues, the other endpoints were not. We decided to reply all support requests with the advice to switch to the v2 feed endpoint to make sure you could continue work. This requires some reconfiguration, but we figured it's better than having very slow access and timeouts connecting to your feeds.

    We checked logs, diagnostics, monitoring reports, read and re-read our code, but could not pinpoint any reason for the default feed endpoint having such high latency. At 5:00 PM (CET) we decided to fail-over to our secondary datacenter. Being a separate environment, we wanted to see if the issue would haoppen there as well. It didn't! So we flipped the DR switch and took our primary datacenter out of the loop to be able to investigate the servers without causing additional issues for our users. This fail-over solved the issue for most of our users, who were able to work with MyGet's default feed endpoint from around 6:00 PM (CET), mind some DNS propagation for some users.

    Digging through more logs, we discovered the issue that caused the default feed endpoint to experience this latency (we'll elaborate in a minute). At 8:00 PM (CET), we rolled out a hotfix to our secondary datacenter to make sure the issue would not resurface there either. We worked on a final fix for the next few hours and at 11:30 PM (CET), we flipped the switch back to our primary datacenter.

    Why was the default feed endpoint so slow?

    As you know, MyGet supports adding upstream feeds and allows to proxy them. This is a very useful feature as one feed can essentially bundle several others, so a developer team only has to configure one and still get packages from multiple.

    MyGet treats all upstream package sources as external feeds, even if an upstream feed is a MyGet feed. The reason for that is security contexts can be diferent and we want to have all security aspects applied at any time. This does mean that a MyGet feed having an upstream MyGet feed as a package source is effectively using the default feed endpoint of this second feed, going oer our internal network through the load balancer. Next to security, this also allows us to fan out queries going to multiple package sources over multiple machines.

    One of our users configured upstream package sources that were referencing themselves: feed A referencing feed B referencing feed A. We prohibit self-referencing feeds, but the scenario this user tried to achieve was unsupported and resulted in a reference loop on the default feed endpoint. The default feed endpoint was bringing itself down by fanning out queries to itself.

    Previous incidents teached us to partition as many things as possible, and luckily we partition all alternative feed endpoints. This allowed us to direct support requests towards the other feed endpoints so users could keep using our service.

    Solution

    As a quick fix, we disabled the faulting package source so our service could be resumed in a stable fashion. Once that was done, we worked on a permanent solution to this issue. Do we want to block self-referencing feeds? Absolutely, as that makes no sense. Do we want to block adding other MyGet feeds as upstream package sources? No.

    We want to keep supporting having upstream package sources that can be MyGet feeds, so that complex hierarchies of feeds and privileges can be configured. Quite a number of feeds do this today, having one feed proxying a series of upstream feeds.

    How do we prevent the same problem from happening again? We are now actively tracking refering feeds and detecting loops in these references. Doing this, we keep supporting all scenarios that were possible before while protecting ourselves from shooting ourselves in the foot. Feed A can reference feed B, which can reference feed A. Users can make use of feed A as the entry pont, as well as feed B, and get all expected packages on the feed. We will make sure reference loops are removed from the feed hierarchy.

    When a MyGet feed references a non-MyGet feed, we will also be adding an HTTP header to all upstream requests, X-NuGet-Feedchain, which contains the full list of refering feeds. Other NuGet feed implementations can make use of this header to detect reference loops on their end as well.

    While we replied to all support requests in under 5 minutes with a workaround to use one of the other endpoints, we will be working on extending our status page at http://status.myget.org, showing details about all endpoints. We also want to be able to post status messages and tips & tricks like switching to the secondary feed endpoints on there.

    Hardware fails, software has bugs and people make mistakes. We strive to mitigate as many of these factors and maintain a high quality service with fast, good support. We'll keep doing that to provide you with the best experience possible.

    Again, we do apologize for the inconvenience caused.

    Happy packaging!

Author: Maarten Balliauw on 10 Feb 2014