MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

Learning NuGet Semantic Version Ranges with SemVer Explorer

When authoring NuGet packages, you can declare package dependency versions using Semantic Versioning. NuGet allows specifying dependencies as floating ranges, using interval notation or using fixed version numbers, as explained in the NuGet docs.

MyGet SemVer Explorer allows you to specify a SemVer dependency range, and will check the target package repository for the package versions that match.

NuGet dependency range explorer

Version ranges can be simple (e.g. 6.1.* to match all packages >= 6.1.0) or more complex using interval notation (e.g. (8.0,9.0.1) to match versions that are between 8.0 and 9.0.1. SemVer explorer lets you try these ranges and see which versions of an actual package match. Once satisfied, version ranges can be used in a packages.config or project.json document for use with NuGet or the .NET Core command line.

Can I target MyGet feeds?

Definitely! By default, the tool is configured to query the v3 NuGet.org repository at https://api.nuget.org/v3/index.json. You can simply change the target feed URL to the v3 NuGet feed of a MyGet repository you have access to, and we'll query that one instead.

Can I target private MyGet feeds?

If you have an access token that grants you read-access to the MyGet repository, you can leverage MyGet's support for pre-authenticated feed URLs. Make sure you target the pre-authenticated v3 NuGet endpoint. See our documentation for further guidance.

Have fun exploring the various semantic version constraints NuGet provides! And happy packaging!

MyGet's NuGet and NPM news from the community (September 2016)

We tried it last month, and feedback was good. That’s why we have a second edition of our NuGet and NPM community news from the past few weeks. In this post, we bring you some interesting blog posts and articles, curated by our MyGet founders Xavier and Maarten. Follow @MyGetTeam on Twitter for more!

NuGet news

NuGet news, curated by MyGetThe NuGet team released a new documentation site, with new quick-start tutorials and end-to-end scenarios. A nice improvement from the old docs, check it out!

The folks at Cake started a blog series on which services they are using and for what purpose. We're honored that their first post is titled "How does Cake use MyGet?".

Nick Randolph blogged "NetStandard, what is it and why do I care?" - a nice and easy digestible post linking to Oren Novotny's more elaborate Portable- is dead, long live NetStandard.

Cori Drew mentioned searching for "nuget kitten dies puppy". Still using msbuild package restore? That is a great search indeed! If you haven’t done yet, learn about switching to proper NuGet package restore.

Using Azure Automation? Tao Yang wrote a blog post demonstrating how to Script Azure Automation Module Imports Directly from MyGet or PowerShell Gallery, re-using components in automation workflows.

The Dotnet Watch Tool is covered in a blog post by Muhammad Rehan Saeed. He demonstrates using it to shorten the feedback loop while developing, by automatically loading changed source files without having to rebuild the entire project.

David Fowler is experimenting with "channels" (or "zero copy streams"), making the good old Stream object in .NET obsolete. He released a preview feed on MyGet, where you can experiment with Channels. David posted some samples as well.

Sitecore CMS now supports NuGet for distributing Sitecore packages. They wrote an extensive FAQ on how to work with their feeds and how to install packages into your web application. And even nicer: they are hosted on MyGet. Thanks guys!

The new Windows Management Framework (WMF) 5.1 added OneGet support for basic authentication against secured package feeds, as well as proxy support. That's pretty neat, as you can now distribute custom PowerShell modules using private feeds.

NPM news

NPM news, curated by MyGetNpm 2.15.11 and 3.10.8 have been released. The version 2 branch does not seem to have any noteworthy changes apart from some dependency updates. The version 3 branch got some updates to npm shrinkwrap, and some bugfixes.

TypeScript 2.0 was released with new features like additional types, optional parameters, expression operators, ... We quite like the way TypeScript makes JavaScript more type safe, and the language itself is close to the language we use to build MyGet, C#.

Tierney Coren wrote 11 Simple npm Tricks That Will Knock Your Wombat Socks Off. In this post, he demonstrates some of the lesser used but really helpful commands npm offers, like opening a package's GitHub repo in the browser. Or automating _npm init_ with useful defaults. And 9 more of those!

Ashley G. Williams has presented A Brief History, a great presentation on modular design. What goes into a module? How do you decide? Tip: it's not about what goes in modules, it's how we compose them all together.

Interested in Streams and Async / Await in Nodejs? Paul Cowan uses Babel to transpile asynchronous, non-blocking code into JavaScript using the async and await keywords that are transpiled into promises.

“This” is not always “this”. Peleke Sengstacke wrote about how scope works in JavaScript in his Grokking Scope in JavaScript.

Tim Severien wrote a tutorial on using ESLint to monitor code quality and detect common code issues, resulting in higher quality code. A nice, thorough explanation on how to set up ESLint and use it.

Let’s see if we can do this type of post next month as well. If you have any news to share or have other feedback, let us know using the comments below or reach out on Twitter.

Happy packaging!

Building NuGet and npm using Atlassian Bitbucket Pipelines

Bitbucket Pipelines is a new continuous integration service (still in beta) from Atlassian, built into Bitbucket. Let’s have a look at how we can use Bitbucket pipelines to build, package and publish a .NET Core library to MyGet so we can consume it in our own projects.

How does Bitbucket pipelines work?

To configure a build on Bitbucket, we’ll need a bitbucket_pipelines.yml file which describes the steps to execute as part of the pipeline. Whenever a commit is made to our source repository on Bitbucket, whether git or Mercurial based, a Docker image is started in which our pipeline will be executed.

Here’s a full write-up on how a .NET Core build would work.

How to package and publish a NuGet package to MyGet?

First of all, we’ll need a bitbucket_pipelines.yml file which loads a .NET Core-enabled Docker image. The pipeline itself will have to run package restore, compile the source code, optionally run tests, then package the library and publish it to our MyGet feed.

We have created a sample library at https://bitbucket.org/myget/sample-pipelines-dotnet/, from which the bitbucket_pipelines.yml file can be copied into your own project. A few environment variables need to be configured for the pipeline (see the header of the bitbucket_pipelines.yml file) to make sure it can publish to our MyGet feed.

Once the pipeline completes, we can look at the build output and find the resulting NuGet package on our MyGet feed. The full build output is available as well.

image

How to package and publish an npm package to MyGet?

First of all, we’ll need a bitbucket_pipelines.yml file which loads a Docker image which has node and npm installed. The pipeline itself will have to run npm install, optionally run tests, then package the library and publish it to our MyGet feed.

We have created a sample library at https://bitbucket.org/myget/sample-pipelines-npm/, from which the bitbucket_pipelines.yml file can be copied into your own project. The header of this file lists a few environment variables that have to be configured for the Bitbucket pipeline. When run completes, we can consult the build output:

Publishing npm from BitBucket

Happy packaging!

MyGet's NuGet and NPM news from the community

Many are returning from summer vacation, others have been enjoying the tranquility of summer holiday. Whichever side you’re on, we at MyGet have been watching the NuGet and NPM community news in the past few weeks. In this post, we bring you some interesting blog posts and articles, curated by our MyGet founders Xavier and Maarten. Follow @MyGetTeam on Twitter for more!

NuGet news

NuGet news, curated by MyGetOn the NuGet blog, the NuGet client 3.5 RC has been announced, with support for new target frameworks and lots of performance improvements. Additionally, the NuGet team started working on better documentation, now available as a preview on http://docspreview.nuget.org.

More from the NuGet team: they made some changes to the expiring API keys policy. At MyGet we’ve always made this opt-in, and the NuGet.org gallery will now do the same.

New to NuGet? Rohit Chopra has you covered with his article “NuGet – A Powerful way to share your code”. While focused on NuGet, it’s a nice summary of why you want to use a package manager in your projects. Xiao Ling has a step-by-step post on creating and publishing .NET Core packages.

Building things in Unity? Wondering what NuGet is? Ashley Davis has you covered with his introduction to using Unity and NuGet. The Unity solution templates don’t easily allow working with NuGet, but there are some easy workarounds. A good example is demonstrated, installing JSON.NET into a Unity project.

Have you been consuming NuGet, and just started looking into creating your own NuGet packages to share them with team mates or with the world? Learn about publishing your first .NET Core NuGet package with AppVeyor and MyGet  - Andrew Lock gives a good step-by-step tutorial on what you need in code, and how AppVeyor and MyGet can be used to build and distribute your code.

On a similar topic, Maarten Balliauw has a post on Building NuGet (.NET Core) using Atlassian Bitbucket Pipelines. Pipelines is Atalassian’s continuous integration service that runs on Docker and Linux. And since .NET Core is a first class citizen on that platform, why not use it to build and test NuGet packages?

NPM news

NPM news, curated by MyGetLet’s start on the tooling side. Node has gotten two new releases, 4.5.0 and 6.4.0. Mostly bugfixes, better profiling support and improvements in objects and function contexts for debuggers. On the npm side, there’s now 2.15.10 and 3.10.7, with improvements to how scoped dependencies are handled and several other bugfixes.

Did you know the two millionth package version was just published to npm? If you have as well, congratulations! This is a pretty epic milestone in the Node.js community.

Laurie Voss, COO at npm, has a great talk titled “Abstractions, npm past, present, future”. It covers what is npm and where it came from, where the ecosystem stands today and what the plans are for the future. Highly recommended!

New to node? Have a look at Node Hero’s blog post series! These thirteen articles cover everything from getting started with node and npm, to building a web app, security, monitoring and all other aspects of building a node application.

Npmjs.org added web hook support a while back. Julian Gruber did a proof-of-concept where updated dependencies are automatically deployed in the application. Not the best idea, given that your deployment may break because of an updated dependency, but still quite cool. Package update? Deploy!

Into the Internet of Things? One such thing is the International Space Station! Dave Johnson has a nice post Node.js IoT: Tracking the ISS through the Sky where he uses JavaScript to capture GPS coordinates from the IIS and compares it to your home location to create a real-time tracker.

We’re thinking about doing this type of post each month. Let us know if you’d like that or not, using the comments below or reach out on Twitter.

Happy packaging!

MyGet by the numbers

As it is a slower time of year and many people are taking some vacation, we decided it would be a nice time to collect some numbers around MyGet. Of course we can’t share all of our statistics, but it’s always fun to look at them. MyGet has a shared tenant, www.myget.org, hosting and serving NuGet, npm, Bower and VSIX packages. And debugger symbols, of course. This shared tenant will be the focus of this post. Our MyGet Enterprise plans, which run on their own dedicated tenant infrastructure, are not included in these statistics.

Number of machines

Our web application is hosted on Microsoft Azure across three different regions. The shared tenant has its primary location in the West Europe region, and a secondary in US East. Both locations run 3 front-end web server machines and 2 back-end machines for handling anything that doesn’t have to happen real-time. These machines are all general purpose “A3” machines. We have a tertiary deployment in US West for some Enterprise customers, where a similar deployment lives.

Since we use auto-scaling based on time of day, we don’t have the same amount of machines throughout the day. On average, we run 10 machines for our front-end.

Another deployment lives in a private datacenter and hosts part of our build services. We also run our CI systems in this private datacenter. Azure is great, but also super expensive for some workloads. Hosting this workload ourselves reduces the operational cost for build services with a factor 6. This infrastructure is all SSD based, and we’re getting loads more horsepower out of the VM’s running here. To give an example: the Visual Studio 2015 Update 3 install ran for 4 hours on our Azure build agents, the private VM’s did the update in half an hour.

Number of packages and downloads

Our shared tenant currently hosts over 1 million unique packages. These packages have been downloaded 17,305,790,397 times so far. Yes, that’s over 17 billion. We handle roughly 400 million downloads each month. Some of our Enterprise customers are pushing similar numbers on their instances.

If we look at package types, roughly 50% of packages are NuGet and symbols packages, including Chocolatey and Octopus Deploy. 30% of packages are npm, with Bower and VSIX following at around 10% each.

Storage, traffic and bandwidth

Right now, our shared MyGet tenant consumes 3.2 TB of storage for just packages and metadata, including backups in secondary Azure regions.

Each month, users of www.myget.org push 2.4 TB of data into our system. Many feeds contain CI packages and retain only the latest 100 nightly builds. Other feeds contain 100 MB of Octopus Deploy artifacts, being updated daily. On the consumption side, we have 8 TB of traffic being consumed each month, spread across our Azure bill and the Switzerland-based KeyCDN.

We used to run off the Azure CDN, but found that KeyCDN has been more reliable at half the price. We still have the Azure CDN configured so we can fail-over in case it’s needed. So far, this hasn’t been required. Swiss efficiency #ftw!

Builds

When we first introduced build services, we envisioned making building NuGet (and later npm) packages as easy as possible. We have a number of open source projects running their builds on MyGet, as well as a number of companies enjoying the convention-based builds.

MyGet Build Services has produced over 65,000 successful builds (producing packages) so far. On average, these builds take 2 minutes to complete, with some people maxing out the 30 minute build time we allow.

Support requests

Ask any MyGet user and they will confirm we offer an excellent and speedy support. We typically manage to respond within the hour, unless we’re asleep. Which happens for a good 7 hours each day when night falls over the CET time zone. Our fastest response  so far providing a solution to the customer was 38 seconds.

We handle a good 45 support cases each day. Some of those are simple: we can often respond with a link to documentation. Others are more complicated and challenging. And if we can, we submit a PR to your project to resolve any code or configuration issues.

The busiest day of the week? Thursday. By far. It seems that is the day where everybody realizes the week is over halfway and things must be shipped. Unexpected to us, holidays are also quite busy. It seems many developers sneak in a few hours of coding time on those days and often want us to be part of that.

If we look at technologies, 80% of questions or client configuration problems are related to NuGet. 9% of requests are related to npm. And less than 1% for Bower and VSIX. What with the other 10%? These are typically orientation questions to see if MyGet is a good fit, or if MyGet fits in the customer’s architecture or development practice.

When we started this adventure, we never thought these numbers would become true. Thanks for making this happen and being a big part of this!

Happy packaging!

Using build services to create Chocolatey packages

Chocolatey is a Machine Package Manager, somewhat like apt-get, built with Windows in mind. It lets us install software onto our machine, supports updates and dependencies, much like NuGet or npm do. MyGet has always supported feeds containing Chocolatey packages, making it easy to distribute software packages across teams or with customers. In this post, we’ll show you a trick on how to build Chocolatey packages using MyGet build services. It’s the least we can do as a Belgian company – our country is known for chocolates after all…

MyGet Build Services has a convention-type build approach that will create NuGet, npm and VSIX packages whenever required files or project types are available. By adding a build.cmd or build.ps1 file, this convention can be overridden – just the thing we want to do to create Chocolatey packages.

Using a little bit of PowerShell, we can call into Chocolatey’s choco.exe which handles packaging and verification. The following can be copy/pasted in a build.ps1 file in the root of a GitHub, BitBucket or VSTS repository:

Write-Host "Building Chocolatey packages..." $nuspecs = Get-ChildItem -Path $PSScriptRoot -Filter *.nuspec -Recurse foreach ($nuspec in $nuspecs) { choco pack $nuspec.FullName } $artifactsFolder = "./artifacts" Remove-Item -Path $artifactsFolder -Force -Recurse -ErrorAction SilentlyContinue New-Item $artifactsFolder -Force -Type Directory | Out-Null Move-Item *.nupkg $artifactsFolder Write-Host "Finished building Chocolatey packages."

Once a build is triggered on MyGet, this script will execute and create (and upload) Chocolatey packages to our MyGet feed, which we can then install on our system.

Happy packaging!

MyGet supports Chocolatey

Chocolatey is something you need if you've ever installed, upgraded, or removed software on Windows. It is an existing, proven, almost 4 year old project. For those familiar with *nix package managers, it is a binary package manager, sort of like yum or apt-get, but for Windows. 

Here's how the team describes the tool on their Kickstarter campaign:

Chocolatey is a tool that automates all the mundane getting and installing software work for you. You just select what you want installed and within a few minutes, Chocolatey has downloaded and installed (or upgraded) that software without need for further input from you. So while Chocolatey does the hard work, you can go get some coffee. Or sleep. Or do other more important things. 

You probably start to see why we like Chocolatey!

Chocolatey has been part of the NuGet ecosystem since forever, and we are happy Chocolatey users ourselves. From the very first beta of MyGet's package sources functionality, we have had built-in support for ChocolateyIn fact: if you're using MyGet Build Services, then you are already using Chocolatey! That's right: all the tools mentioned in our Build Services docs are provisioned on our build agents using Chocolatey!

We also know some of our users are using MyGet to host their own Chocolatey repository, which in turn is just another NuGet feed. It just serves Chocolatey packages instead.

It goes without saying that we are huge fans of the Chocolatey project and we really want to encourage and support the Chocolatey team to take the project to the next level. If you're like us and love Chocolatey, we'd like to invite you to join us in backing the Chocolatey Kickstarter project! MyGet has pledged $750 to help move the project from experiment to experience.


Chocolatey has proven to be a huge time saver and invaluable tool for us, and if you feel the same, the least you can do is to get yourself a nice box of chocolates ;-)

Let's get Chocolatey!

Implementing custom package retention using webhooks

Earlier this week, we got the question if custom retention policies could be enforced on MyGet. More specific, the request was to be able to keep the latest 5 versions of a minor version (e.g. keep 1.0.6 through 1.0.2, but delete 1.0.1 and 1.0.0). We’ve introduced webhooks on MyGet to enable exactly this sort of scenarios!

Our own retention policies run whenever a package is added to a feed, so let’s see if we can implement a webhook handler that does exactly what was asked… The code for this blog post can be found in our GitHub organization.

Building the webhook handler

We’ll first need something that can run our custom logic whenever a webhook event is raised. This can be an ASP.NET MVC, Web API, NancyFx or even a PHP application. In this case, let’s go with an ASP.NET Web API controller. We want to be triggered on POST when a package added event is raised.

// POST /api/retention public async Task<HttpResponseMessage> Post([FromBody]WebHookEvent payload) { // The logic in this method will do the following: // 1) Find all packages with the same identifier as the package that was added to the originating feed // 2) Enforce the following policy: only the 5 latest (stable) packages matching the same minor version may remain on the feed. Others should be removed. string feedUrl = payload.Payload.FeedUrl; // Note: the following modifies NuGet's client so that we authenticate every request using the API key. // If credentials (e.g. username/password) are preferred, set the NuGet.HttpClient.DefaultCredentialProvider instead. PackageRepositoryFactory.Default.HttpClientFactory = uri => { var client = new NuGet.HttpClient(uri); client.SendingRequest += (sender, args) => { args.Request.Headers.Add("X-NuGet-ApiKey", ConfigurationManager.AppSettings["Retention:NuGetFeedApiKey"]); }; return client; }; // Prepare HttpClient (non-NuGet) var httpClient = new HttpClient(); httpClient.DefaultRequestHeaders.Add("X-NuGet-ApiKey", ConfigurationManager.AppSettings["Retention:NuGetFeedApiKey"]); // Fetch packages and group them (note: only doing this for stable packages, ignoring prerelease) var packageRepository = PackageRepositoryFactory.Default.CreateRepository(feedUrl); var packages = packageRepository.GetPackages().Where(p => p.Id == payload.Payload.PackageIdentifier).ToList(); foreach (var packageGroup in packages.Where(p => p.IsReleaseVersion()) .GroupBy(p => p.Version.Version.Major + "." + p.Version.Version.Minor)) { foreach (var package in packageGroup.OrderByDescending(p => p.Version).Skip(5)) { await httpClient.DeleteAsync(string.Format("{0}api/v2/package/{1}/{2}?hardDelete=true", feedUrl, package.Id, package.Version)); } } return new HttpResponseMessage(HttpStatusCode.OK) { ReasonPhrase = "Custom retention policy applied." }; }

Once we have this in place and are hosting it somewhere, we can configure the webhook on our MyGet feed.

Configuring the webhook

On our MyGet feed, we can create a new webhook. It should send application/json for the package added event to the URL where we deployed the above code.

Configure web hook

When this hook now triggers, we will be retaining just the 5 latest minor versions of a package (ignoring prereleases).

That’s it. Using nothing but webhooks, we can run our own retention policies (or other logic) when something happens on our feed (like strong-name signing packages, for example). There are a number of events that we can subscribe to!

Happy packaging!

Using MyGet as a OneGet package source

At the Build conference, Microsoft announced the Windows Management Framework 5.0 Preview which includes Windows PowerShell 5.0, updates to the PowerShell ISE, Network Switch Cmdlets and ... OneGet!

What is OneGet?

OneGet a unified package management interface component with a set of managed and native APIs, a set of PowerShell cmdlets, and a WMI provider. The component accepts both Microsoft-provided and 3rd party-provided plugins which extend the functionality for a given package type.

The OneGet team also has a weekly community meeting of which you can see the first introductionary recording below.

As part of this Preview, OneGet is shipping with a prototype plugin compatible with Chocolatey, the so called ChocolateyProvider. This is a prototype implementation of a Chocolatey-compatible package manager that can install existing Chocolatey packages. This is a clear confirmation for the hard work done by the Chocolatey folks, and both systems will continue to evolve together, as Rob Reynolds explains in this post. If you want to follow-up on OneGet, then check out its GitHub repository and follow PSOneGet on Twitter.

Something about a forest and trees...

NuGet, MyGet, Chocolatey, OneGet... what?! People ask questions and occasionally can't see the forest for the trees. Here's a quick recap:

  • NuGet: a solution-level package management tool, used to manage software dependencies within the scope of a solution. It is accompanied by the NuGet Gallery, the home of many if not all .NET open source components.
  • Chocolatey: a system-level package management tool, used to manage software installations on a Windows system. It (currently) leverages PowerShell and NuGet, supports the Web Platform Installer (WebPI), MSI, RubyGems and many more, and is accompanied by the Chocolatey Gallery where you can find many popular software packages. Rob describes Chocolatey as somewhat like "apt-get", but with Windows in mind.
  • MyGet: a hosted NuGet package server where you can create and secure your own feeds. In essence, MyGet is able to host vanilla NuGet feeds, as well as Chocolatey feeds.
  • OneGet: a a unified interface to package management systems (see above)

So what does this mean? How do these package managers play along?

OneGet supports multiple package sources, and as stated earlier, OneGet comes with a ChocolateyProvider. As MyGet also supports Chocolatey feeds, this effectively means that you can register a MyGet feed as a Chocolatey package source in OneGet! The blow diagram is an attempt to illustrate how they relate:

How do OneGet, Chocolatey, NuGet and MyGet play along?

OneGet supports several commands at this stage:

OneGet Preview cmdlets

How can I use a private OneGet package source?

So how can I register a private OneGet package source? Well, let's first see how you can register any package source using the Add-PackageSource cmdlet. Here's what the built-in help currently says:

OneGet Add-PackageSource Help

Note that this is a Preview: help is incomplete and cmdlets might change name, but this should already give you a good idea of what you can do with this cmdlet!

Now, let's register a MyGet feed on which you can host Chocolatey packages:

Register a MyGet feed as a OneGet package source

Did you notice how OneGet asked you to install the NuGet package manager?

That went easy right? That's because that feed was public :) OneGet does not support basic-authentication at this point, nor does it leverage any nuget.config settings you might configure (tried it). However, MyGet just added the possibility to use a "private-by-obscurity" endpoint on private feeds, which should allow you to use private feeds as well. Note: we don't actively promote this, as it requires you to share one of your feed's access tokens. This is a work-around for clients that don't support the basic-auth flow, and we'd prefer to have proper basic-authentication support in OneGet, so fingers crossed!

You can verify the correct registration of your OneGet package source using the following commands:

Get OneGet package sources List available packages on a OneGet package source

Installing a software package from this MyGet feed is straightforward as well:

Install a software package from a specific OneGet feed

This flow allows you to control what packages get distributed through OneGet, avoids the need to publish your internal software to the general public, and still allows you to leverage the great new scenarios that OneGet offers!

As usual, happy packaging! :)

Join the Global Windows Azure Bootcamp 2014

Global Windows Azure BootcampIt’s no secret that MyGet is running on top of the Windows Azure cloud platform. Because we love it and want to get as many people to learn about it, we’re participating again in this year’s Global Windows Azure Bootcamp.

In April of 2013 the first Global Windows Azure Bootcamp was held at more than 90 locations around the globe. This year’s bootcamp will again offer a one day deep dive class to help thousands of people get up to speed on developing Cloud Computing Applications for Windows Azure. In addition to this great learning opportunity there will be a hands on lab in which everyone can participate. A huge global compute farm will be created by attendees to perform diabetes research!

If you want to learn about Windows Azure or help in diabetes research, find a location near you and join this massive event on Saturday, March 29, 2014. MyGet is giving away a free 2-month Starter subscription to every attendee, and other sponsors are offering swag and licenses as well!

Happy packaging in the cloud!