MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

Building NuGet and npm using Atlassian Bitbucket Pipelines

Bitbucket Pipelines is a new continuous integration service (still in beta) from Atlassian, built into Bitbucket. Let’s have a look at how we can use Bitbucket pipelines to build, package and publish a .NET Core library to MyGet so we can consume it in our own projects.

How does Bitbucket pipelines work?

To configure a build on Bitbucket, we’ll need a bitbucket_pipelines.yml file which describes the steps to execute as part of the pipeline. Whenever a commit is made to our source repository on Bitbucket, whether git or Mercurial based, a Docker image is started in which our pipeline will be executed.

Here’s a full write-up on how a .NET Core build would work.

How to package and publish a NuGet package to MyGet?

First of all, we’ll need a bitbucket_pipelines.yml file which loads a .NET Core-enabled Docker image. The pipeline itself will have to run package restore, compile the source code, optionally run tests, then package the library and publish it to our MyGet feed.

We have created a sample library at https://bitbucket.org/myget/sample-pipelines-dotnet/, from which the bitbucket_pipelines.yml file can be copied into your own project. A few environment variables need to be configured for the pipeline (see the header of the bitbucket_pipelines.yml file) to make sure it can publish to our MyGet feed.

Once the pipeline completes, we can look at the build output and find the resulting NuGet package on our MyGet feed. The full build output is available as well.

image

How to package and publish an npm package to MyGet?

First of all, we’ll need a bitbucket_pipelines.yml file which loads a Docker image which has node and npm installed. The pipeline itself will have to run npm install, optionally run tests, then package the library and publish it to our MyGet feed.

We have created a sample library at https://bitbucket.org/myget/sample-pipelines-npm/, from which the bitbucket_pipelines.yml file can be copied into your own project. The header of this file lists a few environment variables that have to be configured for the Bitbucket pipeline. When run completes, we can consult the build output:

Publishing npm from BitBucket

Happy packaging!

MyGet's NuGet and NPM news from the community

Many are returning from summer vacation, others have been enjoying the tranquility of summer holiday. Whichever side you’re on, we at MyGet have been watching the NuGet and NPM community news in the past few weeks. In this post, we bring you some interesting blog posts and articles, curated by our MyGet founders Xavier and Maarten. Follow @MyGetTeam on Twitter for more!

NuGet news

NuGet news, curated by MyGetOn the NuGet blog, the NuGet client 3.5 RC has been announced, with support for new target frameworks and lots of performance improvements. Additionally, the NuGet team started working on better documentation, now available as a preview on http://docspreview.nuget.org.

More from the NuGet team: they made some changes to the expiring API keys policy. At MyGet we’ve always made this opt-in, and the NuGet.org gallery will now do the same.

New to NuGet? Rohit Chopra has you covered with his article “NuGet – A Powerful way to share your code”. While focused on NuGet, it’s a nice summary of why you want to use a package manager in your projects. Xiao Ling has a step-by-step post on creating and publishing .NET Core packages.

Building things in Unity? Wondering what NuGet is? Ashley Davis has you covered with his introduction to using Unity and NuGet. The Unity solution templates don’t easily allow working with NuGet, but there are some easy workarounds. A good example is demonstrated, installing JSON.NET into a Unity project.

Have you been consuming NuGet, and just started looking into creating your own NuGet packages to share them with team mates or with the world? Learn about publishing your first .NET Core NuGet package with AppVeyor and MyGet  - Andrew Lock gives a good step-by-step tutorial on what you need in code, and how AppVeyor and MyGet can be used to build and distribute your code.

On a similar topic, Maarten Balliauw has a post on Building NuGet (.NET Core) using Atlassian Bitbucket Pipelines. Pipelines is Atalassian’s continuous integration service that runs on Docker and Linux. And since .NET Core is a first class citizen on that platform, why not use it to build and test NuGet packages?

NPM news

NPM news, curated by MyGetLet’s start on the tooling side. Node has gotten two new releases, 4.5.0 and 6.4.0. Mostly bugfixes, better profiling support and improvements in objects and function contexts for debuggers. On the npm side, there’s now 2.15.10 and 3.10.7, with improvements to how scoped dependencies are handled and several other bugfixes.

Did you know the two millionth package version was just published to npm? If you have as well, congratulations! This is a pretty epic milestone in the Node.js community.

Laurie Voss, COO at npm, has a great talk titled “Abstractions, npm past, present, future”. It covers what is npm and where it came from, where the ecosystem stands today and what the plans are for the future. Highly recommended!

New to node? Have a look at Node Hero’s blog post series! These thirteen articles cover everything from getting started with node and npm, to building a web app, security, monitoring and all other aspects of building a node application.

Npmjs.org added web hook support a while back. Julian Gruber did a proof-of-concept where updated dependencies are automatically deployed in the application. Not the best idea, given that your deployment may break because of an updated dependency, but still quite cool. Package update? Deploy!

Into the Internet of Things? One such thing is the International Space Station! Dave Johnson has a nice post Node.js IoT: Tracking the ISS through the Sky where he uses JavaScript to capture GPS coordinates from the IIS and compares it to your home location to create a real-time tracker.

We’re thinking about doing this type of post each month. Let us know if you’d like that or not, using the comments below or reach out on Twitter.

Happy packaging!

Keeping feeds clean with retention rules

MyGet Package Retention Rules help clean up your NuGet npm feedMany developer teams use MyGet for storing their continuous integration and/or nightly builds of NuGet, npm, Bower and VSIX packages. As more and more packages get added, it may become harder to manage them all. Some packages may be used in projects, while others are not. Let’s go over the options available for housekeeping.

By default, MyGet keeps all package versions available on our feeds. Every package pushed is there forever, unless manually removed or removed by package retention. By setting retention rules, it is possible to automatically trim the list of packages to X latest packages, keeping into account package usage in projects and package dependency trees.

Configuring retention rules

Retention rules are defined per feed. Some feeds may have more aggressive retention rules defined, other may not have them enabled at all. From the Retention Rules, we can define:

  • the maximum number of stable versions to keep
  • the maximum number of prerelease versions to keep
  • whether to keep depended packages or not – enabling this makes sure package restores always complete successfully by keeping the dependency tree in its entirety
  • whether to allow removal of packages that have downloads – enabling this option ensures that packages that are being used in projects never get deleted

Setting retention rules

Keeping a specific package around

Retention rules are quite brute-force: they will always remove all packages that match the configured rules. Luckily, MyGet lets us “pin” packages which we want to keep around. For example, we may want to only keep the latest 10 pre-release versions while still keeping around the 20th pre-release version we’re still using in our projects.

From the package details page, we can define which package versions should never be considered by retention rules by using the Pin button next to the package.

Pinning packages so they do not get removed

We can pin package per version, or all versions at once using the button at the top of the Package History list. Of course, we can also Unpin packages using the same approach. Once a package is unpinned, retention rules are allowed to remove them.

Custom retention rules using web hooks

Using the built-in retention rules may not be enough. For example, what if we want to run retention rules based on other conditions than the latest version? What if we want to only remove packages when there is a full moon? Using web hooks, we can subscribe to certain feed events (like “package added”) and run our custom logic to optionally remove packages from our feed. We have a complete example available that helps getting started with web hooks.

Learn more about package retention in our documentation.

Happy packaging!

Using build services to create Chocolatey packages

Chocolatey is a Machine Package Manager, somewhat like apt-get, built with Windows in mind. It lets us install software onto our machine, supports updates and dependencies, much like NuGet or npm do. MyGet has always supported feeds containing Chocolatey packages, making it easy to distribute software packages across teams or with customers. In this post, we’ll show you a trick on how to build Chocolatey packages using MyGet build services. It’s the least we can do as a Belgian company – our country is known for chocolates after all…

MyGet Build Services has a convention-type build approach that will create NuGet, npm and VSIX packages whenever required files or project types are available. By adding a build.cmd or build.ps1 file, this convention can be overridden – just the thing we want to do to create Chocolatey packages.

Using a little bit of PowerShell, we can call into Chocolatey’s choco.exe which handles packaging and verification. The following can be copy/pasted in a build.ps1 file in the root of a GitHub, BitBucket or VSTS repository:

Write-Host "Building Chocolatey packages..." $nuspecs = Get-ChildItem -Path $PSScriptRoot -Filter *.nuspec -Recurse foreach ($nuspec in $nuspecs) { choco pack $nuspec.FullName } $artifactsFolder = "./artifacts" Remove-Item -Path $artifactsFolder -Force -Recurse -ErrorAction SilentlyContinue New-Item $artifactsFolder -Force -Type Directory | Out-Null Move-Item *.nupkg $artifactsFolder Write-Host "Finished building Chocolatey packages."

Once a build is triggered on MyGet, this script will execute and create (and upload) Chocolatey packages to our MyGet feed, which we can then install on our system.

Happy packaging!

Two of my packages are treated as one. Help!

Every once in a while, our support gets a question that is similar to the following:

When requesting the package details for some versions of the NuGet packages in our feed it seems like the service encounters an error preparing the ATOM document, also resulting in NuGet failing to download the package.

Here's an example URL: https://www.myget.org/F/chucknorris/api/v2/Packages(Id='MyPackage',Version='1.2.20160209.1-master')

Is something wrong with MyGet? Definitely not! Let’s look at the packages for this feed:

  • MyPackage 1.2.20160209.1-master
  • MyPackage 1.2.20160209.1-dev

Can you spot the error? It’s very subtle, so let us help here. The version number has three dots (“.”) in it, which means it should be treated as a major.minor.patch.build type of version number, which is numeric. MyGet takes the hint and will strip off the –master and –dev in the above versions, treating both packages as one.

Clearly, the intention here was to use Semantic Versioning (SemVer): the –master and –dev suffixes to the version number were intended to provide a label in the version number. But semantic versioning is in the form of major.minor.patch-label, which means one dot less in the version number should be used. So in this case, the packages should be re-versioned and re-uploaded with the correct version number.

Don’t you guys have validation for this? is one of the questions we’d expect. We do, but it’s opt-in because many of our users start out with non-SemVer packages on their feeds and we don’t want to block those from being uploaded either. From a feed’s Package Settings tab, enable the Forbid packages which are non-compliant with Semantic Version? option.

Forbid packages which are non-compliant with Semantic Version?

This will prevent malformed semantic versions from being uploaded to your feed and prevent the above error from ever happening to you.

Happy packaging!

Dropbox as a package source for NuGet, npm, Bower and VSIX packages

Wouldn’t it be awesome if creating a NuGet, npm, Bower or VSIX feed was as easy as just copying packages into a Dropbox folder? Awesomeness is here: we’ve added Dropbox as a package source type to MyGet. This allows us to link a Dropbox folder to a MyGet feed and automatically upload packages so they can be consumed with the popular package managers out there.

Synchronizing NuGet packages with Dropbox

The Dropbox package source makes it easy to move packages into MyGet. For example, migrating from a network share to MyGet is as easy as copy-paste. Have a build server that drops artifacts into a Dropbox folder? MyGet will add the synchronized artifacts to your feed. Right now we download packages from Dropbox on a schedule (every 15 minutes).

Give it a try and let us know how it goes – feedback is welcome through the comments below or via the MyGetTeam Twitter account.

Happy packaging!

Improved support for localized satellite NuGet packages

At MyGet, we live and breath a culture of componentization, and we are very happy to see package authors adopting and sharing these practices within their organizations. One of these practices is the usage of NuGet's localized NuGet packages feature. The adoption of localized satellite packages is a good indicator of package author maturity. Using a better practice should be rewarding and free of friction. One of our Dear Users provided us with great feedback on how he leveraged MyGet to release localized packages by pushing them upstream. However, some unnecessary friction was involved, which had to be removed.

Localized NuGet Packages

As a NuGet package author, you can provide a localized experience for your libraries. The NuGet client has supported the creation of localized satellite packages since version 1.8.

For those unfamiliar with the concept, here's a basic package graph demonstrating the idea:

My.Framework.1.0.0-alpha0001.nupkg

  • Meta package, installing My.Framework and all localized satellite packages
  • Depends on: all satellite packages (My.Framework.Core.nl, My.Framework.Core.fr, My.Framework.Core.el, ...)
My.Framework.Core.1.0.0-alpha0001.nupkg
  • Package containing an actual framework library (e.g. in lib/net45/My.Framework.Core.dll)
  • No dependencies for the sake of the demo (your libraries most likely have dependencies :))
My.Framework.Core.nl.1.0.0-alpha0001.nupkg
My.Framework.Core.fr.1.0.0-alpha0001.nupkg
My.Framework.Core.el.1.0.0-alpha0001.nupkg
  • 3 localized satellite packages for the My.Framework.Core package
  • Depends on the actual framework package being localized: My.Framework.Core.1.0.0-alpha0001.nupkg
  • Only contains localization resources (e.g. lib/net45/nl/My.Framework.Core.resources.dll, lib/net45/nl/My.Framework.Core.xml)
Or visualized in a graph, here's the dependency tree:


To facilitate discovery of these satellite packages, we added a convenient Localization tab on the package details page if localized satellite packages are detected.


Releasing localized NuGet packages

Ideally, releasing a package that has localized satellite packages from MyGet to another feed, should be a click of a button away, publishing the entire package-set at once. Optionally, one might also want to adjust the prerelease tag and release notes of this package set, either per package individually, or for all at once.

The Push Upstream feature (also check our docs) has been upgraded to now also detect package dependencies and localized satellite packages!

This means that, as in the above sample scenario, you can now publish the entire localized package-set in one go by pushing the My.Framework meta package upstream.


In addition, you can also edit (or simply remove!) the prerelease tag and/or release notes for each package individually, or for all of them at once!

Simply click the edit button  next to one of the packages, modify the fields, and click the water-drop icon  to apply the changes to all other packages in the dialog.


Of course, you can still exclude individual packages from this list if you want to by clicking the  button next to the respective package.

We love this kind of feedback and hope you find this change useful, so keep the feedback coming!

Happy Packaging!

Introducing debugging, source code and symbols for NuGet packages

Shortly after we launched MyGet, we teamed up with SymbolSource.org to provide support for .NET debugger symbols. Today, we’re happy to announce a second option: integrated MyGet symbols support! MyGet symbols support lets consumers of our NuGet packages step through the source code and integrate with Visual Studio and tools like WinDbg. Additionally, symbols and sources can be consumed from within MyGet as well.

Head over to our docs and learn how to get started with MyGet symbols support!

image

How do I get started?

Documentation is available that will help you get started with MyGet symbols very quickly. Symbols in .NET can be tricky sometimes, so if things don’t work out as expected we’ve also compiled a list of PDB troubleshooting tips.

To make things easy, the endpoint for pushing symbols packages is the same as that for your regular NuGet packages: https://www.myget.org/F/<feedname>/api/v2/package. MyGet will determine the type of package and act accordingly. This makes pushing symbols packages from your build server, like TeamCity or Visual Studio Team Services similar to pushing regular NuGet packages.

Where do my symbols packages go when I use MyGet Build Services?

For new feeds, symbols packages are pushed to your MyGet feed by default. For existing feeds, symbols will be uploaded to SymbolSource (if this was enabled). From the build configuration, you can select where to push symbols packages.

What happens to the SymbolSource integration?

SymbolSource integration is still available. Our customers asked very loudly for integrated symbols support, so we’re delivering on that. We are still providing the option to use SymbolSource with your MyGet feeds. In fact, they have some great new features planned as well!

Head over to our docs and learn how to get started with MyGet symbols support!

Happy packaging! (and efficient debugging, of course)

Package details showing GitHub project README

We’re happy to introduce a few user interface enhancements which have been available for all users of npm private feeds on MyGet. We’ve now rolled these out to the package details page for NuGet, npm, Bower and Vsix packages. These pages now display all “at-a-glance” information on the right. Package owners, authors, license information and downloads can be seen from here.

The wider part of the package details page now displays the README.md contents retrieved from GitHub. That is, if the GitHub repository is accessible for us. This makes it easier for consumers of your feed to see installation instructions, links to documentation and so on from the package details page.

MyGet showing GitHub readme contents on package details page

We're looking forward to hearing your feedback through the comments below. Or tweet us via @MyGetTeam.

Happy packaging!

Preview support for the NuGet V3 protocol

A few weeks back, NuGet 3.0 was released. It’s a redesign of the NuGet client targeted at Visual Studio 2015, with a number of improvements and changes from earlier versions of NuGet. One of the changes the NuGet team made was on the server side, with a new protocol. We’re happy to announce that MyGet now has preview support for this new protocol as well!

The new NuGet server-side API was first written about in April 2014. And even before that, there was a talk Evolution of NuGet at MonkeySpace 2013. In it, the NuGet team describes what the NuGet team envisioned: a faster and more reliable API for .NET developers to consume packages. When opening the NuGet Package Manager Dialog in Visual Studio 2015 this becomes very visible: the well-known OData V2 feeds feel slow. MyGet now supports the NuGet V3 protocol (in preview), sharing the vision the NuGet team laid out over 2 years ago and made happen a few weeks ago: make NuGet package management faster.

Note: make sure to read the entire blog post when using private feeds - the NuGet client that ships with Visual Studio 2015 NuGet only supports pre-authenticated V3 feeds. An update is available with full private feed support.

How to make use of the NuGet V3 protocol with MyGet?

The URL of a MyGet feed typically looks like this: https://www.myget.org/F/feedname/api/v2. For the V3 protocol, this URL becomes https://www.myget.org/F/feedname/api/v3/index.json. This feed can be registered in Visual Studio 2015. From the Tools | NuGet Package Manager | Package Manager Settings menu, we can select the Package Sources tab and add a V3 feed in the same way as a V2 feed can be added. We can give the feed a descriptive name, and enter its URL.

Registering MyGet V3 API NuGet

Once added, we can manage packages for our project or solution, and select the newly added feed from the dropdown.

MyGet feed with NuGet V3 API protocol

Does this also work with private (authenticated) feeds?

It’s perfectly possible to consume private (authenticated) feeds on MyGet using the new NuGet client in Visual Studio 2015. Unfortunately there is a bug in the client that makes this a little hard: if it has to prompt for credentials, consuming the feed will fail. The NuGet team confirmed a fix is ready to be released (in fact, an RC version is out), but in the meanwhile a pre-authenticated feed URL can be used to consume a private V3 feed.

A pre-authenticated feed comes in the following format: https://www.myget.org/F/feedname/auth/accesstoken/api/v3/index.json, where the feedname is of course the name of the feed, and accesstoken is one of the access tokens that can be managed via the MyGet website.

Can upstream feeds be proxied?

The MyGet V3 feeds support proxying upstream V2 feeds (not V3 yet). This allows for registering one V3 feed URL in Visual Studio while proxying one or more external (V2) feeds over the same URL. This greatly speeds up working with NuGet! Have a look at our documentation to learn more about feed proxying.

When browsing MyGet, you may notice the V3 feed URL is not being advertised anywhere. The reason for that is twofold: we’re offering the V3 protocol as a preview. We want to make sure the feed behaves the way it should. Second, we want to continue improving V3 protocol support, for example by enhancing support for proxying upstream feeds with upstream V3 sources.

We would love to hear your feedback via @MyGetTeam or through the comments below.

Happy packaging!