MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

Checking NuGet package vulnerabilities with OWASP SafeNuGet

Edit - October 14, 2016 - We have a new, integrated vulnerability scan service.

A couple of days ago, OWASP released a new NuGet package which is able to check known vulnerabilities in NuGet packages. Use of libraries with known vulnerabilities can be an issue for software and components you create: check the excellent whitepaper "The Unfortunate Reality of Insecure Libraries". In the OWASP Top 10 2013, consuming vulnerable packages is listed under A9 Using Known Vulnerable Components.

There is a simple solution to this: by installing one additional package in your projects, automatic checking for known vulnerabilities can be done. The SafeNuGet package contains an MSBuild task which will warn you about this.

A repository with vulnerable packages and the reason for that can be found on the SafeNuGet GitHub project. When running a build which references vulnerable NuGet packages, the warnings list will contain some information about this as well as a link with some explanation:

Checking package vulnerabilities

And of course when such library is built using MyGet Build Services, a warning will also be displayed in the build log:

MyGet build services security scan

It would be great if the build would fail entirely when such package is found, right? Well, that is a simple configuration parameter for the SafeNuGet package. Find the SafeNuGet.targets file and update its contents to:

<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <UsingTask AssemblyFile="SafeNuGet.dll" TaskName="SafeNuGet.AreNuGetPackagesSafe"  />
  <Target Name="AfterBuild">
    <AreNuGetPackagesSafe ProjectPath="$(MSBuildProjectDirectory)"
         CacheTimeInMinutes="10" DontBreakBuild="false" />
  </Target>
</Project>

Want to make sure known vulnerabilities are shown in your builds? You know the drill:

SafeNuGet

Happy and safe packaging!

Migrate away from MSBuild-based NuGet package restore

Back in the days...

NuGet package restore used to be MSBuild-based. You had to explicitly enable it using the context menu on a Visual Studio solution: right-click the solution and select Enable NuGet Package Restore. In fact, if you go to the NuGet docs, you'll see that this scenario is still fully documented. A quick search for "package restore" will throw this old scenario "in your face", as it is the first hit in the search results.

First hit in search results when looking for Package Restore on the NuGet docs

To be fair, the page does highlight that there's a new way of doing this. But many people don't read. At best some look at the pictures. That's why I won't even include a screenshot of that page, as it is full of project setup details that no one should ever go through again. Instead, I'll give you a clear picture of what you should not do :)

Don't do this!

You're doing it wrong!

I can't stress it enough. I'm a huge proponent of NuGet package restore! But if you follow this workflow, then please do it right! (and design for failure, obviously).

The MSBuild-based NuGet package restore has issues. For one: it's MSBuild-based. This means that anything that happens during package restore is run within the MSBuild process, which is particularly annoying for packages that want to modify project files and inject MSBuild targets (as these aren't picked up until the next run).

The moment you manually enable NuGet package restore through the context menu, you're actually installing a few NuGet packages: NuGet.Build, which depends on NuGet.Commandline. The nuget.exe along with a nuget.config and a nuget.targets file are created within a .nuget folder, and all projects that have NuGet package references will be modified to import the NuGet.targets file. The nuget.targets file ensures that nuget.exe is invoked during builds (as in: not before builds!).

The right way

All you need to do is to make sure that your Visual Studio options allow NuGet to download any missing packages in a pre-build phase (note: even before MSBuild compilation starts!). I'm not going to rephrase step-by-step what you should do as David Ebbo already has a great post explaining all of this!

Ensure NuGet is allowed to download missing packages

If you're cloning a new project that did not commit any NuGet packages (and is not using the old MSBuild-based restore), then it just works!

Migrating from the old way

If you still have a .nuget folder in your repository, then please migrate away from it! Think about all those adorable kittens...

Did you know this has been documented on the NuGet Docs all along?! Follow this how-to and save yourself and everyone using your codebase some trouble and follow it step-by-step.

But... my precious (build server)

Here you go: set this environment variable to true and be done with it.

EnableNuGetPackageRestore=true

The following tools support the new automatic package restore out-of-the-box and Just Work™!

The next list of tools requires some minor modifications to the build process:

  • Visual Studio Online / Team Foundation Server (how-to)
  • TeamCity (how-to)

Note that you don't need to worry about development machines! As long as you all have the latest NuGet Visual Studio extension installed.

Upgrading your NuGet extension is generally a good idea anyway, as there are lots of improvements in the latest versions!

Going forward

Here's what I'd love to see happen going forward:

  • The NuGet Docs should by default show the new non-MSBuild-based package restore instructions. There are close to none, but this should be thrown in your face when looking for it.
  • Migration instructions should be clearly linked to.
  • The old MSBuild-based instructions should be archived, perhaps even removed.
  • The context menu-item to manually enable NuGet package restore (MSBuild-based) should be completely removed from the extension. I don't see any reason to keep it. Do you? If you do, please comment on this CodePlex issue, if you agree, then vote for it :)
  • Preferably, the next NuGet Visual Studio extension detects you are using the "old" restore option when you open a solution, and asks you to migrate/upgrade to the new way. Ideally, this removes the targets and import statements, and custom package sources and credentials are taken into account if they are in the nuget.config file.

I'm happy to take on an issue or send PR's for any of the above, but some of the bullet-points seem too big to me to be taken in as a PR.

MyGet Documentation site redesigned

When we first launched the MyGet Documentation site, we decided to fork the NuGet documentation site and apply our own colors and content to it. After our website redesign a few months ago, we felt it was time to work on our documentation site’s design, too.

Documentation on how to use MyGet

The front page looks completely different. We decided to put a search engine central, as well as some popular articles that can help you get started.

One of the things we want to encourage everyone to do is comment on documentation: explain how you did something, ask questions and get help. If we see there are some things that are not completely clear from these comments, we’ll work on additional documentation there. Therefore, every article now gets a section where you can add your comments.

Add comments to MyGet documentation

Not that we are lazy, but if you feel you can do a better job at an article, spot a typo or want to add something, every article features a direct link to our GitHub repository where you can send us a pull request with changes. And that’s not work you’re doing for free: for every accepted Pull Request, you get a free one month extension of your current subscription.

image

Happy packaging!

Package not found during package restore

When working with your own feed, whether private or public, chances are you want to consume more than just that feed. We see many people using their MyGet feed and the NuGet.org feed simultaneously. Sometimes, an interesting error occurs during package restore.

Unable to find version xxxx of package yyyy

That’s not funny! You know that the package is on the feed as you’ve installed it from there in the first place! The reason this happens is because the NuGet command line, the NuGet Visual Studio Extension and the NuGet PowerShell Console all have a configuration option specifying which package source to install from. When this setting is changed to one specific feed, other feeds will be ignored and the error above will be shown during package restore.

The solution is very simple: you can set the active package source to aggregate in Visual Studio, or simply configure NuGet to always use the aggregate package source for the current project. NuGet has an inheritance system for NuGet.config files, where the NuGet.config file closest to the solution file gets the last say. So in short, if you add the following NuGet.config file next to the solution file for your project, you should be fine:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>
</configuration>

We can take this one step further and instead of configuring your MyGet feed globally for your system (and requiring other devs on your team to do the same), why not distribute a NuGet.config along with the sources?

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>
  <packageSources>
    <add key="nuget.org" value="https://www.nuget.org/api/v2/" />
    <add key="MyGet" value="https://www.myget.org/F/chcuknorris/" />
  </packageSources>
  <disabledPackageSources />
  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>
</configuration>

This really makes working with multiple feeds a breeze. But we can go even further and use only MyGet, proxying packages from NuGet.org along the way. For more info on how that works, check the documentation on upstream package sources.

Happy packaging!

Publishing packages to NuGet.org during build

Ever wanted to push a package to NuGet.org or another feed during a build on MyGet Build Services? Affraid of checking in the API key to source control just to be able to do that? Well here’s a little trick that will help you do that without spilling secrets.

When we implemented support for NuGet Package Restore, we’ve also added support for transfering package source credentials to the build server in a safe way. From the Package Sources tab on your feed, you can use the Add package source button to specify all details about a feed that should be available during build, both for consuming and pushing packages. You can add any feed you want: a Chocolatey feed, a TeamCity feed or another MyGet feed.

NuGet Push in MyGet build

After specifying an API key through MyGet, you can simply push packages during build, from your build.bat. Let’s push all packages in the build’s release folder to NuGet.org:

nuget push release/*

Prefer pushing MyPackage 1.0 to another feed? Add it as a package source in MyGet, specify the API key and push from build.bat:

nuget push MyPackage.1.0.nupkg -Source http://other-feed

Note that the packages generated during build will also be added to your MyGet feed.

Happy packaging!

GitHub Releases and MyGet Build Services

During the summer, GitHub added a great feature for many projects: releases. Projects on GitHub can create a release and label sources, add binaries and release notes. Following the conventions of many Git projects, releases are tied to Git tags. You can use an existing tag, or let releases create the tag when it's published.

Since MyGet Build Services supports labeling and we can label sources when pushing packages upstream, GitHub Releases make a great combination with MyGet! Let’s see.

Labeling Sources from MyGet

When enabled in the build source configuration on MyGet, source code can be labeled with the build number. This can be done for successful builds only (recommended) as well as for failed builds.

Label Sources during Build and make it a GitHub release

It is also possible to base tags for GitHub Releases on having a fresh package pushed to, say, NuGet.org. This can be done while pushing packages upstream or “promoting” them. A dialog will provide you with additional options, e.g. configure the package version to be used upstream, as well as the option to label the sources for packages originating from MyGet Build Services.

GitHub release when pushing to NuGet.org

Releasing through GitHub

Regardless of the approach taken for creating a label, all that is left now is creating the release on GitHub. Since either the build process or the package promotion process have created the label already, all that GitHub requires is a set of release notes and a description for the release.

Integrating GitHub Releases and MyGet

Pretty sweet, no? Let us know your thoughts through the comments below or in our forums!

Happy packaging and shipping!

Labeling Sources when Pushing to NuGet.org

When adding package sources through your feed’s settings, a very nice scenario becomes available: the package promotion workflow. In other words: pushing a package from one feed to another. Or in other words: publishing nightlies to MyGet and promoting specific package versions to NuGet.org.

With the newly introduced labeling feature, it is now possible to label sources when pushing a package upstream. When enabled, MyGet will find the build from which the package originated and will add a label to the source control revision it was built from. Note that the build must originate from MyGet Build Services for this to work.

Choose the package you want to promote and with a click of a button you can push it upstream. A dialog will provide you with additional options, e.g. configure the package version to be used upstream.

Label sources when pushing upstream

An important note: If you want to make use of labeling, you will have to specify credentials to connect to the remote repository, or remove and add the build source again. Labeling will fail if this is neglected.

Let us know what you think about this feature through the comments below or in our forums!

Happy packaging!

GitHub Commit Status API now supported

A while ago, GitHub launched their Commit Status API which allows integrating the status of a build with commits and pull requests on GitHub. When a MyGet build source is linked to a GitHub repository and has credentials specified, we have started reporting status messages back to GitHub.

GitHub Commit Status API MyGet

When a build succeeds or fails, you will see a status message posted to GitHub, linking to the build log on MyGet.

MyGet reports build status to GitHub

To enable GitHub Commit Status messages on your builds, make sure the build configuration has credentials specified. Specifying credentials can be done by removing and adding the build configuration again, a method which doesn’t require you to enter your password. You can also specify credentials manually by editing the build source.

Happy packaging!

NuGet Package Restore and MyGet Build Services

With the release of NuGet 2.7, a new way of doing package restore came to life. Package restore allows you to keep only your source code and packages.config under source control and just download and install NuGet dependencies during build.

What does this mean for MyGet Build Services? Let’s say we’re making it easier for you! When building from a Visual Studio solution or project, there is nothing you should do: we will run package restore automatically. Let’s dive into the package restore conventions we have in place. Full details on the build process are available from our documentation.

MyGet Build Services

Package Restore Conventions

MyGet Build Services runs NuGet Package Restore as part of every build of solution or project files even if it's not enabled for the solution. Note that package restore is not run for builds making use of batch or PowerShell scripts. In those cases, you are the responsible for running package restore.

In order of precedence, the following package restore commands are run. When one succeeds, package other commands will be skipped.

  • nuget restore MyGet.sln -NoCache -NonInteractive -ConfigFile MyGet.NuGet.config
  • nuget restore MyGet.sln -NoCache -NonInteractive -ConfigFile NuGet.config
  • nuget restore <your solution file> -NoCache -NonInteractive -ConfigFile MyGet.NuGet.config
  • nuget restore <your solution file> -NoCache -NonInteractive -ConfigFile NuGet.config
  • nuget restore packages.config -NoCache -NonInteractive -ConfigFile MyGet.NuGet.config
  • nuget restore packages.config -NoCache -NonInteractive -ConfigFile NuGet.config
  • nuget restore MyGet.sln -NoCache -NonInteractive
  • nuget restore <your solution file> -NoCache -NonInteractive
  • nuget restore packages.config -NoCache -NonInteractive

If you are working with other feeds than the default NuGet.org feed, there are some options available.

Restoring from other Package Sources

If you want MyGet Build Services to restore packages from a specific feed, there are several options available to do this. The easiest way of making a package source available to the build process is by adding it through the UI.Making package source available during build

Using the Add package source button, you can add any feed that you want to make available during build: a Chocolatey feed, a TeamCity feed or another MyGet feed. It is even possible to store credentials so an authenticated feed can be used during build.

If you prefer to have all feed information in your source repository, that’s an option too. Adding a MyGet.NuGet.config file to your repository is the key to success. See the NuGet docs for more information on how such file can be created. The following is a sample registering a custom NuGet feed for package restore.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="Chuck Norris feed" value="https://www.myget.org/F/chucknorris/api/v2/" />
  </packageSources>
  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>
</configuration>

 

A small remark: if you have an authenticated feed but do not wish to add credentials to source control, credentials can be added to the feed's package source. These credentials will be available during build and allow you to consume a protected feed with ease. Authentication of feeds

Summary

Using NuGet Package Restore in MyGet Build Services is a breeze. We follow standard NuGet conventions to do your builds and allow adding package sources which will be made available during build through the UI. If you do need to customize things, there are several hooks available too. Full details on the build process are available from our documentation.

Let us know what you think about this feature through the comments below or in our forums!

Happy packaging!

Online gaming with OCTGN and NuGet

OCTGN online card and table game networkWe’ve been looking through some of the feeds that are hosted on MyGet and have found a nice use case for both NuGet and MyGet: the open-source project OCTGN, the Online Card and Table Game Network. As a result of that, we’ve had a chat with Kelly Elton, developer and maintainer of OCTGN.

Can you tell us a little bit about OCTGN?

“The title of the game kind of summarizes this, but OCTGN is a gaming platform that allows users to play card and board games with each other on the Internet. After installation, gamers get a virtual table top where they can download additional games from a repository, chat with other gamers and compete against each other.”

We’re assuming that the games repository is in fact a NuGet feed, is that correct?

Yeah, that’s an interesting one. Started using NuGet a while ago and it occurred to me that you could use NuGet as an application update system. It has metadata in there which you can query and the NuGet.Core assembly gives you a nice library for working with it.

Originally, our users would download an OPC (Open Packaging Conventions) file which is essentially a ZIP file. A NuGet package essentially also is an OPC package, so we were already using some form of packaging for distributing games before. And then we did some rework and tore some pieces apart and realized we could start packaging using NuGet to get things like searchable metadata, a formal upgrade system and so on.”

A lot of the games are card games, are you embedding the card artwork in the NuGet packages as well?

“No. Sets of cards can be 100 MB to 200 MB in size. Not all players will use all cards all the time and we don’t want game updates to download all artwork when it hasn’t changed between versions. Also we have a card proxy which generates images from card data on the fly if they don't exist, which allows us to cram a game that used to require 100MB of images to play into a 10MB package, with the option to download the cards from another source if they so choose.

We developed a system that allows you to specify the metadata for the cards which is kind of a specialized markup language and then download the artwork on-the-fly, only when needed.”

Who is developing the games for OCTGN? How can someone contribute?

In-game screenshot”Most of the games are developed by community members. OCTGN has been around since 2006 and has changed hands a few times but it’s essentially run by the community. I’m currently doing most of the development but I have some people helping me. The games themselves are developed by people, using Python and XML. We have a builder (o8build) which comes with OCTGN, takes the Python code and XML files and packages everything up so our game developers don’t have to know too much about NuGet themselves. There’s some info about that on our GitHub space.

Contributions are done in several ways. We have different NuGet feeds that are managed by game owners, we also have our official feeds that contain games that we want to be part of our system. You get added to that feed based on different metrics like game quality, how active development is and so on. There are different feeds maintained by other members of the community and they can take on some games as well.

We only host the official feed, but since it's so easy to get setup with MyGet, people in the community have had no problems at all setting up their own feeds.

Are all feeds public feeds?

“Some of our game developers asked if it would be possible to make a private feed for their games, we’re now looking into adding support for that in our client. We weren’t sure how we could authenticate the user towards these private feeds as the NuGet client libraries aren’t that well documented. But I’ve seen there is an ICredentialProvider interface in there we can probably use.”

How has your experience with MyGet been so far?

“Very good, we’ve enjoyed it quite a bit. We didn’t quite know if it was going to be the right system for the job at first as we were almost going to abstract away the way packages were offered to the game. Our big concern was that we have almost 40.000 registered users and we didn’t want to step on your toes with regards to package sizes, bandwidth consumed and so on. We wanted to make sure we weren’t causing any cause of problems as a free user of MyGet.

As far as quality goes we have absolutely had no problems with MyGet at all. It’s really easy to upload and manage packages and we’re now looking into using the build system you have as that’s pretty neat.”

As a last question, do you have any advice for people starting with NuGet or MyGet?

“For MyGet not so much as it can’t be more straightforward. Even people that didn’t know what NuGet was were able to setup feeds and upload their games in a matter of a minutes. That was really easy and I would encourage people to try it out. You guys are doing some awesome stuff!

One thing I’ve noticed with NuGet that tripped people up is versioning, sometimes the version numbers weren’t sorted in the way we’ve expected but we’ve tackled that. I believe Xavier had a blog post about this.

Another thing is most people seem to be focused on creating or consuming packages and that is also what all the documentation on the NuGet site is about. Once you realize that NuGet can also be used as a distribution system of packages for your own application, things get more interesting. You can get started developing such thing by adding the NuGet.Core assembly to your project and navigating around with IntelliSense. The whole system is pretty slick and I’m using it in other applications I work on as well.

NuGet has a lot more uses than what can be seen at first sight, there is a whole lot to explore!”


Thank you Kelly for this chat, really appreciated!

Just like OCTGN, if you are working on an open source project you can get an Open Source subscription plan for your project!

Happy packaging!