Inspecting audit logs in MyGet Enterprise

A couple of weeks back, we released an audit log viewer on MyGet Enterprise. Administrators of a MyGet Enterprise plan can inspect every action that happens on their MyGet instance and see who did what, when, and where.

From the MyGet Enterprise administration dashboard, all actions performed on the Enterprise installation can be consulted:

The list of audit entries is searchable and can be exported to a CSV file so additional querying can be done in, for example, Excel. Details for each audit entry can be consulted and display the action that was performed, who performed it, and where it was executed. We can also see the location the user performed the action from, including the IP address (last octet always 0 for privacy reasons).

With audit logs available on MyGet Enterprise, we are confident MyGet can help larger teams and enterprises with auditing and dependency lifecycle management.

Happy packaging!

Author: Maarten Balliauw on 16 Nov 2017

PHP Composer packages just arrived on MyGet

MyGet supports hosting private PHP Composer packages

Good news everyone! We just shipped PHP Composer support on MyGet! If you are building PHP applications and libraries, you can now package them and add these to your MyGet feeds.

PHP Composer support is available for all MyGet accounts - check the PHP Composer features described in our documentation

Which features are available?

We currently support almost all features we have available for other package managers. Of course you can upload your own packages (via the web UI as well as via a curl POST) or packages from upstream repositories like Packagist.

Packages can be consumed in any PHP Composer-based project. It’s possible to proxy upstream repositories into your MyGet feed. You can manage permissions and users, inspect package licenses and vulnerabilities, …

Build services are supported as well: as long as there is a composer.json in your repository, we’ll run tests against it, package it up and make it available as a PHP Composer package on your Myget feed.

Sounds great! How do I get started?

Quite easy: head over to www.myget.org and sign in (or register). You can then create a feed and start adding packages. Our getting started documentation has some more details on how to upload your first PHP Composer package to MyGet.

We’re really excited about introducing PHP Composer support on MyGet! You can now use MyGet to securely host and collaborate on NuGet, symbols and sources, Chocolatey, PowerShell, NPM, Bower, Maven, PHP Composer and VSIX packages.

Happy composing!

Author: Maarten Balliauw on 11 Sep 2017

Using a private MyGet feed with JetBrains Rider

image_thumb2JetBrains just released a new .NET IDE: Rider. At MyGet, we’ve been using Rider for our internal development since it was announced. So far, we have really enjoyed this IDE built around ReSharper! And since it comes with a lightning-fast NuGet client, let’s see how we can consume packages from a MyGet feed.

Adding a MyGet feed package source

The first step in connecting Rider to a MyGet feed is adding it as a package source. We can do this using NuGet.exe (via good old NuGet.config), or from within Rider. From the NuGet tool window, open the Sources tab. This will show us all of the NuGet configuration files that are in play, and a list of all feeds configured.

image25

From here, we can add our MyGet feed (or edit an existing entry). We will have to give our feed a name so we can easily recognize it in Rider, and the URL to our feed. This URL can be found on the MyGet feed details page after logging in to www.myget.org.

image31

The NuGet client in Rider supports working with public and private MyGet feeds. While Rider supports using pre-authenticated feeds as well as feeds that require entering credentials, we recommend using the latter. Rider safely stores our MyGet username/password in its password store, which is based on KeePass.

Using MyGet together with JetBrains Rider makes it possible to develop .NET applications and let your development team consume both public and private packages hosted securely on MyGet.

Happy packaging!

Author: Maarten Balliauw on 04 Aug 2017

New and improved gallery details page

The MyGet Gallery contains a collection of interesting feeds where open-source projects and software vendors share their NuGet, npm, Bower and VSIX packages with the world. Most often the packages in the gallery are nightly builds or preview versions, so we can keep our projects on the cutting edge of technology using the latest dependencies.

We just deployed some improvements to the gallery details page:

  • We now display the feed’s README, where we render Markdown and the feed owner can provide additional information like links to GitHub, documentation and so on.
  • Underneath, the list of packages on the feed is shown, including a description. We also added a search box so we can do a quick search across the packages listed on the feed.
  • The top bar has a Connect to Feed button, which will provide connection details to, for example, connect to the feed from Visual Studio or npm.

Using nightly builds for MongoDB

Have a look at the feeds in the MyGet Gallery, and let us know what you think using the comments below or via Twitter.

Happy packaging!

Author: Xavier Decoster on 01 Aug 2017

MyGet 2017.1 Release Notes

As MyGet is a software-as-a-service leveraging a subscription model, we're transitioning our versioning scheme towards a format that is more understandable: YYYY.R. As such, these release notes comprise our first milestone of 2017, hence the version number 2017.1

The MyGet 2017.1 milestone was tagged on June 1st, 2017.

Highlights

MyGet again adds some new functionality to the service. The following are the major highlights of this milestone.

We've built a MyGet Credential Provider for Visual Studio 2017! This extension allows you to authenticate against your MyGet feeds using OAuth. Install it from the Visual Studio Gallery!

Install the MyGet Credential Provider for Visual Studio 2017!

We added Maven support, and welcome Java/Android developers to the MyGet family! (Announcement | Docs)

Getting started with Maven on MyGet!

We've built a web utility to help you learn and adopt Semantic Versioning: check out our MyGet SemVer Explorer!

MyGet SemVer Explorer

We've partnered with OSSIndex.net to check for potential package vulnerabilities on your MyGet feeds! (Announcement | Docs)

Check for potential package vulnerabilities on your MyGet feeds!

Features

MyGet (all plans)

The following applies to all MyGet plans:

  • NPM: added support for token authentication
  • NPM: added support for upstream token authentication, which now also supports Telerik's NPM registry as an upstream package source
  • NPM: added support for the fast search endpoint
  • NPM: added support for package deprecation
  • NPM: added support for package tagging
  • NPM: added support for dist-tag
  • NuGet: added support for NuGet's SemVer2 protocol, and added support for modifying build metadata on push upstream dialog
  • Maven: introduced support for Maven artifacts
  • Maven: introduced support for Android AAR artifacts
  • Symbols: added a toggle to support pushing symbols upstream as well
  • Symbols: when the upstream target feed is a MyGet feed, we automatically also push the symbols upstream
  • Usability: no longer show symbols packages separately on the Gallery's feed details view
  • Usability: minor modifications to the Gallery feed details UI to improve the user experience
  • Usability: added a download all button to the packages dropdown in build results view
  • Usability: hide pre-authenticated feed endpoints from Feed Details view when the feed is not a private feed
  • Usability: added a copy-to-clipboard button to the connection details popup on Gallery feeds
  • Security: we've built a MyGet Credential Provider for Visual Studio 2017! This extension allows you to authenticate against your MyGet feeds using OAuth.
  • Security: we've consolidated the login page: one page to rule them all!
  • Security: we no longer display access tokens (you can still copy them though)
  • Security: improved auditing
  • Security: added support for feed and privilege scopes to access tokens / api keys (in addition to expiration support which we already had)
  • Integrations: SymbolSource.org integration has been retired in favor of MyGet's own Symbols functionality
  • Integrations: added OSSIndex.net integration to detect package vulnerabilities and report them on your feed details view

MyGet Enterprise

The enterprise plan has all functionality from the paid subscription plans, and more! The following applies only to the MyGet Enterprise plan:

  • Usability: the Gallery index is now the default landing page when authenticated on MyGet Enterprise
  • Security: added support for marking users as external to the tenant. This prevents the external user from accessing Enterprise feeds, unless privileges are explicitly assigned at the feed level. (see Feed Types)

MyGet Build Services

  • Added support for Visual Studio 2017.NET Core and the new PackageReference project format (Announcement)
  • AssemblyInfo patching now supports globbing patterns (like **\**.cs)

Bug Fixes & Other Improvements

  • MyGet has been upgraded to run on .NET Framework 4.6.2, which seemed to have positive effect on performance
  • Overwriting source symbols is now blocked when forbid overwrite is enabled on the feed
  • Fixed a bug in semantic version range parsing of npm dependencies (tilde and carret ranges)
  • Show quota per feed on user profile page (helps answer the question: 'which of my feeds consumes most?')
  • Fixed an issue caused by breaking changes in VSTS API (repository remoteUrl returned by VSTS API no longer contained VSTS collection name)
  • Fixed an issue in the Gallery index view related to feed icons
  • Fixed an issue that caused an HTTP 500 when a nuspec contained some invalid data
  • Fixed an issue that caused NPM push upstream to fail when no package description was given
  • Fixed an issue with the symbols code browser when a file was not found or could not be displayed
  • NuGet: allow packages.config files to be uploaded without version number specified

Please tell us how we're doing by taking 10 seconds of your time to answer a single question (and optionally provide any feedback you want). We love hearing from you, so keep that feedback coming! MyGet is built for you!

Happy packaging!

Host your packages on MyGet!

Author: Xavier Decoster on 17 Jul 2017

Working with MyGet upstream sources

Upstream sources play a key role in a professional approach towards Package Management. MyGet gives you the option to specify one or more upstream sources for a package feed. Even though this feature has been available on MyGet for years now, we feel it upstream sources deserve a place in the spotlight once: they enable various scenarios that are impossible on any other package management service, and above all: they are a huge facilitator for a smooth and automated development workflow (and thus developer happiness :-)).

In this post:

  • Why upstream sources?
  • Supported upstream sources

Why use upstream sources?

In bullet-form:

  • Upstream sources make it very easy to pull in packages from other package sources onto your downstream MyGet feeds.
  • You can also target these upstream sources to push packages upstream from your MyGet feeds.
  • Any configured upstream source on your MyGet feed will be made available to you in MyGet Build Services, without having to commit any secrets such as credentials or API keys in your source repository. If you're using NuGet: no need to create a nuget.config file!

If you are confused about the usage of "upstream" and "downstream" in the context of package sources, we've got a little poetic explanation for you, which may already help you visualize the relationship between package consumers, your MyGet feeds, and your feed's upstream sources.

Consider the direction in which packages are flowing from a given package source to an ocean of consumers.Your package may have dependencies "upstream", to packages on another feed. From the point of view of those dependencies, the depending package is located "downstream". When a user consumes the downstream package, it will also fetch the upstream dependencies.The consumer, however, is only allowed to fetch or query those upstream packages if the feed being queried (downstream) is also configured to proxy (or mirror) the upstream package source.


<h2>Supported upstream sources</h2>

By default, MyGet feeds have the public, central repositories configured for each package type. This includes:

  • NuGet: https://www.nuget.org/api/v2
  • Bower: https://bower.herokuapp.com
  • npm: http://registry.npmjs.org
  • Maven: https://repo1.maven.org/maven2

To configure an additional upstream source for your MyGet feed, navigate to Feed Settings > Upstream Sources. Then click Add Upstream Source and select the package source type you want to add.

A dialog will prompt your for upstream source information and will also expose a few common presets for you to take advantage of. Did you know we even support Dropbox?!

Upstream Source Credentials

If you have any access privileges to other MyGet feeds, you will see those in the MyGet Feeds presets, so you can easily build a chain of package sources to facilitate a package promotion flow.

If you select a private MyGet feed you have access to as an upstream source, there's no need to provide credentials to be able to restore packages from it on MyGet Build Services. MyGet will impersonate your user account when authenticating against that upstream source.

For any non-MyGet upstream source that requires authentication to pull packages, you'll have to provide username and password to be used during Basic Authentication.

Warning! Be very careful with password managers and browser add-ons providing auto-completion of credentials!
We recommend disabling these credential managers on the MyGet web site to avoid issues when editing upstream sources. Oftentimes, these tools auto-complete the credentials fields with out-dated credentials (even when editing different settings in the dialog). 
When running into package restore failures on MyGet Build Services, or when noticing that upstream packages are no longer available downstream, this is the most common source of the issue.

In the opposite direction, in order to push packages from your downstream MyGet feed to the upstream source, you may need to configure a (scoped) API key or access token.

Package Source Filtering

Applies to: NuGet (v2 only!)

When the upstream source is a v2 NuGet feed, you may configure additional OData filtering. Filtering is based on the OData v3 Filtering System. Valid filters are similar to Id eq 'jQuery' or IsLatestVersion eq true and Id ne 'Foo'.

Warning! This capability may go away at some point in favor of newer NuGet v3 APIs.
We currently still keep the feature around for some scenarios that are not yet fully supported on NuGet v3.

Adding a package from an upstream source

You can easily add packages to your MyGet feed originating from an upstream source, such as nuget.org, nmpjs.org, etc. This is using the feed's configured upstream sources under the hood. If you want to add a package from another feed onto your MyGet feed, the other feed needs to be configured as an upstream source to that feed.

Adding a package from an upstream source can happen in three ways: manually, by reference (proxying), or by value (mirroring).

  • Manually: you can add packages from an upstream source to your feed manually by using the Add Package button you will find under your feed's Packages page.

Select From Feed in the dialog that prompts.

  • Proxying: the package metadata is copied to the MyGet feed, the package itself remains hosted on the upstream source. When querying the package, we call the upstream source to fetch the package.
  • Mirroring: the package metadata and the package itself are copied onto the MyGet feed. When querying the package, we server the package directly and don't use the upstream source. Mirroring of a package version happens upon the first request for that given package version.

Configuring upstream sources on your MyGet feed unlocks quite a few integration scenarios and automation opportunities!

Proxy packages from an upstream source

You can configure an upstream source to proxy upstream packages through your MyGet feed to your feed consumers. Proxying makes it easy to have a single MyGet feed aggregate packages from multiple sources. Package consumers need only to configure a single MyGet feed, and all packages available on upstream, proxied package sources will become available to them.

Benefits:

  • upstream packages do not count against your MyGet storage quota
  • authentication against upstream, private MyGet sources happens automatically (see Upstream Source Credentials)
  • especially for NuGet: no longer subject to chatty clients reaching out to all configured feeds during dependency resolution (MyGet can be smarter server-side)

Drawbacks:

  • every package request will incur additional latency as opposed to storing the package onto the MyGet feed

Warning! Avoid configuring multiple package source proxies on a single feed, or in a chain of feeds, as this will magnify the disadvantages, and result in very slow feed response times.

The following diagram illustrates the effects of upstream source proxying.

To enable upstream source proxying, you must tick the check-mark next to Make all upstream packages available in clients.

Mirror packages from an upstream source

You can configure an upstream source to mirror upstream packages onto your MyGet feed. This configuration is similar to package proxying, but takes it one step further.

Whenever someone requests a particular package from your MyGet feed for the first time, MyGet will query the upstream source and copy the package onto the MyGet feed.

Benefits:

  • No additional latency (except for the first hit that triggers the package mirroring)
  • Protected against upstream source availability issues
  • Protected against upstream package removal
  • Authentication against upstream, private MyGet feeds happens automatically (see Upstream Source Credentials)
  • Faster builds!

Drawbacks:

  • Mirrored packages count towards your MyGet storage quota (a classic storage versus speed trade-off, you can always upgrade your subscription or request a quote!)

The following diagram illustrates the effects of upstream source mirroring.

To enable upstream source mirroring, you must tick the check-mark next to Automatically add downloaded upstream packages to the current feed (mirror).

Optionally, you can also tick the third check-mark to indicate that any package found upstream is to be considered a package dependency (and should not be consumed directly). This will hide those packages from search results, whilst still allowing you to restore them.

Once upstream source mirroring is enabled, we can consume our MyGet feed from Visual Studio which will also list upstream packages. For example, the example acmecompany feed only lists one package:

One package on our feed

When searching in Visual Studio, we do see packages that originate from upstream sources:

Visual Studio showing upstream packages

After installing this package, our feed now automatically contains a copy of the jQuery package:

Mirror upstream pckages

From now on, the package is available from our MyGet feed directly, without having to explicitly add it manually from the upstream source.

Using a MyGet feed as a staging area (before pushing upstream)

Many development teams are using some kind of package promotion workflow: pushing a package from one feed to another based on quality gates, target audience, or any other criteria. This is very typical scenario for which upstream sources are essential.

Of course, all of this can happen in an automated fashion using package manager client. However, as promoting a package typically involves some kind of human intervention (e.g. release manager approval), we've also made it a first-class feature in the MyGet web site.

Simply pick the package version you want to promote from the package details page, and hit the Push button to initiate the package promotion flow.

A dialog will provide you with additional options. MyGet is also smart enough to detect any package dependencies you might want to push along in one go as part of this package promotion flow.

At this point, you can still make a few metadata changes before pushing upstream. This dialog allows you to:

  • modify or remove the prerelease label of the upstream package version. This allows you to e.g. drop the prerelease label to release a package without rebuilding/repackaging.
  • add release notes to be included in the package metadata. MyGet will even support release notes written in markdown and render them properly on the web site!
  • modify or remove the SemVer2 build metadata part of the upstream package version
  • exclude any detected dependencies or satellite packages from the push action
  • apply source labeling if the package was built using MyGet Build Services. When enabled, MyGet will find the build from which the package originated and will add a label to the source control revision it was built from.

To edit a package's metadata, simply click the Edit button next to it and make the modifications. To apply a given modification to all packages in the dialog, hit the rain drop button next to the editable field.

Using upstream sources on MyGet Build Services

Applies to: NuGet, npm

Upstream sources for a feed are also available during build. This can be useful in the following scenarios:

  • An additional upstream source is needed during build. MyGet will make the upstream source available during build if it has been added to the feed's upstream sources.
  • If you have a private feed requiring authentication but do not wish to add credentials to source control, credentials can be added to the feed's upstream source. These credentials will be available during build and allow you to consume a protected feed with ease.

Applies to: NuGet

  • The API key for an upstream source is also made available during the build process. This means during a build, you can call into nuget.exe push and push packages to configured upstream sources.
  • If you want to make use of nuget.exe push in a build script without having to specify the -Source parameter. This requires a default upstream source to be defined.

Applies to: npm
We strongly suggest to proxy registry.npmjs.org to be able to run `npm install` during build, as npm will default to the MyGet feed as the default registry.

Setting default upstream sources to be used on MyGet Build Services

Applies to: NuGet

The NuGet.config file on our build agents is configured using NuGet's defaults, enriched with all NuGet upstream sources configured for your MyGet feed. Based on these defaults, the following conventions are active:

  • The default upstream source is set to (Aggregate Source), meaning all feeds will be queried for packages in the order defined in the feed's upstream sources.
  • The default push source (when using nuget push without the -Source parameter) is NuGet.org.

Both of these conventions can be overridden by editing the build source configuration.

Auto updating packages

MyGet feeds can automatically fetch package updates made available through the upstream sources.

When adding or editing a upstream source, we can enable this behaviour per package source, as well as an interval when MyGet should check for updates.

Package Source Options

The following options are available:

  • E-mail me when package updates are available: Sends an e-mail to the specified recipient(s) when package updates are available on the upstream source.
  • Include prerelease versions: By default, MyGet will only consider stable packages. When enabled, we will also check pre-release packages from the upstream source.
  • Automatically update packages to their latest versions: Enables the behavior of automatically updating packages from the upstream source.
  • Update interval: Depending on your subscription plan, we can specify how often MyGet should check for updates (up to every 30 minutes on a Professional subscription)

As you can see, MyGet's support for upstream package sources unlocks a wide range of package management scenarios that may help you streamline your development flow and package governance even more. If you haven't tried the above scenarios yet, do give them a try and experience how it may make your life easier.

Oh, and we do support pushing your private symbols packages upstream along with your NuGet packages, too!

Happy Packaging!

PS: Please take 10 seconds of your precious time to tell us how we're doing

Author: Xavier Decoster on 12 Jul 2017

MyGet's NuGet and NPM news from the community (April 2017)

Another month, another episode of MyGet's NuGet and NPM news from the community! We'll look at some interesting blog posts and articles found on the Internet, curated by our MyGet founders Xavier and Maarten. Follow @MyGetTeam on Twitter for more!

Note that this will be the last episode of our monthly news for now. Let us know if you'd like to see this series continue!

NuGet news

The NuGet team is considering improving package identity and trust by allowing to verify accounts and reserve package prefixes. For example, System.* could be reserved for the .NET Foundation so consumers always know the package comes from a thrustworthy source. Good read, good thoughts!

Naeem Khedarun built a tool that scans of outdated packages. NuGetXray, as the tool is called, helps identify outdated packages and can visualize this in a nice report (which can be included in a TeamCity build).

More tools! Maarten blogged about extending .NET CLI with custom tools, with an example tool dotnet init that initializes your NuGet package metadata.

MORE TOOLS! Alistair Chapman is introducing the Cake Build Systems Module which gives CI superpowers. For example on MyGet Build Services it will use service messages to add additional tracing to the build log. On TeamCity, the currently running build task is displayed. And more - give it a go!

NPM news

A GitHub PR was opened, discussing npm@5 and what will be included. Work in progress!

Axel Rauschmayer blogged about setting up multiplatform npm packages. In an example, he describes creating a package that targets ES5+, ES6 and webpack at the same time. 

Happy packaging!

Author: Maarten Balliauw on 29 Apr 2017

How Stackify uses MyGet to manage their .NET dependencies and their product

We love developer stories! The folks at Stackify - who build, among other things, a free .NET Profiler, Prefix - wanted to share why and how they use MyGet to solve their .NET dependency management. Next to using MyGet for dependency management, they also share their nightly builds with customers and key users to be able to gather early product feedback. We'll let Matt Watson tell the story:

As a Microsoft developer, I was excited for .NET to solve the nightmares of dll hell. It did… but has since created a new problem: package hell. MyGet helps solve some of the challenges of NuGet. MyGet has been a big help for my team and can probably help yours, too!

At Stackify, we use MyGet for a few different reasons, which we will cover in this post. You may have heard of us via our awesome & free .NET Profiler, Prefix, or our APM solution, Retrace.

Pre-Release Packages

We started playing with .NET Core when it was in the early betas. At that time, the only way to get the latest versions of .NET Core nuget packages was via the MyGet package feed. Things were rapidly changing and we were constantly trying to keep up with what the .NET community was releasing.

As you can imagine, Microsoft does not ship daily builds of every .NET framework library to NuGet for everyone to access. They only release thoroughly tested and reviewed updates.

Between those updates they can use MyGet to get pre-release versions to their beta testers. They could publish pre-release versions to NuGet, and sometimes they do.

Publishing pre-release version to NuGet also broadens who has access to the packages. By putting them on MyGet, it keeps the casual developer from downloading beta versions and having lots of problems.

MyGet is a great solution for publishing daily builds of your packages. You can then tell your power users how to access them.

At Stackify, we can do the same sort of process to internally test new versions of our packages before we give them to our clients or air our dirty laundry on NuGet.

Hosting Internal Private Packages

At Stackify, we have several projects that are shared across multiple applications. We have elected to split out some of these projects in their own source code repositories. It helps us keep our repo size(s) down and also helps enforce some good practices around changing shared dependencies.

NuGet packages are the preferred way to share those class libraries across applications. Since the code is for internal purposes, MyGet’s private package feeds are a great solution for us. It provides a secure way to use NuGet packages.

We created a build and deployment process up in our Bamboo build server for the projects. After we check in our code, we manually kick off a build and it will take care of building the code and publishing a new package to MyGet for us.

After the shared packages are updated on MyGet, we can update the applications that use those shared packages.

Publishing NuGet Packages with Debug Symbols

One of our applications requires that we use tools like WinDbg to regularly analyze crash dumps. To do this, we need the debug symbols from our projects.

For security reasons, we don’t ship our debug symbols anywhere. Trying to keep track of the matching debug symbols for every version of our class library is a nightmare.

We solved this problem by always publishing a package that includes the debug symbols to our private MyGet package feed. Now anytime we need to use WinDbg, we can point our symbol source at MyGet and WinDbg automatically retrieves the debug symbols from MyGet. It is almost magical!

This also works well for debugging the libraries we have pulled out into their own repos as shared dependencies. It allows developers to test their package but still step through the code.

Thanks for the story, Matt! Make sure to check out both Stackify and MyGet!

Guest post by Matt Watson. Matt is the Founder & CEO of Stackify. He has been a developer/hacker for over 15 years and loves solving hard problems with code. While working in IT management he realized how much of his time was wasted trying to put out production fires without the right tools. He founded Stackify in 2012 to create an easy to use set of tools for developers.

Author: Maarten Balliauw on 20 Apr 2017