MyGet Blog

Package management made easier!

NAVIGATION - SEARCH

A very useful MyGet PowerShell suite

We recently shed a light on how you can easily use additional build tools on MyGet Build Services. However, there is more. A lot more!

We always say how much we love our users and this blog post is yet another illustration why we do. One of our users, Peter Rekdal Sunde, created an awesome PowerShell utility pack to make it even easier to customize your MyGet Build Services experience. The result is a complete build suite for creating NuGet packages and interacting with the MyGet Build Services environment. The scripts not only work on MyGet but also on your local development computer (you do need to have msysgit installed though). The entire code base was generously opensourced (MIT license) and is available on GitHub: https://github.com/peters/myget.

How does it work?

Simply include the myget.include.ps1 script in your build.ps1 on MyGet and use the provided functions.

Where do I begin?

To illustrate its purpose, we provide you a glimpse at some of the functionality provided by these scripts:

Build agent communication

  • MyGet-Write-Diagnostic - writes a diagnostic message to the standard output
  • MyGet-Build-Success - report build success
  • MyGet-Die - report build failure

NuGet utility functions

  • MyGet-NuGetExe-Path - path to NuGet.exe
  • MyGet-NuGet-Get-PackagesPath - returns the value of the repositoryPath attribute in nuget.config for a given project folder

Build steps

  • MyGet-Build-Bootstrap - starts a build (including NuGet package restore)
  • MyGet-Build-Solution - starts a build of a solution file
  • MyGet-Build-Project - starts a build of a project file
  • MyGet-Build-Nupkg - creates a NuGet package based on a specified .nuspec file. The .nuspec can contain additional replacement tokens, taking benefit from some of the variables provided by default by MyGet Build Services. More information at https://github.com/peters/myget#nuspec-substitutions.

Test runners

  • MyGet-TestRunner-Nunit - invoke NUnit
  • MyGet-TestRunner-Xunit - invoke XUnit

We recommend you to check out the readme and the samples for a detailed view of what's available though. Especially the test runner support is really nice, just check the below example!

Thank you Peter!

Happy packaging!

MyGet Year in Review 2013

2013 was a good year. We had tons of fun developing MyGet and judging from the feedback we get from you we can tell you are having fun with MyGet as well. And that is awesome! Thank you so much for sharing your feedback and your continued support!

Speaking of feedback: what do you think our 2014 roadmap should look like? Let us know through this survey!

Looking back at 2013 also involves some numbers, so here goes:

  • Average response time went from 3 081 ms back in January to 538 ms in December. That's almost a factor 6 improvement! Now let's scale those improvements to various other components of MyGet.
  • Average uptime in 2013 was 99.83%. We had 6 out of 12 months with 100% uptime. That's not too bad knowing that we combine various components with a 99.99% service availability each. Full history can be found on our status page.
  • Despite a 5min glitch on October 10, we have to go back to September 12 to find our last service issue caused by a circular package dependency, which in turn caused our package retention rules engine to choke on it. Measures have been taken to prevent this from happening on MyGet and a full root cause analysis is available on our blog.
  • We are now hosting 2394 feeds (excluding the Enterprise plan feeds), good for 85.000 unique packages and 40 GB of storage (plus 3 months worth of backups, that's 600 GB in total). Bandwidth is currently just below 1 TB a month.
  • Build services has ran 4.800 builds this year, and we're seeing quite some growth there over the last months.

Even though we deploy continuously, we tagged the following releases in 2013:

  • MyGet 1.5 (January 5th): new profile page, activity streams, feed cloning, package retention policies, CodePlex and BitBucket integration in MyGet Build Services.
  • MyGet 1.6 (February 25th): feed setting for SemVer validation, download feed as ZIP, package sources out-of-beta, lots of improvements to build services (versioning, SDKs and tools, logs, ...)
  • MyGet 1.7 (May 17th): new documentation site, search and filtering enhancements, NuGet 2.5 compatibility, Package Source Discovery, auto-publish symbols feed-setting, build service enhancements (assemblyinfo-patching, links to produced packages, support for build.ps1/.cmd/.bat, build parameters)
  • MyGet 1.8 (September 10th): NuGet 2.7 compatibility (new package restore!), auto-update metadata from upstream packages, package pinning, source labeling when pushing packages upstream, automatic package mirroring for feed-proxies, build services enhancements (build labeling compatible with GitHub releases, support for MyGet.ps1/.cmd/.bat).
  • And the obvious bug fixes (and occasionally new bugs) along the way...

We also shipped our second book: Pro NuGet, 2nd Edition (October 7th), packed with tons of new stuff and a dedicated chapter of recipes. Thanks again to Apress and the NuGet team for the high-quality reviews and helping us ship what is probably the most complete NuGet guide out there.

Let's make 2014 another great year for MyGet, NuGet and dependency management in general!

Happy New Year! And happy packaging!

PS: Let us know your feedback on what you want from MyGet in 2014 through this survey!

Using additional tools in the build process

Out of the box, MyGet Build Services supports building projects that depend on various SDK's and/or test runners. A full list of supported project types and SDK's at your disposal is available on our docs.

However, sometimes you run into a missing tool or SDK you absolutely need to compile your projects. Or perhaps you need a different version than the one available by default. If that's the case, you have two options:

  • Download the tools you need and commit them to your own repository
  • Use the brand new MyGet Build Tools repository on GitHub as a submodule to your repository

The first option is likely what you're doing today and allows you to cherry-pick the required tools without polluting your build with tools you don't need, making the build process faster.

The second option will bring down the tools from this repository during a build while they are technically not in your own GitHub repository. It currently contains the following tools:

  • NuGet 2.5, 2.6, 2.7, 2.7.1
  • NUnit 2.6.2
  • XUnit 1.9.6
  • curl

To add a submodule, run the following from your repository root (make sure you use the https endpoint):

git submodule add https://github.com/myget/BuildTools.git myget

From then on, you can use the tools from the myget/tools folder in your builds. For example, NuGet 2.5 can be used by calling myget/tools/nuget/2.5/nuget.exe.

Happy Packaging!

Online gaming with OCTGN and NuGet

OCTGN online card and table game networkWe’ve been looking through some of the feeds that are hosted on MyGet and have found a nice use case for both NuGet and MyGet: the open-source project OCTGN, the Online Card and Table Game Network. As a result of that, we’ve had a chat with Kelly Elton, developer and maintainer of OCTGN.

Can you tell us a little bit about OCTGN?

“The title of the game kind of summarizes this, but OCTGN is a gaming platform that allows users to play card and board games with each other on the Internet. After installation, gamers get a virtual table top where they can download additional games from a repository, chat with other gamers and compete against each other.”

We’re assuming that the games repository is in fact a NuGet feed, is that correct?

Yeah, that’s an interesting one. Started using NuGet a while ago and it occurred to me that you could use NuGet as an application update system. It has metadata in there which you can query and the NuGet.Core assembly gives you a nice library for working with it.

Originally, our users would download an OPC (Open Packaging Conventions) file which is essentially a ZIP file. A NuGet package essentially also is an OPC package, so we were already using some form of packaging for distributing games before. And then we did some rework and tore some pieces apart and realized we could start packaging using NuGet to get things like searchable metadata, a formal upgrade system and so on.”

A lot of the games are card games, are you embedding the card artwork in the NuGet packages as well?

“No. Sets of cards can be 100 MB to 200 MB in size. Not all players will use all cards all the time and we don’t want game updates to download all artwork when it hasn’t changed between versions. Also we have a card proxy which generates images from card data on the fly if they don't exist, which allows us to cram a game that used to require 100MB of images to play into a 10MB package, with the option to download the cards from another source if they so choose.

We developed a system that allows you to specify the metadata for the cards which is kind of a specialized markup language and then download the artwork on-the-fly, only when needed.”

Who is developing the games for OCTGN? How can someone contribute?

In-game screenshot”Most of the games are developed by community members. OCTGN has been around since 2006 and has changed hands a few times but it’s essentially run by the community. I’m currently doing most of the development but I have some people helping me. The games themselves are developed by people, using Python and XML. We have a builder (o8build) which comes with OCTGN, takes the Python code and XML files and packages everything up so our game developers don’t have to know too much about NuGet themselves. There’s some info about that on our GitHub space.

Contributions are done in several ways. We have different NuGet feeds that are managed by game owners, we also have our official feeds that contain games that we want to be part of our system. You get added to that feed based on different metrics like game quality, how active development is and so on. There are different feeds maintained by other members of the community and they can take on some games as well.

We only host the official feed, but since it's so easy to get setup with MyGet, people in the community have had no problems at all setting up their own feeds.

Are all feeds public feeds?

“Some of our game developers asked if it would be possible to make a private feed for their games, we’re now looking into adding support for that in our client. We weren’t sure how we could authenticate the user towards these private feeds as the NuGet client libraries aren’t that well documented. But I’ve seen there is an ICredentialProvider interface in there we can probably use.”

How has your experience with MyGet been so far?

“Very good, we’ve enjoyed it quite a bit. We didn’t quite know if it was going to be the right system for the job at first as we were almost going to abstract away the way packages were offered to the game. Our big concern was that we have almost 40.000 registered users and we didn’t want to step on your toes with regards to package sizes, bandwidth consumed and so on. We wanted to make sure we weren’t causing any cause of problems as a free user of MyGet.

As far as quality goes we have absolutely had no problems with MyGet at all. It’s really easy to upload and manage packages and we’re now looking into using the build system you have as that’s pretty neat.”

As a last question, do you have any advice for people starting with NuGet or MyGet?

“For MyGet not so much as it can’t be more straightforward. Even people that didn’t know what NuGet was were able to setup feeds and upload their games in a matter of a minutes. That was really easy and I would encourage people to try it out. You guys are doing some awesome stuff!

One thing I’ve noticed with NuGet that tripped people up is versioning, sometimes the version numbers weren’t sorted in the way we’ve expected but we’ve tackled that. I believe Xavier had a blog post about this.

Another thing is most people seem to be focused on creating or consuming packages and that is also what all the documentation on the NuGet site is about. Once you realize that NuGet can also be used as a distribution system of packages for your own application, things get more interesting. You can get started developing such thing by adding the NuGet.Core assembly to your project and navigating around with IntelliSense. The whole system is pretty slick and I’m using it in other applications I work on as well.

NuGet has a lot more uses than what can be seen at first sight, there is a whole lot to explore!”


Thank you Kelly for this chat, really appreciated!

Just like OCTGN, if you are working on an open source project you can get an Open Source subscription plan for your project!

Happy packaging!

How SharpDevelop uses MyGet and NuGet

This is a guest post by Matt Ward, working on SharpDevelop, an open source Integrated Development Environment (IDE) for .NET applications which supports the development of applications written in C#, Visual Basic.NET, F# and IronPython. It is and written in C#. In this post, Matt shares their story on using MyGet to host and discover add-ins for SharpDevelop.

If you would like to share your story too, let us know! We love to hear good stories!

SharpDevelop NuGet MyGetThe SharpDevelop team recently announced a new Addin Manager for SharpDevelop 5 and an online gallery for addins that is hosted on MyGet.

SharpDevelop has been extensible through addins since it was first created but it has never before had an online gallery. The majority of IDEs have an online gallery that can be used to find and install extensions. WebMatrix is one example that provides extensions through from its own NuGet feed just like SharpDevelop.

Addins for SharpDevelop 5 can now be published as NuGet packages to a MyGet gallery. NuGet gives SharpDevelop all the benefits of versioning and updating addins whilst MyGet provides a central place where SharpDevelop addins can be found and installed from.

SharpDevelop's addin gallery on MyGet is a community gallery which is open to anyone to publish their own addins.

Background

About a year ago there was a discussion on the SharpDevelop mailing list about having an online source of addins for SharpDevelop. Andreas Weizel had started work on a way to obtain addIns from an online source but at the time it was not based on NuGet. Christoph Wille, Project Manager for SharpDevelop, suggested looking at re-using the NuGet gallery for hosting the packages and using NuGet to download and install addins. One idea was to put the addins on the main NuGet.org feed and add a special tag to them to distinguish them however it made more sense for SharpDevelop to have its own gallery. Rather than setting up a server and hosting a NuGet gallery ourselves it was quicker and easier to use MyGet.

Now let us take a look at how you can make your own addins available for SharpDevelop 5. If you have published a NuGet package to the gallery hosted on NuGet.org then the following procedure will be very familiar.

Publishing an Addin with MyGet

Here we will look at the steps taken to publish an addin that provides support for compiling against Mono. The source code for this sample addin is available on GitHub. The first step is to compile the addin. With the Mono addin compiled we need to create a .nuspec file that will be used to generate a NuGet package containing all the files needed for the addin:

    <?xml version="1.0"?>
    <package xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <metadata xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
            <id>Mono</id>
            <version>1.0</version>
            <authors>Matt Ward</authors>
            <owners>SharpDevelop</owners>
            <requireLicenseAcceptance>false</requireLicenseAcceptance>
            <description>Mono support for SharpDevelop</description>
            <iconUrl>http://community.sharpdevelop.net/blogs/mattward/SharpDevelop.png</iconUrl>
            <summary>Mono support for SharpDevelop</summary>
            <releaseNotes></releaseNotes>
            <language>en-US</language>
            <tags>mono</tags>
        </metadata>
        <files>
            <file src="..\..\..\AddIns\Samples\Mono.Addin\**\*.*" target="" />
        </files>
    </package>

In the above .nuspec file we are adding all the addin files, including any subdirectories, by using the double wildcard ** so we do not have to explicitly specify every file. SharpDevelop requires the main addin assembly and its associated .addin configuration file to be in the root of the NuGet package. This is why the target for these files defined in the .nuspec file has been left empty.

Now we can generate the NuGet package by running NuGet.exe from the command line:

nuget pack mono.nuspec

On running this command you will see a "Assembly outside lib folder" warning which can safely be ignored since we are not going to be adding this NuGet package to a .NET project. We have now generated a Mono.1.0.nupkg file which is what we will publish to the MyGet gallery.

To publish to MyGet you will need your MyGet account's personal API key. Your API key can be found under Account Settings in the Security section. Copy your API key and in the following command line replace the Your-API-Key with your key:

Running the above command will publish the Mono addin to MyGet. Now anybody using SharpDevelop 5 can install the Mono addin by selecting AddIn Manager from the Tools menu.

image

You can see the list of available addins by clicking the Available section of the dialog. To install the Mono Addin select it and click the Install button. The addin will be available after SharpDevelop is restarted.

Matt Ward is developer at Pebble Code. In his spare time he works on SharpDevelop, which he has been involved with since 2004, and more recently has been working on a NuGet addin for MonoDevelop and Xamarin Studio.

Blog: community.sharpdevelop.net/blogs/mattward Twitter: @sharpdevelop