Dealing with Multiplatform Project Files

open source, code 0 comments suggest edit

Octokit.net targets multiple platforms. This involves a large risk to my sanity. You can see the general approach here in the Octokit directory of our project:

 octokit-projects

Mono gets a project! MonoAndroid gets a project file! Monotuch gets a project file! Everybody gets a project file!

Each of these projects references the same set of .cs files. When I add a file to Octokit.csproj, I have to remember to add that file to the other four project files. As you can imagine, this is easy to forget.

It’s a real pain. So I opened up a feature request on FAKE, the tool we use for our build (more on that later) and asked them for a task that would fail the build if another project file in the same directory was missing a file from the “source” project file. I figured this would be something easy for F# to handle.

The initial response from the maintainer of FAKE, Steffen Forkman, was this:

What you need is a better project system ;-)

Touché!

This problem (along with so many project file merge conflicts) would almost completely go away with file patterns in project files. I’ve been asking for this for a long time (I asked the Visual Studio team for this the day I joined Microsoft, or maybe it was the first month, I don’t recall). There’s a User Voice item requesting this,go vote it up! (Also, go vote up thisplatform restriction issuethat’s affecting Octokit.net as well)

In any case, sorry to say unlimited chocolate fountains don’t exist and I don’t have the project system I want. So let’s deal with it.

A few days later, I get this PR to octokit.net. When I ran the build, I see the following snippet in the build output.

Running build failed.
Error:
System.Exception: Missing files in  D:\Octokit\Octokit-MonoAndroid.csproj:
Clients\OrganizationMembersClient.cs

That’s telling me that somebody forgot to add the class OrganizationMembersClient.cs to the Octokit-MonoAndroid.csproj. Wow! Isn’t open source grand?

A big thanks to Steffen and other members of the FAKE community who pitched in to build a small but very useful feature. In a follow-up post, I’ll write a little bit about why we moved to using FAKE to build Octokit.net.

Update

I opened an issue to take this to the next step. Rather than just verify the project files, I want some way to automatically modify or generate them.

Update 2

FAKE just got even better with the new FixProjects task! For now, we’ve added this as an explicit command.

.\build FixProjects

Over time, we may just integrate this into the Octokit.net build directly.

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

24 responses

  1. Avatar for khalidabuhakmeh
    khalidabuhakmeh November 7th, 2013

    Why not just generate the csproj files from a manifest or pattern file? That way you manage a parent manifest then just create a script file called "generate_projects". The cool thing is, you could do this using Ruby or ScriptCs quiet easily. You could also integrate this generation of csproj files into your build process so that on each check in you could have your build server generate them.

    Just a thought, I could be totally off base here.

  2. Avatar for jdhardy
    jdhardy November 7th, 2013

    Is there a reason you use a project-per-platform rather than one project with different configurations?

  3. Avatar for ozzymcduff
    ozzymcduff November 7th, 2013

    https://github.com/Lundalog... to add files automatically or use a branch of albacore

  4. Avatar for haacked
    haacked November 7th, 2013

    That's probably the next step. I just learned about gyp (https://code.google.com/p/g... and may try that out. The key thing is, I need to make it easy for contributors and they're going to want to add a file using VS. I don't want them having to bother with manifests etc.

  5. Avatar for khalidabuhakmeh
    khalidabuhakmeh November 7th, 2013

    I saw your conversation on twitter about it. I think you could use the current notification process and gyp. A two pronged approach. Good luck!

  6. Avatar for Yves Reynhout
    Yves Reynhout November 7th, 2013

    Dude, csproj is just xml. Xslt all the things, easy.

  7. Avatar for haacked
    haacked November 7th, 2013

    They create their own world of pain.

    As @paulcbetts tells me,

    "Nobody understands them, they're hard to edit, WinRT and WP8 love to fuck with them, it's hard to build multiple configurations at once..."

    "VS always magically gets stuck on the wrong one, so then people are like "Why didn't X platform build??""

  8. Avatar for Mike McG
    Mike McG November 7th, 2013

    (Minor note: First link in the article is broken.)

  9. Avatar for Nik Molnar
    Nik Molnar November 7th, 2013

    Couldn't you just use wild card includes in the .csproj file?

    Thats what Glimpse does with XML that looks like this:

    <compile include="..\**\*.cs" exclude="Properties\AssemblyInfo.cs;">
          <link/>
        </compile>
        <compile include="Properties\AssemblyInfo.cs"/>

    New files are automatically picked up no problem.

    (We also have project specific AssemblyInfo.cs files so we don't include those...)

  10. Avatar for mythz
    mythz November 7th, 2013

    ServiceStack generates all its Custom builds and signed projects with a single node.js script that merges an empty project template with the files references in the master .csproj to generate the custom project .csprojs we use to build from:
    https://github.com/ServiceS...

  11. Avatar for Stuart Hillary
    Stuart Hillary November 7th, 2013

    An approach I use is to place all your common project properties and code files in a separate msbuild project that is imported by each project.

    For example, you could have an msbuild project called Common.props in the same directory as your other project files that contained something like:

    <project defaulttargets="Build" xmlns="http://schemas.microsoft.co...">
    <propertygroup>
    <rootnamespace>Octokit</rootnamespace>
    <assemblyname>Octokit</assemblyname> <defineconstants>TRACE;CODE_ANALYSIS;SIMPLE_JSON_OBJARRAYINTERNAL;SIMPLE_JSON_INTERNAL;NET_45</defineconstants\>
    ...
    </propertygroup>
    <itemgroup>
    <externalcompile include="..\SolutionInfo.cs">
    <link>Properties\SolutionInfo.cs</link>
    </externalcompile>
    <externalcompile include="Clients\ActivitiesClient.cs"/>
    <externalcompile include="Clients\AssigneesClient.cs"/>
    <externalcompile include="Clients\CommitStatusClient.cs"/>
    ...
    </itemgroup>
    <choose>
    <when condition=" '$(Configuration)' == 'Debug' ">
    <propertygroup>
    <defineconstants>$(DefineConstants);DEBUG</defineconstants>
    ...
    </propertygroup>
    </when>
    </choose>

    Instead of having entries in this script for each .cs file you could make a list using a wildcard instead.

    To use this msbuild file in your projects use an import at the start of each csproj

    <import project=".\Common.props"/>

    and an item group further down in the file

    <itemgroup>
    <compile include="@(ExternalCompile)"/>
    </itemgroup>

    Any other properties and items that are common to each project could be moved to this Common.props file.

    The downside to this approach is that if you add a new file in Visual Studio the reference is placed inside the csproj file rather than the Common.props file (if you are running the FAKE task it should still pick up the problem though, you just have to move the reference into just one msbuild script rather than several csproj files).

    EDIT. The xml seems to be a bit screwed above.

  12. Avatar for jdhardy
    jdhardy November 7th, 2013

    Agree on the pain. Single project/multiple configs is how IronPython is built. It has over a dozen projects, so one per platform isn't really workable. Still trying to come up with a better solution.

  13. Avatar for haacked
    haacked November 7th, 2013

    Wait, that works already? Damn!

  14. Avatar for haacked
    haacked November 7th, 2013

    Ah, so I asked @paulcbetts about that and he notes:

    So, Wildcards work, but VS gets really slow since it's scanning directories constantly, *and* any time someone adds a file in the normal way, it gets fucked

    Or rather, "Add new File..." will break your csproj.

    We want contributors to be able to add files to the solution using Visual Studio in the normal way. Does that work?

  15. Avatar for Manuel Grundner
    Manuel Grundner November 8th, 2013

    Sry was not ment to be a reply

  16. Avatar for Manuel Grundner
    Manuel Grundner November 8th, 2013

    What about http://visualstudiogallery....
    Syncronizes the changes in multiple project files.

  17. Avatar for Matt Davey
    Matt Davey November 8th, 2013

    This is the approach that I take in my projects, I've never noticed any particular performance degradation.

    Adding files manually in visual studio does cause issues - seems like VS only has half-hearted support for this scenario, definitely room for improvement!

  18. Avatar for Steffen Forkmann
    Steffen Forkmann November 8th, 2013
  19. Avatar for Nik Molnar
    Nik Molnar November 8th, 2013

    Like @disqus_asZ0mfCnmj:disqus , I've not noticed any particular performance issues - but I also don't doubt that it could happen. (Particularly at Paul's word.)

    VS CERTAINLY will screw up the wildcards if "Add new file" is used. We solve that by having the "main" project that people use being a "normal" project file. The wildcard project files are then only used during the build.

    We separate out our wildcard projects into a separate solution folder, but you could also put them in a *.Build.sln or not have them referenced by any solution at all - just by your build script.

  20. Avatar for Eric Falsken
    Eric Falsken November 8th, 2013

    you can also do this by editing the project file manually to include all of the files from another project file. MSBuild has the tools that you need. It's just Visual Studio that doesn't present all possible options. Creating a PS script to create an msbuild file listing of all of the CS files in your directory is simple simple simple.

  21. Avatar for Eric Falsken
    Eric Falsken November 8th, 2013

    Use a PS script to "extract" the files from a single "source" project and create the manifest automatically and import them into all the other projects with a simple edit to your .csproj file.

  22. Avatar for haacked
    haacked November 8th, 2013

    Ah, I like the source project idea! The FAKE team just added a FixProjects task that solves this problem for us as well. :)

  23. Avatar for Nik Molnar
    Nik Molnar November 8th, 2013

    Awesome! Fake looks pretty good. I'm looking forward to your next post about it. (Wasn't a fan of your last post. :P)

  24. Avatar for nho biet yeu
    nho biet yeu November 12th, 2013

    Is there a reason you use a project-per-platform rather than one project with different configurations? - tai opera mini mien phi