Posted in Phoenix / Firestorm, Programming, Second Life, tips, Viewers

Compiling your own Viewer

Last updated: 18-Jan-2018.


NOTE: This article assumes that you are a competent software developer who is comfortable with build environments and command windows. Most people will have no need to build their own version of Firestorm.


Back in September 2010 I wrote about the pain of compiling your own Viewer and of the efforts of my friends Mariana and Forestaurora to try to write tutorials on doing it. Fortunately, things have moved on enormously since then and now it is fairly easy to do a private build. A majority of the work is in setting up your build environment.

After encountering and solving a few “gotchas”, I have successfully built Firestorm v5.0.x under Windows 7 64-bit and also Ubuntu 16.04 LTS 64-bit and Ubuntu 17.04 64-bit.

I thought it was worth noting down the “gotchas” I encountered, which is what this article is about.  If it helps just one other person in short-cutting the issues I have had, then this article will have been worth writing.

So without further ado, here they are.

Although this article is primarily about compiling Firestorm, it could prove useful for other third-party viewers too, although your mileage may vary.

Setting up the build environment

Setting up your build environment is something you only need to do once. I will cover Windows and then Linux.

Windows

The follow section applies only to Windows builds.

The build system now uses Microsoft Visual Studio 2013 (VC12) and the Firestorm build instructions are fairly out of date right now, and still reference Visual Studio 2010.

You will need to follow the instructions on the official Linden Lab page to set up your build environment, but after that the Firestorm ones are still relevant, although out of date.
In particular, the location of autobuild has changed and you need the latest version in order to compile successfully (see below).

You can use the IDE of Visual Studio 2015 (VC14) but you must not let it upgrade everything to the VC14 toolset or it won’t compile. However, VS2015 seems to be clever enough to be able to compile in VC12 compatibility mode (although since I have both installed, I don’t know if it’s just calling VS2013).

Note: The free Community editions of VS2013 and VS2015 work just fine – you don’t need a full paid-for version. The license for the Community editions specifically allows use for contribution to Open Source projects.

A few “gotchas” I have encountered are:

Autobuild

You must use NickyD’s version of autobuild, not Linden Labs’ version. If you use the Linden Labs version then it will not compile.

However, the Firestorm build instructions for Windows are out of date, and tell you to clone from http://hg.secondlife.com/autobuild/ which is incorrect. Fortunately the linux build instructions are up to date and they say that the correct location of autobuild is now https://bitbucket.org/NickyD/autobuild-1.0

[Credit: Thank you to Zenny3D for pointing that out in the comments section]

Python version

The build instructions say to install Python. The actual wording is “Version 2.7.1 works with the build scripts”, which is slightly misleading if you are not aware that Python 3 is not backwardly compatible with Python 2

So, make sure you install Python 2 (currently v2.7.14)

Cygwin

This is a minor one. I’m running the 64-bit version of Windows 7, and when I installed Cygwin I specified the install location as d:\cygwin and added d:\cygwin\bin to my path. I was therefore a little surprised to get various errors about being unable to find 'bash' and various other Cygwin components. I double checked the installed components, scratched my head a little, and then suddenly realised that Cygwin had actually decided to overrule me and install to d:\cygwin64 instead. Therefore my path wasn’t pointing to where Cygwin was actually installed but was instead pointing to where I had told it to install to. Once that was fixed the problem went away.

Note: Do not install the GCC compiler or make / cmake. If you do then I have found that Firestorm won’t compile because it picks up Cygwin’s version of cmake rather than the one it needs, and they are not compatible with each other.

7Zip

I encountered an error saying that autobuild was not able to find 7Zip. This isn’t mentioned in any of the build instructions. I have 7Zip installed on my PC but it is not in the path – adding it to the path cured this.
Note: It’s possible that I encountered this issue because I have 7Zip installed.

Linux (Ubuntu)

The following has been tested with Ubuntu 16.04 LTS (64-bit) and Ubuntu 17.04 (64-bit).
Other versions of Linux are supported, but I did not try them.

The only gotcha here is that there are several versions of the build instructions for compiling Firestorm under Linux. They are all subtly different, and look like a bit of a maintenance nightmare – someone isn’t keeping them in step.

I used the build instructions specifically for Ubuntu 16.04 64-bit, and I followed the build instructions to the letter. The only erroneous information in the instructions is occasionally they omit a “sudo” when it is needed.

It’s interesting to note that these instructions currently mandate gcc-4.7, however another version says gcc-4.8 and another says gcc-4.9.

I used a fresh install of Ubuntu 16.04 LTS running as a Virtual Machine (VM) under VirtualBox in order to verify the procedure, and it compiled without a problem with the vanilla build command in the instructions with gcc-4.7, as recommended.

I repeated the procedure with a VM of Ubuntu 17.04 and gcc-4.7, with the same results.

After installing gcc-4.9 and making it the default compiler with Ubuntu 17.04, I was not able to get a successful build. Trying gcc-4.8 was also unsuccessful. Downgrading to gcc-4.7 again then succeeded again.

Therefore I would conclude that you must use gcc-4.7, and not gcc-4.8 or above.

I would also conclude that Ubuntu 17.04 is fine to use and you are not limited to Ubuntu 16.04 LTS.

Update: I recently updated my Ubuntu 17.04 machine to 17.10 and can report that currently there is no g++-4.7 package available for this release, which means you cannot build Firestorm under 17.10 and should stick with 17.04 or 16.04 LTS for now.

Building the libraries

This applies to both Windows and Linux.

Most of the libraries that are required by a Viewer are pulled in automatically by autobuild, with the exception of FModEx and KDU.

FModEx is the library that the Viewers use to play sounds. If you don’t have this library linked in, your Viewer will behave as if you have muted all sounds. The Windows build instructions detail how to build this library (and are also applicable to Linux builds), but do not tell you how to link it in. I will shortly mention how.

KDU (Kakadu) is a library that is to do with rendering JPEG2000 images. Unlike FModEx, the build instructions do not say how to build this, but there is a mention that if you want to use it you must license it yourself and build it yourself. Currently a personal non-commerical license is US$250 which rules it out for me. Fortunately you do not need it as without it the Viewer just uses the OpenJPEG library instead which, although not as fast as KDU, is free.

The trouble is that the predefined build configurations have a bit of an “all or nothing” approach. If you build with ReleaseFS then it tries to link both FModEx and KDU, and fails because KDU is missing. If you build with ReleaseFS_open then it uses neither and you are left with a muted Viewer.

It’s not immediately clear from the build instructions what you are meant to do about this, but the solution is simply to add the switch --fmodex to the end of the configuration command. Whilst you’re there you can also use --avx to turn on AVX optimisations. If you want to use LeapMotion then add --leapmotion (I don’t have one so didn’t bother).

KDU

As mentioned above, it is not feasible to build KDU but it can be omitted.

FModEx

The build instructions for FModEx under Windows are also applicable to Linux, and really should be included in the Linux build instructions too.

You only need to do this once, and FModEx very rarely changes.

Get the source code by executing the following command:

hg clone https://bitbucket.org/NickyD/3p-fmodex

There is a bit of an problem, however. A commit comment in the FModeEx project says “fmodex cannot be downloaded without login anymore, thus the files get downloaded form a private url” (sic).

The trouble is that this “private url” is a local IP address which is not accessible over the internet.
The script will fail to download the fmodapi package, and then terminate with failure (possibly “curl: (7) Failed to connect to 192.168.1.115” or “curl: (28) Connection timed out after 300305 milliseconds”, followed by “ERROR: building configuration”). Either way it doesn’t work.

You must therefore get the libraries yourself.

You will need to go to the FMod website and create a login (if you don’t already have one). However, this will not grant you immediate access, as the FMod Ex project is retired and no longer accessible.

You must then send an email to support, stating your username and asking for access and explaining why (saying you want to build the Second Life Firestorm viewer was sufficient for me), and then they will then grant access and allow you to download the pre-built libraries.
[Credit: Many thanks to Zenny3D for telling me about this in the comments section]

You will have access to the current and all previous versions. As of the time of writing, the most recent version is v4.44.64 but the version you require is v4.44.61 so be sure to download that one.

Download the package appropriate for your targeted operating system. The easiest thing to do is to place it in the same directory as FModEx (usually 3p-fmodex)

You’ll then need to change the value of a variable called URL_BASE in the file build-cmd.sh

Since the package is now in the same directory as the script, you can simply change the value of URL_BASE to "file://./" and it will pick it up.

If you downloaded to a different location, then you can simply adjust the value of URL_BASE accordingly.

Now that you have the package, you can build it. The procedure for building is the same for both Windows and Linux.

From the 3p-fmodex source directory, simply execute the following commands.

autobuild build --all
autobuild package

When built, you will get an output similar to the following (although obviously the values will be different).

packing fmodex
wrote /home/becca/src/3p-fmodex/fmodex-44461-linux-201706151710-r24d.tar.bz2
md5 a1ea2bad2489718f49707e245cfc52a1

You will need these values later on.

Performing a build

Windows

  1. Open a command window in the location of your source folder.
    (Note: A standard Windows command prompt, not the VS2013 command prompt)
  2. Set up the VC12 build environment for 64-bit compilation, which I do with a batch file that executes the following commands
    pushd "\Program Files (x86)\Microsoft Visual Studio 12.0\VC"
    call vcvarsall amd64
    popd
  3. Update autobuild.xml with the fmodex information.
    Note: You only need to do this the first time you do a build, or if you revert the file.

    autobuild installables edit fmodex platform=windows hash=md5 url=file://wrote

    The values of md5 and wrote are the values output when you built FModEx.

  4. Execute the following command
    autobuild -m64 build -c ReleaseFS_open -- --chan MY_CHANNEL --package --fmodex --avx --avx2 -DFS_UPGRADECODES='GUID1,GUID2'

    MY_CHANNEL is the name of your private build, eg. AwesomeSauce, which would result in a version of Firestorm that identifies itself as “Firestorm-AwesomeSauce”. If you omit this, it will use the name of your computer, prefixed by “private-“.
    Do not use spaces or punctuation!

    GUID1 and GUID2 are generated by, for example, the Visual Studio GUID Generator. Make sure to always use the same GUIDs per channel. They identify the installation for upgrades and installation.

    Note: You should use different GUIDs for GUID1 and GUID2

This should then build a 64-bit version of Firestorm for Windows that has sound and works in both Second Life and OpenSim.

You can either run the resulting executable directly for testing purposes, or else run the installer in order to install it properly.

Linux

  1. Open a terminal window in the location of your source directory.
  2. Update autobuild.xml with the fmodex information.
    Note: You only need to do this the first time you do a build, or if you revert the file.

    autobuild installables edit fmodex platform=linux hash=md5 url=file://wrote

    The values of md5 and wrote are the values output when you built FModEx.
    So in the example I posted earlier, this would be.

    autobuild installables edit fmodex platform=linux hash=a1ea2bad2489718f49707e245cfc52a1 url=file:///home/becca/src/3p-fmodex/fmodex-44461-linux-201706151710-r24d.tar.bz2

    (Note the three slashes! The first two are for file:// and then the third is the start of the path)
    Note: You need to specify an absolute path; relative paths don’t seem to work

  3. Execute the following commands
    autobuild -m64 build -c ReleaseFS_open -- --chan MY_CHANNEL --package --fmodex --avx

    MY_CHANNEL is the name of your private build, eg. AwesomeSauce, which would result in a version of Firestorm that identifies itself as “Firestorm-AwesomeSauce”. If you omit this, it will use the name of your computer, prefixed by “private-“.
    Do not use spaces or punctuation!

This should then build a 64-bit version of Firestorm for Linux that has sound and works in both Second Life and OpenSim.

I have verified this by installing a resulting build on a bare metal Linux box running Ubuntu 17.04, an nVidia GTX 780, and the latest nVidia proprietary drivers, and it ran absolutely fine and as fast as the official Firestorm build. And, crucially, had sound.

Summary

The above builds an OpenSim-compatible Firestorm that uses OpenJPEG but also links in FModEx, giving a Firestorm that connects fine to Second Life and has sound. Exactly what I want. The lack of KDU doesn’t make it noticeably slower for me, but I do have a powerful PC so your mileage may vary.

Although the above looks a little daunting, most of the work is in setting up your build environment, and it’s not as bad as it looks.

One slightly annoying aspect of switching between the installed Firestorm and the private build one, is that it seems to clear my cache. That means that every time I switch between them, I rezz as a cloud and have to wait ages for my inventory to load. I haven’t found a way round this yet, although I’m sure one exists. In the meantime it’s something to be aware of.

Anyway, I hope that this post has been useful to someone.


Links and Further Reading

Get source and compile – Official Second Life Viewer build instructions
Firestorm Windows build instructions – build instructions tailored for Windows builds. Currently out of date.
Firestorm 64-bit Ubuntu 16.04 build instructions – build instructions tailored for Ubuntu 16.04
Compiling Firestorm Viewer – a top-level page for all platforms

Advertisements

17 thoughts on “Compiling your own Viewer

  1. What advantage would I gain with a personal build? Can I tweak options and remove limits in debug settings if I build my own? Can I completely remove features I don’t use? Will that “break” the build. If I remove features do I gain some performance? How do we make the custom tweaks? Are there tutorials available for Noobs?

    1. The main advantage is being able to apply your own code changes. On my personal build, you don’t slump when you go AFK, don’t go AFK when minimised, and snapshots to disk are named with the date and time in the filename.

      To make changes, you need to be proficient in the C++ programming language. There are many tutorials around, but it is not a beginners programming language and the learning curve would be pretty steep.

      1. Well that’s me out of the loop then unless there are people who do “requests” for a build with the specific features you want. Would that be matter of deleting code rather than “additions” in most cases? Is that easier?

      2. To be honest, there would be no advantage in removing stuff. For a start, it would be hard for you to know what to remove without breaking something. And the end result wouldn’t give you any advantage as it would be compiled without Kakadu.
        Best bet would be to put a Feature Request in to the Firestorm team if there is a specific feature you want.

      3. My request would be to remove features I don’t need actually. I’m guess that would be a performance gain since the V1 UI seems to be faster than V3 UI. If you remove anything to do with features that are using bandwidth would help (chat and any media or audio for example?).

  2. Hello Becca, just wanted to let you know how to go about grabbing FMOD Ex for windows (Or any platform, really). All you have to do is sign up on fmod.com and contact support at support@fmod.com asking for access to the FMOD Ex downloads. They will ask you what you want to use it for, or you can blatantly specify in your request email as I did. They got back to me within a few days. You can then find your downloads at https://fmod.com/download under the “FMOD Ex” tab, category “FMOD Ex API”. Hope this helps!

      1. I have a question myself, now. Maybe you can help place me in the right direction since I’m not entirely a buff in bash files and google it is so ambiguous I can’t help myself. I’m autobuilding fmod right now, and so far I’ve gotten it to work right up to the very last line of build-cmd.sh , which is “pass”. Cygwin has no installable package named “pass”, there is no “pass.exe” in cygwin’s bin folder, and google has no idea what it is either. Running autobuild package doesn’t work because autobuild-package.xml is missing at this point, so I assume “pass” needs to complete. Any idea what this line accomplishes or how to fix it? Thanks.

      2. I didn’t use Cygwin’s build environment. I suspect you have installed the full development tools of Cygwin and you are picking up the Cygwin version of cmake and autobuild, rather than the Linden Labs’ / Phoenix Firestorm one. I mentioned this as a “gotcha” in the Cygwin section.

        If it’s not that then I’m afraid I don’t know. 😦

      3. I’m using LL’s autobuild. Are you saying there’s a version of it for firestorm instead? Cygwin is not conflicting as far as I’m aware, just that pass is not a command that exists on my system. Maybe I can just rewrite it to accomplish the same thing without pass. Or rename the files myself.

      4. No, I don’t think there is a specific version of autobuild from Firestorm – I’m pretty sure it uses the LL version. It does sound like it is picking up something it should not though. I didn’t experience the issue that you are having and I don’t know how to resolve it I’m afraid. 😦

      5. After about 11 hours of hardships and torment, I’ve finally gotten a complete build. I will admit most of my problems came from a minor discrepancy between LL’s guide and Firestorm’s. It seems LL uses their own autobuild here https://bitbucket.org/lindenlab/autobuild-1.1 (or https://bitbucket.org/lindenlab/autobuild-1.0) where as firestorm (and your guide) works with https://bitbucket.org/NickyD/autobuild-1.0 . Note that only NickyD’s autobuild has -m64 as an argument. The rest have -A 64 which will not compile firestorm correctly. Possibly my fault, but just writing it here in case anyone has similar issues. Thanks to your guide for helping me along the way!

      6. I’ve updated the article with the new info in these comments – specifically on getting the FModEx libraries and also on the caveat about autobuild. Thank you so much for taking the time to share – I really appreciate it. ♥

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s