Compiling your own Viewer

Note: Although this article was first published in February 2014, I continually update it.

Last updated: 19-Jun-2017.


NOTE: This article assumes that you are a competent software developer who is comfortable with build environments and command windows. Most people will have no need to build their own version of Firestorm.


Back in September 2010 I wrote about the pain of compiling your own Viewer and of the efforts of my friends Mariana and Forestaurora to try to write tutorials on doing it. Fortunately, things have moved on enormously since then and now it is fairly easy to do a private build. A majority of the work is in setting up your build environment.

After encountering and solving a few “gotchas”, I have successfully built Firestorm v5.0.x under Windows 7 64-bit and also Ubuntu 16.04 LTS 64-bit and Ubuntu 17.04 64-bit.

I thought it was worth noting down the “gotchas” I encountered, which is what this article is about.  If it helps just one other person in short-cutting the issues I have had, then this article will have been worth writing.

So without further ado, here they are.

 

Although this article is primarily about compiling Firestorm, the majority could just as easily apply to the Official Viewer, or any of the other Third Party Viewers – they are unlikely to be Firestorm-specific.

Setting up the build environment

Setting up your build environment is something you only need to do once. I will cover Windows and then Linux.

Windows

The follow section applies only to Windows builds.

The build system now uses Microsoft Visual Studio 2013 (VC12) and the Firestorm build instructions are fairly out of date right now, and still reference Visual Studio 2010.

You will need to follow the instructions on the official Linden Lab page to set up your build environment, but after that the Firestorm ones are still relevant, although out of date.
In particular, the location of autobuild has changed and you need the latest version in order to compile successfully.

You can use the IDE of Visual Studio 2015 (VC14) but you must not let it upgrade everything to the VC14 toolset or it won’t compile. However, VS2015 seems to be clever enough to be able to compile in VC12 compatibility mode (although since I have both installed, I don’t know if it’s just calling VS2013).

Note: The free Community editions of VS2013 and VS2015 work just fine – you don’t need a full paid-for version. The license for the Community editions specifically allows use for contribution to Open Source projects.

Python version

The first “gotcha” was that the build instructions said to install Python. The actual wording was “Version 2.7.1 works with the build scripts”, which is slightly misleading as I read that to mean that that v2.7.1 had been explicitly tested but anything greater than that would also be fine. This is not the case. I installed v3.3.4 which installs by default to c:\Python33 on Windows. When I got to the point of building FModEx, I got the following build error:

D:\projects\firestorm\3p-fmodex>autobuild build --all
Traceback (most recent call last):
File "D:\bin\autobuild\bin\autobuild", line 38, in 
from autobuild import autobuild_main
File "D:\bin\autobuild\autobuild\autobuild_main.py", line 36
print parser.format_help()
      ^
SyntaxError: invalid syntax

A friend suggested that I tried Python 2.7 as the instructions had said v2.7.1 had been tested, so I installed v2.7.6 (the latest in the 2.7 series at the time) and the problem went away. I have to say that I am surprised by the break of backward compatibility, so am calling this a “gotcha”. So, in summary: you must use v2.7.x for it to work.

Cygwin

This is a minor one. I’m running the 64-bit version of Windows 7, and when I installed Cygwin I specified the install location as d:\cygwin and added d:\cygwin\bin to my path. I was therefore a little surprised to get various errors about being unable to find 'bash' and various other Cygwin components. I double checked the installed components, scratched my head a little, and then suddenly realised that Cygwin had actually decided to overrule me and install to d:\cygwin64 instead. Therefore my path wasn’t pointing to where Cygwin was actually installed (as it was pointing to where I had told it to install to, rather than where it had decided to install to). Once that was fixed the problem went away.

Note: Do not install the GCC compiler or make / cmake. If you do then I have found that Firestorm won’t compile because it picks up Cygwin’s version of cmake rather than the one it needs, and they are not compatible with each other.

7Zip

Another build failure I encountered was autobuild not being able to find 7Zip. This isn’t mentioned in any of the build instructions. I have 7Zip installed on my PC but it is not in the path – adding it to the path cured this.
Note: It’s possible that I encountered this issue because I have 7Zip installed.

Linux (Ubuntu)

The following has been tested with Ubuntu 16.04 LTS (64-bit) and Ubuntu 17.04 (64-bit).
Other versions of Linux are supported, but I did not try them.

The only gotcha here is that there are several versions of the build instructions for compiling Firestorm under Linux. They are all subtly different, and look like a bit of a maintenance nightmare – someone isn’t keeping them in step.

I used the build instructions specifically for Ubuntu 16.04 64-bit, and I followed the build instructions to the letter. The only erroneous information in the instructions is occasionally they omit a “sudo” when it is needed.

It’s interesting to note that these instructions currently mandate gcc-4.7, however another version says gcc-4.8 and another says gcc-4.9.

I used a fresh install of Ubuntu 16.04 LTS running as a Virtual Machine (VM) under VirtualBox in order to verify the procedure, and it compiled without a problem with the vanilla build command in the instructions with gcc-4.7, as recommended.

 

I repeated the procedure with a VM of Ubuntu 17.04 and gcc-4.7, with the same results.

After installing gcc-4.9 and making it the default compiler with Ubuntu 17.04, I was not able to get a successful build. Trying gcc-4.8 was also unsuccessful. Downgrading to gcc-4.7 again then succeeded again.

Therefore I would conclude that you must use gcc-4.7, and not gcc-4.8 or above.

I would also conclude that Ubuntu 17.04 is fine to use and you are not limited to Ubuntu 16.04 LTS.

 

Libraries

Most of the libraries that are required by a Viewer are pulled in automatically by autobuild, with the exception of FModEx and KDU.

FModEx is the library that the Viewers use to play sounds. If you don’t have this library linked in, your Viewer will behave as if you have muted all sounds. The Windows build instructions detail how to build this library (and are also applicable to Linux builds), but do not tell you how to link it in. I will shortly mention how.

KDU (Kakadu) is a library that is to do with rendering JPEG2000 images. Unlike FModEx, the build instructions do not say how to build this, but there is a mention that if you want to use it you must license it yourself and build it yourself. Currently a personal non-commerical license is US$250 which rules it out for me. Fortunately you do not need it as without it the Viewer just uses the OpenJPEG library instead which, although not as fast as KDU, is free.

The trouble is that the predefined build configurations have a bit of an “all or nothing” approach. If you build with ReleaseFS then it tries to link both FModEx and KDU, and fails because KDU is missing. If you build with ReleaseFS_open then it uses neither and you are left with a muted Viewer.

It’s not immediately clear from the build instructions what you are meant to do about this, but the solution is simply to add the switch --fmodex to the end of the configuration command. Whilst you’re there you can also use --avx to turn on AVX optimisations. If you want to use LeapMotion then add --leapmotion (I don’t have one so didn’t bother).

KDU

As mentioned above, it is not feasible to build KDU but it can be omitted.

FModEx

The build instructions for FModEx under Windows are also applicable to Linux, and really should be included in the Linux build instructions too.

You only need to do this once, and FModEx very rarely changes.

Get the source code by executing the following command:

hg clone https://bitbucket.org/NickyD/3p-fmodex

There is a bit of an problem, however. A commit comment in the FModeEx project says “fmodex cannot be downloaded without login anymore, thus the files get downloaded form a private url” (sic).

The trouble is that this “private url” is a local IP address which is not accessible. It’s referenced by a variable called URL_BASE in the file build-cmd.sh
The script will fail to download the fmodapi package, and then terminate with failure.

Going to the FMod website and creating a login is no good, as even with a login the FMod Ex project is retired and no longer accessible.

The solution is to either change the value of URL_BASE to a correct URL, or else manually download the package file locally and add a path to it. The easiest way to do this is to place it in the same directory as FModEx (usually 3p-fmodex), and then change the value of URL_BASE to "file://./"

As for where to download it from, I can’t help you there. But I would caution you to make sure that the md5sum checksums match to make sure that the file hasn’t been tampered with. Fortunately the script does this checking for you.

What I did for Linux was to google for the actual checksum which brought me to a Slackware page where I could download the package from. This didn’t work for Windows though. With Windows, I no longer know where I got my ‘good’ version from.

The procedure for building is the same for both Windows and Linux.

From the 3p-fmodex source directory, simply execute the following commands.

autobuild build --all
autobuild package

When built, you will get an output similar to the following (although obviously the values will be different).

packing fmodex
wrote /home/becca/src/3p-fmodex/fmodex-44461-linux-201706151710-r24d.tar.bz2
md5 a1ea2bad2489718f49707e245cfc52a1

You will need these values later on.

Performing a build

Windows

  1. Open a command window in the location of your source folder.
    (Note: A standard Windows command prompt, not the VS2013 command prompt)
  2. Set up the VC12 build environment for 64-bit compilation, which I do with a batch file that executes the following commands
    pushd "\Program Files (x86)\Microsoft Visual Studio 12.0\VC"
    call vcvarsall amd64
    popd
  3. Update autobuild.xml with the fmodex information.
    Note: You only need to do this the first time you do a build, or if you revert the file.

    autobuild installables edit fmodex platform=windows hash=md5 url=file://wrote

    The values of md5 and wrote are the values output when you built FModEx.

  4. Execute the following command
    autobuild -m64 build -c ReleaseFS_open -- --chan MY_CHANNEL --package --fmodex --avx --avx2 -DFS_UPGRADECODES='GUID1,GUID2'

    MY_CHANNEL is the name of your private build, eg. MyBuild, which would result in a version of Firestorm that identifies itself as “Firestorm-MyBuild”. If you omit this, it will use the name of your machine, prefixed by “private-“.
    GUID1 and GUID2 are generated by, for example, the Visual Studio GUID Generator. Make sure to always use the same GUIDs per channel. They identify the installation for upgrades and installation.

This should then build a 64-bit version of Firestorm for Windows that has sound and works in both Second Life and OpenSim.

You can either run the resulting executable directly for testing purposes, or else run the installer in order to install it properly.

Linux

  1. Open a terminal window in the location of your source directory.
  2. Update autobuild.xml with the fmodex information.
    Note: You only need to do this the first time you do a build, or if you revert the file.

    autobuild installables edit fmodex platform=linux hash=md5 url=file://wrote

    The values of md5 and wrote are the values output when you built FModEx.
    So in the example I posted earlier, this would be.

    autobuild installables edit fmodex platform=linux hash=a1ea2bad2489718f49707e245cfc52a1 url=file:///home/becca/src/3p-fmodex/fmodex-44461-linux-201706151710-r24d.tar.bz2

    (Note the three slashes! The first two are for file:// and then the third is the start of the path)

  3. Execute the following commands
    autobuild -m64 build -c ReleaseFS_open -- --chan MY_CHANNEL --package --fmodex --avx

    MY_CHANNEL is the name of your private build, eg. MyBuild, which would result in a version of Firestorm that identifies itself as “Firestorm-MyBuild”. If you omit this, it will use the name of your machine, prefixed by “private-“.

This should then build a 64-bit version of Firestorm for Linux that has sound and works in both Second Life and OpenSim.

I have verified this by installing a resulting build on a bare metal Linux box running Ubuntu 17.04, an nVidia GTX 780, and the latest nVidia proprietary drivers, and it ran absolutely fine and as fast as the official Firestorm build. And, crucially, had sound.

Summary

The above builds an OpenSim-compatible Firestorm that uses OpenJPEG but also links in FModEx, giving a Firestorm that connects fine to Second Life and has sound. Exactly what I want. The lack of KDU doesn’t make it noticeably slower for me, but I do have a powerful PC so your mileage may vary.

Although the above looks a little daunting, most of the work is in setting up your build environment, and it’s not as bad as it looks.

One slightly annoying aspect of switching between the installed Firestorm and the private build one, is that it seems to clear my cache. That means that every time I switch between them, I rezz as a cloud and have to wait ages for my inventory to load. I haven’t found a way round this yet, although I’m sure one exists. In the meantime it’s something to be aware of.

Anyway, I hope that this post has been useful to someone.


Links and Further Reading

Get source and compile – Official Second Life Viewer build instructions
Firestorm Windows build instructions – build instructions tailored for Windows builds. Currently out of date.
Firestorm 64-bit Ubuntu 16.04 build instructions – build instructions tailored for Ubuntu 16.04
Compiling Firestorm Viewer – a top-level page for all platforms

Advertisements

7 thoughts on “Compiling your own Viewer

  1. What advantage would I gain with a personal build? Can I tweak options and remove limits in debug settings if I build my own? Can I completely remove features I don’t use? Will that “break” the build. If I remove features do I gain some performance? How do we make the custom tweaks? Are there tutorials available for Noobs?

    • The main advantage is being able to apply your own code changes. On my personal build, you don’t slump when you go AFK, don’t go AFK when minimised, and snapshots to disk are named with the date and time in the filename.

      To make changes, you need to be proficient in the C++ programming language. There are many tutorials around, but it is not a beginners programming language and the learning curve would be pretty steep.

      • Well that’s me out of the loop then unless there are people who do “requests” for a build with the specific features you want. Would that be matter of deleting code rather than “additions” in most cases? Is that easier?

      • To be honest, there would be no advantage in removing stuff. For a start, it would be hard for you to know what to remove without breaking something. And the end result wouldn’t give you any advantage as it would be compiled without Kakadu.
        Best bet would be to put a Feature Request in to the Firestorm team if there is a specific feature you want.

      • My request would be to remove features I don’t need actually. I’m guess that would be a performance gain since the V1 UI seems to be faster than V3 UI. If you remove anything to do with features that are using bandwidth would help (chat and any media or audio for example?).

  2. Pingback: The pain of compiling your own Viewer | Becca's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s