IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:Ubuntu_1804_Install

Building IPP on Ubuntu 18.0 : 2018.10.22

When building on Ubuntu systems, it is necessary to install some packages at the system level (to provide the compiler, etc):

  • need to make sure you have the 'build-essential' package, which supplies libc, standard headers, as well as gcc: apt-get install build-essential
  • It is also necessary to install the following libraries and build-related tools: 
      apt-get install libx11-dev libncurses-dev pkg-config automake libtool libmysqlclient-dev subversion

If you encounter build problems related to some of the necessary libraries, it can be helpful to install the 'developer' versions of the libraries via apt-get as well. This installs the C header files needed to build against those libraries. In the past, I have had a problem in which installing the IPP version of libz broke my Ubuntu installation of evince (by providing a version of libz which did not have gzopen64 included). If you encounter trouble, it might be safest to install the developer version of libz as well. Here is a list of packages to consider: 

apt-get install libjpeg-dev
apt-get install libgsl-dev
apt-get install libpng-dev
apt-get install zlib1g-dev

When building on Ubuntu (since 12.04), it is necessary to supply the -no-as-needed option to psbuild. Note: when building on Gentoo, it is now necessary to specify the IPP tagset since the Ubuntu build is currently picked up by psbuild as the default. Here are the two commands to use:

  • Ubuntu: psbuild -extbuild -clean -rebuild -dev -optimize -no-as-needed
  • Gentoo: psbuild -extbuild -clean -rebuild -dev -optimize ipp-3.1

gcc and the --as-needed option

The version of gcc supplied with Ubuntu since 12.04 (gcc version >= 4.5) includes a build-time option which forces gcc to pass the option --as-needed to the linker (ldd). This was initially introduced in development versions of Ubuntu 11.04, then disabled as the change caused too much grief for the Ubuntu developers, then finally turned on for real in Ubuntu 11.10. Some links related to this change are given below. The purpose of this change was to allow the linker to skip symbols which were not needed (eg, a function defined in some library but not actually called by any of the code). The change has three effects:

First, programs cannot rely on indirect linking. That is to say reliance on libraries which link against other libraries. It is now necessary to explicitly list all required libraries. This issue was the primary driver of the change by Ubuntu and Debian, but it is not a problem with IPP code (we always explicitly link already).

Second, shared libraries now need to be listed in the correct order: if a program requires symbols from libA and that in turn requires symbols from libB, it is necessary to link in the order gcc -o main main.c -lA -lB. This is because linking is done in a single pass. If libA were listed second (after libB), then the needed symbols from B would not be loaded because the linker would not realize they were needed by another library. This is the reason the Ubuntu 11.04 release did not include the change to gcc before it shipped: the developers had not resolved all such link order problems.

The third problem is, I believe, a bug in the linker. I do not have a definitive test case, but the following case was observed: A program (addstar_client) does not use symbols from libmysql or libpng, but it uses symbols from libm. Running the link without -lmysql or -lpng succeeds. Running the link with -lpng -lm claims missing symbols from libm are needed, while linking -lm -lpng works. This is backwards, as the linker should be able to find all needed symbols from libm since it is last on the list. Similar behavior is seen with -lmysql. Installing a local version of libmysql avoids this error, but a local version of libpng does not.

The --as-needed option can be overruled by using the gcc option --no-as-needed. This is best done by setting LDFLAGS to include -Wl,--no-as-needed.

In the IPP build system, this issue is addressed by the following steps:

  • psconfig checks for the gcc version >4.6.3 from Ubuntu (gcc --version reports both a version >= 4.5 and Ubuntu). In this case, it adds -Wl,--no-as-needed to LDFLAGS for the user.
  • psbuild now has the option -no-as-needed, which is tells psbuild to pass --enable-no-as-needed to the configure scripts. This in turn sets the LDFLAGS in the build via autogen scripts. This option should be used if --no-as-needed is required but the case is not caught by psconfig.

Here are links describing the --as-needed option and issues:

Other issues encountered for Ubuntu 18.04

Each new release of gcc adds additional tests and warnings for C-code concerns. Since the IPP build uses -Wall and -Werror flags, any C-build warning raises and error and breaks the build. For the current version of gcc (7.3.0), a warning was added to cases calls to snprint in which the compiler is concerned that the buffer limit may result in truncation. For example: 

char line[10];
int i;
// i assigned to some arbitrary value
snprintf (line, 10, "TESTLINE%d", i);

In this case, gcc does not know if 'i' will be only 1 digit (in which case the output fits in the 10 chars of 'line') or more (in which case the output will be truncated). Note that the purpose of snprintf is to truncate the length of the output to fit in the buffer, so there is no danger of a buffer overflow. Instead, the result is perhaps an unintended text truncation. However, this situation is sometimes (a) avoided by a limited allowed range of values for i (which gcc cannot know) or uninteresting (since the user may not care that a message was truncated). But, since a warning is an error, this situation breaks the IPP build.

To avoid such cases, I have added a version of snprintf which does not trigger the warning : snprintf_nowarn (actually, one is added to Ohana and another to psLib, ps_snprintf_nowarn). This function tests the output from snprintf and then ignores a problem in a way that lets gcc think we are carefully handling the error.

Perl modules in Ubuntu 18.04

The Perl modules have evolved so much over time that it is not possible to have a single set of modules which will work on the older (e.g., gentoo) builds and more recent builds. The current psconfig directory now contains a new tagset for Ubuntu which supplies a complete, internally-compatible, set of Perl modules. This means it is necessary to distinguish the tagsets for the Gentoo and Ubuntu builds. At the moment, the Ubuntu tagset is the default, so the builder must supply the tageset for the Gentoo builds:

  • Ubuntu: psbuild -extbuild -clean -rebuild -dev -optimize -no-as-needed
  • Gentoo: psbuild -extbuild -clean -rebuild -dev -optimize ipp-3.1

(see Ubuntu_1204_Build_Issues for other issues affecting the build)

Last modified 8 years ago Last modified on Oct 22, 2018, 9:04:30 AM
Note: See TracWiki for help on using the wiki.