IPP Build Updates : 2012.06.26

This page documents changes & improvements to the build scripts motivated by Ubuntu 12.04 and related issues.

gcc 4.3.6 and the --as-needed option

Ubuntu 12.04 has two major changes which affect the IPP build system: it uses gcc 4.6.3 with some specific configuration choices, and it uses Perl 5.14. The IPP machines at MHPCC use gcc 4.3.2 and Perl 5.8.8, and other build attempts have probably been successful with later version of both gcc and Perl. However, there are some specific features in these two that caused problems with our code.

The Ubuntu installation of gcc includes a build-time option which forces the gcc to pass the option --as-needed to the linker (ldd). This was initially introduced in development versions of Ubuntu 11.04, then disabled as the change caused too much grief for the Ubuntu developers, then finally turned on for real in Ubuntu 11.10. Some links related to this change are given below. The purpose of this change was to allow the linker to skip symbols which were not needed (eg, a function defined in some library but not actually called by any of the code). The change has three effects:

First, programs cannot rely on indirect linking. That is to say reliance in the link on libraries which link against other libraries. It is now necessary to explicitly list all required libraries. This issue was the primary driver of the change by Ubuntu and Debian, but it is not a problem with IPP code (we always explicitly link already).

Second, shared libraries now need to be listed in the correct order: if a program requires symbols from libA and that in turn requires symbols from libB, it is necessary to link in the order gcc -o main main.c -lA -lB. This is because linking is done in a single pass. If libA were listed second (after libB), then the needed symbols from B would not be loaded because the linker would not realize they were needed by another library. This is the reason the Ubuntu 11.04 release did not include the change to gcc before it shipped: the developers had not resolved all such link order problems.

The third problem is, I believe, a bug in the linker. I do not have a definitive test case, but the following case was observed: A program (addstar_client) does not use symbols from libmysql or libpng, but it uses symbols from libm. Running the link without -lmysql or -lpng succeeds. Running the link with -lpng -lm claims missing symbols from libm are needed, while linking -lm -lpng works. This is backwards, as the linker should be able to find all needed symbols from libm since it is last on the list. Similar behavior is seen with -lmysql. Installing a local version of libmysql avoids this error, but a local version of libpng does not.

The --as-needed option can be overruled by using the gcc option --no-as-needed. This is best done by setting LDFLAGS to include -Wl,--no-as-needed.

I have addressed these issues above by the following steps:

  • psconfig checks for the gcc version 4.6.3 from Ubuntu (gcc --version reports both 4.6.3 and Ubuntu). In this case, it adds -Wl,--no-as-needed to LDFLAGS for the user.
  • psbuild now has the option -no-as-needed, which is tells psbuild to pass --enable-no-as-needed to the configure scripts. This in turn sets the LDFLAGS in the build via autogen scripts. This option should be used if --no-as-needed is required but the case is not caught by psconfig

Here are links describing the --as-needed option and issues:

Perl 5.14 Issues

Perl 5.14 has two changes which affect the IPP modules (actually, external modules used by IPP). First, the function ref has changed its behavior: the list of allowed results has been expanded to include Regexp. This breaks Params::Validate < 0.96. I have updated that module to deal with this situation. (This also forced me to upgrade Module::Build to 0.3601 since Params::Validate claims it requires it. Does it really? I cannot easily upgrade to Module::Build 0.40 since that in turn has other dependencies we do not currently meet.).

Second, older perl c-code (not sure of the correct term, basically C-code that links against perl) has a set of values with names like sv_undef. These were changed in perl 5 to PL_sv_*, but support for the old names was maintained for a while. The old names disappeared sometime around Perl 5.12. I needed to update DBI and DBD::mysql to handle these changes.

gcc pedantic issues

The latest version of gcc has a number of new pedantic warnings, which in turn become build errors when using -Werror. The most obvious is the requirement that a variable be used, not just defined and set. To deal with this, I have added code in some places with empty brackets (eg, if (foo) { } ). This is a bit ugly, but it allows us to keep some values which we may want to use.

Another important pedantic change is stricter handling of enums. This has forced us to use the PS_DATA_XX values for comparisons with metadata items rather than the PS_TYPE_XX entries (and vice versa in some cases). Another change comes from the ps*ErrorCode values, which are defined as enums separately for each IPP component (eg, psLib, psModules). The ranges are maintained by us to be distinct, but the compiler does not realize these different enums are related. So, it complains when we compare the result of e.g., psErrorLast to errors from psLib, psModules, etc. To deal with this, I have cast the error to an int instead of a psErrorCode.

To allow for a build with these as warnings, I have added the -debug-build option to psbuild.

auto-detect the hidden paths

I have update the psbuild script pschecklibs (which builds the external libraries) and the Ohana configuration script to check for hidden search paths used by the linker and the C preprocessor. This is driven by the fact that Ubuntu and other system builders have taken to putting required libraries (like libm) in hidden, architecture dependent locations. (This is largely driven by the 64bit vs 32bit library problems). To find these hidden paths, I have added code to examine the output of ld --verbose and cpp --verbose. This fix probably will help with some builds, but may be a bit fragile.