Statically linking to a library that's also "included" in a shared library

Statically linking to a library that's also "included" in a shared library

Post by fia_wrc_fa » Wed, 19 Aug 2009 23:21:11


I've run into a problem with a couple of libraries that my (Linux) app
relies on. My application links to a shared-library version of
Ghostscript. But another part of my application needs to link to
jpeglib (Independent JPEG Group) which I am building as a static
library.

The problem is that Ghostscript builds its _own_ copy of jpeglib which
is statically linked into the Ghostscript shared library file. Now my
application is calling Ghostscript's copy of jpeglib at run-time and
NOT the version that I separately built as a static library resulting
in (as best as I can understand) a memory corruption leading to a
segfault.

How can I solve this problem?
Is there any way to modify the Ghostscript build process so that its
"embedded" copy of jpeglib is not exported?

The platform is CentOS 5.3 with gcc-4.1.2 and glibc-2.5.

The link command line for my app specifies "-ljpeg" before "-lgs" so
that I try and link to my copy of jpeglib first and then to
Ghostscript.

Any help or advice on this situation is appreciated.

Thanks!
 
 
 

Statically linking to a library that's also "included" in a shared library

Post by Thomas Zim » Wed, 19 Aug 2009 23:57:07

Hi


Why don't you rely on Ghostscript's version of libjpeg in the first
place?


You should ask this one the Ghostscript ML.

As a more general note, with GCC 4 it is possible to have functions with
are not exported from a shared library. So in theory there should be a
way.


I've found the following text passage in Ghostscript's install manual [1]

The unix*.mak files explicitly compile and link the JPEG, PNG, and zlib
libraries into the executable. If your Unix system already includes the
PNG and zlib libraries -- probably in /usr/lib/libpng.{a,so} and /usr/
lib/libz.{a,so} -- and you would rather use those copies, change the
definition of SHARE_LIBPNG and SHARE_ZLIB from 0 to 1 in the relevant
unix*.mak file before compiling. Note that if you do this, you will get
non-debug versions of these libraries even if you selected DEBUG in the
makefile. At the insistence of some users, we have also provided the
ability to do this with the JPEG library (SHARE_JPEG), but should not
use it: in order to be compatible with Adobe interpreters (which do not
follow the JPEG standard exactly), Ghostscript has to compile the IJG
code with the non-standard definition

#define D_MAX_BLOCKS_IN_MCU 64

and since shared versions of this library will not have been compiled
this way, you awill get errors on some valid PostScript and PDF input
files.

So simply exchanging the JPEG library doesn't look like a good idea. If
the Ghostscript library does the same as the Ghostscript binary, then
drop the library, fork and exec the binary, and pipe your commands to the
child process.

Thomas

[1] http://www.yqcomputer.com/ ~ghost/doc/AFPL/5.50/Make.htm#Unix_build

 
 
 

Statically linking to a library that's also "included" in a shared library

Post by David Schw » Thu, 20 Aug 2009 21:34:06


There's a technology specifically designed to solve this problem. It's
called 'dynamic linking'. You might want to look into it. If for some
reason you absolutely can't do that, quarantine the problem libraries
with wrapper processes.

DS
 
 
 

Statically linking to a library that's also "included" in a shared library

Post by John Reise » Thu, 20 Aug 2009 23:03:22


It would be useful to understand exactly what causes the corruption.
Usually shared libraries are designed and built to be re-entrant,
and capable of handling multiple "clients" [ghostscript and your code]
at the same time.


See the documentation "info ld" and search for:
--start-group ARCHIVES --end-group
If "--start-group -ljpeg --end-group" appears before (to the left of)
"-lgs", and if "-ljpeg" effectively designates the archive library
libjpeg.a (and not any libjpeg.so), then all possible references
should be satisfied by *.o members retrieved at static bind time
from libjpeg.a instead by from dynamic runtime references to libgs.so.
If you have any libjpeg.so, then "-ljpeg" prefers the shared library
over the archive, so the application will get both libjpeg.so and
libgs.so. [What does "ldd my_app" show?] When multiple shared libraries
provide the same symbols, then the runtime symbol resolver can choose
arbitrarily. [It can be controlled, but it is very esoteric.]
If you have both libjpeg.a and libjpeg.so, and if you want the archive
library, then you must use the full name "libjpeg.a" because
"-ljpeg" prefers libjpeg.so.

--
 
 
 

Statically linking to a library that's also "included" in a shared library

Post by fia_wrc_fa » Fri, 21 Aug 2009 00:07:25


If you are referring to building (my own copy of) libjpeg as a shared
library (.so), I've already tried that but the problem persists.
 
 
 

Statically linking to a library that's also "included" in a shared library

Post by David Schw » Fri, 21 Aug 2009 09:31:09


Then give us details about that case. That's the case where there
should be no problem. (Did you make sure GS uses dynamic linking too?)

DS
 
 
 

Statically linking to a library that's also "included" in a shared library

Post by David Schw » Fri, 21 Aug 2009 09:33:28


This is a common cause of memory corruption and segfaults. Here's a
hypoethetical:

1) His code calls into ghostscript to process a document. He gets back
a structure that GS understands.

2) He asks GS to give him the first image in the document. He gets
back a JPG structure from GS' JPG library.

3) He looks at the image and frees it, calling functions from his JPG
library.

Now, if the JPG structure is different in the two libraries, this can
easily cause corruption and crashes. Consider if one library uses two
'malloc' calls for parts of the structure and the other uses only a
single 'malloc' call for a larger containing structure.

DS
 
 
 

Statically linking to a library that's also "included" in a shared library

Post by David Schw » Sat, 22 Aug 2009 01:38:17

> |The problem is that Ghostscript builds its _own_ copy of jpeglib which


Huh? This is a common cause of corruption. It's quite clear how this
would cause a problem.

For example, suppose his code is:

some_struct=JPG_alloc(...);
// some code
JPG_free(some_struct);

Now, imagine if 'JPG_free' is a macro that translates to 'free
(some_struct)' because that's appropriate for the libjpeg he compiled
against. But suppose the JPG_alloc calls the JPG_alloc in the library
GS was compiled with -- one which does not allocate a JPG structure
with one big call to malloc.

Boom.

That's just one obvious example. Mismatch of header files and library
binaries is a common cause of crashes, memory corruption, and the
like.

DS
 
 
 

Statically linking to a library that's also "included" in a shared library

Post by David Schw » Sat, 22 Aug 2009 13:51:32


Although it's much less likely, even two copies of the exact same code
can get you into trouble.

For example, suppose the library supports two types of JPGs and has a
structure that contains pointers to the serialization function, so if
it's a type 1 JPG, then jpg->serialize is a pointer to the
serialize_jpg1 function and if it's a type 2 JPG, then jpg->serialize
is a pointer to the serialize_jpg2 function.

What happens if he gets a type 1 JPG from GS and passes it to a
library function that does this:

void do_stuff(JPG *jpg)
{
if(jpg->serialize==serialize_jpg1)
{
/* this is a type 1 JPG */
...

Now there are two pointers to two copies of the 'serialize_jpg1'
function, and this test may return false when it should return true.

However, this is getting progressively less likely.

DS