Skip to content

Instantly share code, notes, and snippets.

@jiacai2050
Forked from loderunner/osx-ld.md
Created March 28, 2022 14:29
Show Gist options
  • Select an option

  • Save jiacai2050/195ea85897815b5280086423d6ec7057 to your computer and use it in GitHub Desktop.

Select an option

Save jiacai2050/195ea85897815b5280086423d6ec7057 to your computer and use it in GitHub Desktop.
potential blog posts

ld – Wading through Mac OS X linker hell

Intro

Friend: I tried looking at static linking in Mac OS X and it seems nearly impossible. Take a look at this http://stackoverflow.com/a/3801032

Me: I have no idea what that -static flag does, but I'm pretty sure that's not how you link to a library. Let me RTFM a bit.

Minutes later...

Me: I'm gonna have to write this stuff down.

Reading the fantastic manuals

FANTASTIC!

clang

First things first, gcc isn't the default compiler in Mac OS X anymore. Since Xcode 5, the Apple developer toolchain uses clang, and gcc only aliases to clang.

NOTE: All shell outputs in this document were produced with the default bash on Mac OS X El Capitan 10.11.3 with Xcode 7.2 installed.

$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.3.0
Thread model: posix

So let's look into clang.

$ man clang

clang is a C, C++, and Objective-C compiler which encompasses preprocessing,
parsing, optimization, code generation, assembly, and linking. Depending on
which high-level mode setting is passed, Clang will stop before doing a full
link. While Clang is highly integrated, it is important to understand the stages
of compilation, to understand how to invoke it. These stages are:

Driver
    The clang executable is actually a small driver which controls the overall
    execution of other tools such as the compiler, assembler and linker.
    Typically you do not need to interact with the driver, but you transparently
    use it to run the other tools.

Preprocessing
    This stage handles tokenization of the input source file, macro expansion,
    #include expansion and handling of other preprocessor directives. The output
    of this stage is typically called a ".i" (for C), ".ii" (for C++), ".mi"
    (for Objective-C) , or ".mii" (for Objective-C++) file.

Parsing and Semantic Analysis
    This stage parses the input file, translating preprocessor tokens into a
    parse tree. Once in the form of a parser tree, it applies semantic analysis
    to compute types for expressions as well and determine whether the code is
    well formed. This stage is responsible for generating most of the compiler
    warnings as well as parse errors. The output of this stage is an "Abstract
    Syntax Tree" (AST).

Code Generation and Optimization
    This stage translates an AST into low-level intermediate code (known as
    "LLVM IR") and ultimately to machine code. This phase is responsible for
    optimizing the generated code and handling target-specific code generation.
    The output of this stage is typically called a ".s" file or "assembly" file.

    Clang also supports the use of an integrated assembler, in which the code
    generator produces object files directly. This avoids the overhead of
    generating the ".s" file and of calling the target assembler.

Assembler
    This stage runs the target assembler to translate the output of the compiler
     into a target object file. The output of this stage is typically called a
     ".o" file or "object" file.

Linker
    This stage runs the target linker to merge multiple object files into an
    executable or dynamic library. The output of this stage is typically called
    an "a.out", ".dylib" or ".so" file.

So static linking is an option of the linker stage of compilation. So let's try ld's manual.

ld

$ man ld

OPTIONS
   Options that control the kind of output
     -execute    The default. Produce a mach-o main executable that has file
                 type MH_EXECUTE.
     
     -dylib      Produce a mach-o shared library that has file type MH_DYLIB.
     
     -bundle     Produce a mach-o bundle that has file type MH_BUNDLE.
     
     -dynamic    The default. Implied by -dylib, -bundle, or -execute
     
     -static     Produces a mach-o file that does not use the dyld. Only used
                 building the kernel.
     

Now we have it. -static does not control how the output links to libraries; it controls the type of output produced by the linker. In this case, -static is used to indicate that no dynamic linking should occur with this binary. Ever. The only file to ever need this option is the kernel.

So how do we tell the linker which library to link against?

$ man ld

Options that control libraries
  -lx         This option tells the linker to search for libx.dylib or libx.a in
              the library search path. If string x is of the form y.o, then that
              file is searched for in the same places, but without prepending
              `lib' or appending `.a' or `.dylib' to the filename.
 
  -Ldir       Add dir to the list of directories in which to search for
              libraries. Directories specified with -L are searched in the order
              they appear on the command line and before the default search
              path. In Xcode4 and later, there can be a space between the -L and
              directory.

These are the only two options you need to know to link to most (static or dynamic) libraries. More can be said about lazy linking, frameworks, and many other aspects of linking and libraries but these topics are outside of the scope of this post.

-l is used to tell the linker the name of the libraries to look into for symbols. If you decide to compile a program that uses functions from libpng, you would add -lpng to the arguments passed to clang.

-L is used to tell the linker where to look for the library files. ld maintains a list of directories to search for a library to use. The default library search path is /usr/lib then /usr/local/lib. The -L option will add a new library search path.

At this point is it important to point out that, if the linker finds both a .a (static) and .dylib (dynamic) file in the search paths, it will always choose the dynamic library.

Example

UNIX man pages are infamous for being dry, to the point and absolutely impossible to read for first-timers. Let's try an example to see how all this fits together.

Basic libcurl program

We'll be using a very simple C program that uses libcurl to retrieve the content at http://google.com.

#include <curl/curl.h>
#include <stdio.h>

int main(int argc, char** argv) {
    CURL *curl;
    CURLcode res;

    curl = curl_easy_init();

    if (curl) {
        curl_easy_setopt(curl, CURLOPT_URL, "http://google.com");
        curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
        res = curl_easy_perform(curl);

        if(res != CURLE_OK)
            fprintf(stderr, "curl_easy_perform() failed: %s\n",
                curl_easy_strerror(res));

        curl_easy_cleanup(curl);
    }

    return 0;
}

Dynamically linking against default libcurl

libcurl is installed by default with Mac OS X, so we don't need to provide headers or libraries to compile this program.

$ ls -1 /usr/include/curl
curl.h
curlbuild.h
curlrules.h
curlver.h
easy.h
mprintf.h
multi.h
stdcheaders.h
typecheck-gcc.h

$ ls -1 /usr/lib/libcurl*
/usr/lib/libcurl.3.dylib
/usr/lib/libcurl.4.dylib
/usr/lib/libcurl.dylib

So let's try to compile our program!

$ clang -o curl_example-system curl_example.c

OH NO! We get a bunch of "Undefined symbols" errors. Why?

We simply forgot to specify that we wanted to link against libcurl at the linker stage. Let's add the suitable -l option.

$ clang -o curl_example-system -lcurl curl_example.c 
$ ./curl_example-system 
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="fr"
><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta 
content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop=
"image"><title>Google</title>...

Great! We have a working program that links against libcurl.

So did we link dynamically or statically? When we inspected the contents of /usr/lib, we only found .dylib files, so we can expect that this executable links against dynamically. We can confirm this by using otool.

otool is a command-line tool to display different parts of object files. In our case, we want to see which libraries the executable links against, so we use the -L option.

$ otool -L curl_example-system 
curl_example-system:
    /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 8.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

We can see our executable links against libcurl and libSystem.

That went pretty smoothly, so let's see what happens when we try to link against a custom build of libcurl

Dynamically linking against custom libcurl

First of all, let's build our own version of curl.

$ git clone https://github.com/curl/curl.git
$ cd curl
$ ./buildconf
$ ./configure
$ make

Supposing everything went smoothly, you now have your own freshly brewed curl and libcurl.

$ cd ..
$ ls -1 curl/lib/.libs
libcurl.4.dylib
libcurl.a
libcurl.dylib
...

So inside curl/lib/.libs we have our libraries, both static and dynamic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment