Exporting and Importing Classes

Definitions

Terminology and Background

• For all of this section it is assumed that the user intends to derive class D from class C and that both classes reside in different DLLs.

• Class C is called DLL-derivable, if class D can be derived from C.

• Class C is called non-sharable if it is marked using __ declspec(notshared) or using the macros NONSHARABLE_CLASS(x) or NONSHARABLE_ STRUCT(x). It is not possible to DLL-derive from a non-sharable class. Otherwise class C is called sharable.

• Class Impedimenta are entities that are emitted by the compiler for any class, but do not have a direct representation in source. Examples of class impedimenta are:

  • the class’s virtual table, or

  • a virtual table’s thunks, or

  • the class’s run-time type information

  Class impedimenta are a key element of the ABI and are identified through ELF symbols. They play a crucial role in enabling polymorphism, RTTI and derivation. This means that they also play a key role in DLL-derivation. In fact, if the Class Impedimenta of class C are not exported, it is not possible to DLL derive from class C or in other words class C is non-sharable.

  This raises the interesting question as to how it is possible to export Class Impedimenta – and thus enable DLL-derivation – if they have no representation in source. The answer to this question is given in The Simple Rule – Sharable Classes section.

• Non-callable exports are Class Impedimenta that are exported. They are often used as synonym for Class Impedimenta in this document.

• A key function is the first non-inline, non-pure virtual function of a class.

Symbian Conventions

Symbian defines macros for exporting and importing classes to hide implementation details:

Class Exporting Rules

This section is a programmer’s summary of the Class Exporting Rules.

Exporting Objects

A function, method of a class or data item is exported by marking its definition using the EXPORT_C macro. A function, method of a class or data item is imported by marking its declaration using the IMPORT_C macro. If an object is declared IMPORT_C and used in code, the linker will error if the definition of this object is not marked EXPORT_C.

Although not required by the class exporting rules it is good practice to

• Always mark all declarations of an exported object with IMPORT_C.

• Never mark objects that are not exported with IMPORT_C.

The Simple Rule – Sharable Classes

The compiler will automatically export allClass Impedimenta, i.e. compiler implementation specific symbols of a class C such as run-time-type-information and virtual tables, that are required to enable DLL-derivation from class C. This means that all classes are by default sharable.

The compiler will not export these symbols for classes that are marked as non-sharable using the macros NONSHARABLE_CLASS(x) or NONSHARABLE_STRUCT(x). Symbian requires that all classes that are derived from a non-sharable class are also marked non-sharable.

Other methods or functions in classes are not automatically exported. They have to be explicitly exported as described in Exporting objects section. Note that a class with virtual functions or virtual may not be DLL-derivable if the conditions in Dll derivable classes section do not hold.

Example: The following example shows the correct syntax to define a class as non-sharable:

NONSHARABLE_CLASS(NonSharableClass) : public BaseClass … 

Note: The Simple Rule has consequences in terms of how many entries are present in DEF files and for shared DEF files. See The Shared DEF File Problem section for more details.

DLL-Derivable Classes

The Simple Rule alone does not guarantee that the class C is DLL-derivable. Depending on the form of the class, additional conditions on exporting members of class C apply. This section outlines the conditions under which class C is DLL-derivable.

C has no Virtual Base

For class C to be DLL-derivable,

• Class C has to be sharable AND

• Every virtual function in C or C's bases that is not overridden in D needs to be exported

• If the class C has no key function, i.e. it only has pure virtuals or inlined virtuals, all bases of C also have to be DLL-derivable.

C has one or several Virtual Bases

For class C to be DLL-derivable,

• Class C has to be sharable AND

• Every virtual function in C or C's bases, virtual or otherwise, needs to be exported AND

• All bases of C, virtual or otherwise, have to be DLL-derivable

Note that link errors will occur if an attempt is made to DLL-derive a class from a base class that is not DLL-derivable.

Boundary cases where Importing and Exporting Symbols is Illegal

Static Symbols and Symbols in Anonymous Namespaces

The following code examples are illegal. But note that not all ABI compliant compilers produce an error or a warning in these situations. This is because the behaviour of export and import is neither defined by the C++ standard, nor is it defined by the ABI for the ARM Architecture. The ABI describes a binary interface and is not concerned with matters of syntax.

Exporting/importing static symbols

Foo.cpp

static IMPORT_C int i; 
static EXPORT_C int j;

Exporting/importing symbols defined in anonymous namespaces

Foo.cpp

namespace { // anonymous namespace
class CTest { 
public: 
    IMPORT_C CTest() { m = 0; }; 
    EXPORT_C int get() { return m; }; 
private: 
    int m; 
};
}

Why is this illegal?

These patterns are illegal because they do not make sense. In fact it is impossible to write any C or C++ client code outside the compile unit in which the imports/exports are defined (e.g. Foo.cpp), that makes use of any statically defined object or an object defined in an anonymous namespace.

This means that it is impossible that any of the exported symbols in the above examples can ever be used by client code that is in another DLL.

What happens if I use these patterns?

The symbols that have been exported will appear in DEF files even though it is impossible to ever link against these symbols. Further the symbols contain a magic number. In the case of symbols in anonymous namespaces the symbol contains a filename and a magic number. For example:

_ZN30_GLOBAL__N__7_ Foo_cpp _5b46ece4 5CTestE

Where Foo_cpp is the filename and 5b46ece4 is the magic number.

The ABI specifies that the magic number is unique. The creation of the magic number depends on the compiler. RVCT creates the magic number by computing a hash from the modification time-stamp of the source file and the top-level source directory.

This means that in essence, the DEF file can never be frozen. If it is frozen, the Symbian toolchain will produce link errors, every single time the symbol changes. For RVCT this will happen every single time the source is changed, or when the user tries to build the source from a different source location than the person who froze the DEF file. Because the exported symbols can never be used, this is a trade-off without benefit.

Classes in Anonymous Namespaces

Because of the Simple Rule (see The Simple Rule – Sharable Classes) any class or structure that is defined in an anonymous namespace will export its Class Impedimenta. This means that by default, the compiler exports symbols that can never be used and cause a series of problems (see Static Symbols and Symbols in Anonymous Namespaces).

The ABI for the ARM architecture v2.0 specifies this behaviour, which is at the very least unintuitive, if not incorrect.

The following code snippet will by default export the Class Impedimenta of the shown class and structure and thus gives rise to the problems described in Static Symbols and Symbols in Anonymous Namespaces section. Note that the exported Class Impedimenta can never be used by any client code. In fact by default, any class that is defined in an anonymous namespace is non-sharable. This is true, regardless of whether Class Impedimenta are exported or not.

Foo.cpp (incorrect)

namespace { // anonymous namespace 
class CTest {…};
struct STest {…};
}

Thus, if you use anonymous namespaces, you must make sure to define them as non-sharable. For example:

Foo.cpp

namespace { // anonymous namespace 
class NONSHARABLE_CLASS(CTest)  {…};
struct NONSHARABLE_STRUCT(STest) {…};
}

This has no draw-backs and avoids any difficulties with frozen DEF files. If you have used anonymous namespaces incorrectly in the past, add the relevant NONSHARABLE macro and re-freeze your DEF file using abld freeze –r armv5. This will re-freeze the DEF file, leaving gaps where the removed exports would have been and thus does not affect binary compatibility.

Auto-Exporting

The Simple Rule only covers compiler implementation specific data items that are required to enable DLL-derivation. However the compiler may also emit and export implementation specific functions. This process is called auto-exporting. This section describes under which circumstances the compiler auto-exports. Note that the compiler also may generate symbols for EABI library functions (these start with __cxa or __eabi).

Constructors and Destructors

For each constructor/destructor in source, the compiler may create several instances of constructors/destructors in the object file, depending on how the constructor and destructors are used. If the constructor/destructor is exported, then all of the generated constructors/destructors are auto-exported. In such a case the following symbols may appear in your DEF file:

_ZN…C1… complete object constructor

_ZN…C2… base object constructor

_ZN…C3… complete allocating constructor

_ZN…D0… deleting destructor

_ZN…D1… complete object destructor

_ZN…D2… base object destructor

Thunks

Under the C++ ABI thunks occur in the presence of multiple inheritance or virtual inheritance. They are used to adjust the this pointer of a class before calling virtual functions. Details can be looked up in the ABI. The export of a thunk is triggered, when a virtual function that needs a thunk is exported. Thunks have the form _ZTh… or _ZTv…

Class Impedimenta

When checking DEF files non-callable exports and other compiler specific symbols may be present. The following list shows what some of these symbols mean:

_ZTV… Virtual Table (VTABLE)

_ZTI… Run-time Type Information (RTTI)

_ZTT… Construction VTABLE

_ZTh… Thunk emitted using multiple inheritance

_ZTv… Thunk emitted using virtual inheritance

ARM ABI Thunk Offset Problem

Symptoms of the ARM ABI Thunk Offset Problem

The "EABI Thunk Offset Problem" is the name that Symbian uses to describe a particular kind of build error which arises when multiple inheritance is used, and the size of a base class is changed. Here is an example of a typical symptom:

MAKEDEF ERROR: 1 Frozen Export(s) missing from object files:
\src\example\MyDLLU.DEF(3) : _ZThn8_N7Derived3fooEv @2
MAKEDEF WARNING: 1 export(s) not yet Frozen in \src\example\MyDLLU.DEF:
..\..\..\EPOC32\BUILD\src\example\group\MyDLL\ARMV5\MyDll{000a0000}.def(7) : _ZThn12_N7Derived3fooEv @6

This shows a problem with a frozen DEF file: the export at ordinal 2 is missing, and a new unfrozen export has been added at ordinal 6. When comparing the two symbols, they look suspiciously similar to each other, and to a third symbol in the DEF file:

_ZN7Derived3fooEv @1 
_ZThn8_N7Derived3fooEv @2        // this one is missing 
_ZThn12_N7Derived3fooEv @6        // this one has appeared

The exports beginning with _ZTh are compiler generated functions called thunks (see Class Impedimenta), and the information between _ZTh and the next underscore is the offset associated with the thunk. Our problem is that for some reason, the offset associated with the thunk has changed from –8 to –12 (the n denotes a negative offset). Note that there is another variant of this problem that involves thunks beginning with _ZTv.

These generated functions are a feature of the Itanium C++ ABI, on which the ABI for the ARM architecture builds upon. Hence the name "ARM ABI Thunk Offset Problem".

What causes this problem?

The problem is caused because the symbol name generated for the thunk contains an offset number. More details can be found in http://www.codesourcery.com/cxx-abi/abi.html under section 5.1.4. This offset may change, when the signature of the base class is changed. For example when a data member is added or removed.

Another condition to trigger the problem needs to hold as well: multiple inheritance with virtual functions in more than one of the base classes. If this condition does not hold, the compiler will not generate a thunk and thus there is no problem.

Note that this is always a Binary Compatibility break, which shows up as a change to symbols in DEF files.

How do I fix it?

There are three choices to fix it:

1. The first option is to refreeze the DEF file: this would be OK if you are not maintaining a frozen interface, and your customers will in any case need to rebuild because of the Binary Compatibility break. The easiest way to refreeze is to delete all of the exports from your existing DEF file, build again, and then use "abld freeze armv5 " to update the DEF file. After updating the DEF file, build again: this time it should build cleanly.

2. The second option is to use the attached script to fix the ABI Thunk Offsets. It expects to read a build log containing the MAKEDEF errors and warnings and will modify the DEF file to replace each missing export with the corresponding unfrozen export. Run the script with no arguments to get further details.

   

   After fixing the DEF file, you will need to rebuild the DLL which uses the DEF file.

3. The last option is that you could change your mind about adding that extra member data. This will only be an option if it is your change which causes the problem: if your supplier has changed the size of a class that they own and caused this problem, then you are forced to change your DEF file.

   If you own the class which has the extra member data, it is worth noting that this change is likely to affect your customers as well. They will have to rebuild because of the BC break. Adding to this they will also see the ABI Thunk Offset Problem if they derive from your class. This includes simple inheritance from a class which shows the problem, if it re-implements any of the virtual functions which require thunks.

   When Symbian breaks compatibility in a way likely to cause this problem, the corresponding entry in the Compatibility Break spreadsheet will say "BC+ Break: Rebuild & Check/Fix Def-File EABI Thunk Offsets".

Tell me the full details: What is a thunk? What causes its offset to change?

In a C++ class hierarchy involving both virtual functions and multiple inheritance, objects can be accessed as though they were several different types. A typical Symbian platform example would be a CBase-derived class which also derives from an M-class, perhaps to provide an observer interface: for example CCoeControl, which derives from both CBase and MObjectProvider.

The virtual functions which can be called on an object depends on the type it currently appears to be.A CCoeControl object can be viewed as a CBase object, in which case it has one set of virtual functions, or as an MObjectProvider, in which case it has another. The compiler constructs separate virtual tables for each of the possible interfaces, and these tables contain information about how to convert back to the underlying CCoeControl object. When converting from a CCoeControl pointer to an MObjectProvider pointer, the compiler will adjust the value of the pointer, so that it points to the "MObjectProvider" part of the object, and not the full CCoeControl object.

The MObjectProvider class defines a pure virtual function MopSupplyObject, which is implemented in CCoeControl. Even when the object is presenting it's MObjectProvider interface, the vtable must use the correct implementation of MopSupplyObject, which expects to be used in the context of a CCoeControl. The solution used by the compiler is to create a virtual function override thunk function which makes any necessary adjustments between the calling context (a pointer to MObjectProvider) and the execution context (a pointer to CCoeControl).

This could have been implemented using the names of the two contexts, but instead the ABI uses the amount by which the this pointer needs to be adjusted to make the switch: this is the offset encoded in the name.

Here is a small example:

eabi_thunk_offset_problem.cpp

#ifndef COUNT 
#define COUNT 1 
#endif 

class Base 
{ 
public: 
    int iBaseMember[COUNT]; 
    virtual ~Base(); 
}; 

class MInterface 
{ 
public: 
    virtual int foo(); 
}; 

class Derived : public Base, public MInterface 
{ 
public: 
    virtual int foo(); 
    int iDerived; 
}; 

class MoreDerived : public Derived 
{ 
public: 
    virtual int foo(); 
    int iMoreDerived; 
}; 

int Derived::foo() { return iDerived; } 
Derived* fun1() { return new Derived; } 
MInterface* fun2() { return new Derived; } 

int MoreDerived::foo() { return iMoreDerived; } 
MoreDerived* fun3() { return new MoreDerived; } 
MInterface* fun4() { return new MoreDerived; }

Compile this with armcc -S eabi_thunk_offset_problem.cpp to get an assembly listing. Compile it again with an extra argument "-DCOUNT=2" to change the size of the base class, and compare the two files: there will be various differences in the code, but also differences in the _ZTh symbols - including the differences used in the "typical symptom" above.

If you use virtual inheritance, then you may see another version of the problem. With virtual inheritance, there are two offsets involved and the thunk symbols will begin with _ZTv. The same symbol may appear in several thunks, each with different offsets.

The Shared DEF File Problem

What is the Problem with Shared DEF Files?

The class exporting rules (see Class Exporting Rules) by default will generate non-callable exports for all classes that are not marked non-sharable in source. Say two DLLs, A and B share one DEF file, in effect implementing similar but different functionality towards the same public interface. Further say, no classes are marked non-sharable. Say there are some classes that are shared between DLL A and DLL B and that these classes have names of the form CShared<xyz>. Classes that are specific to DLL A have names of the form CA<xyz>, classes specific to DLL B have names of the form CB<xyz>. When DLL A is built, DEF file entries for non-callable exports from CShared<xyz> and CA<XYZ> are automatically added to the DEF file. When DLL B is built, exports from CShared<xyz> and CB<XYZ> are added. So in fact the DEF file would be the sum of all non-callable exports from CShared<xyz>, CA<xyz> and CB<XYZ>. It also will contain symbols from functions that are marked for export using EXPORT_C. However, this means that neither A or B can be linked. This is because when A is built, the code linking against the non-callable exports of CB<XYZ> do not exist in A and vice versa.

Use-cases for Fixing Shared DEF Files

Use-Case 1: Polymorphic “Plug-ins”

Several DLLs are built using the same DLL interface (DEF file). Typically the DEF file has very few entries (1 or 2) and is maintained manually. This means that new functions are added by editing the shared DEF file. Also typically no import libraries are needed as the knowledge about the DLL interface is hard-coded into the client code of the "plug-in". The plug-ins do not have to be loaded at run-time. Some are always built but not always included in the ROM.

The Fix:

1. If the shared DEF file is in \epoc32\include\def\EABI then locate the original DEF file by searching all BLD.INF files for the appropriate line in PRJ_EXPORTS

2. Remove all non-callable exports that have caused warnings or errors from the original DEF file.

3. Add NOEXPORTLIBRARY to all MMP files that share that component, ensuring that the build system does NOT try and re-freeze these automatically the next time you build. Otherwise the build system will re-introduce these non-callable exports.

Note:

If you want to use the re-freeze mechanism – say to add a new export, you have to temporarily remove NOEXPORTLIBRARY from the MMP file, then generate a new DEF file by re-building the component, re-freeze, possibly edit (to remove unwanted non-callable exports) and then insert the keyword NOEXPORTLIBRARY into the MMP file again.

Use-Case 2: Polymorphic “Plug-ins” on which Other Components Depend

This is very similar to use-case 1, except that some other component depends on one of the plug-ins. This means that an import library is required.

The Fix: The build structure must be such that

• One MMP file generates the import library from the shared DEF file using the target type IMPLIB. It may be necessary to create a new MMP file which does this.

• All the other MMP files use NOEXPORTLIBRARY as described in use-case 1

Note:

If you want to use the re-freeze mechanism – say to add a new export, you have to temporarily remove NOEXPORTLIBRARY from the MMP file, then generate a new DEF file by re-building the component, re-freeze, possibly edit (to remove unwanted non-callable exports) and then insert the keyword NOEXPORTLIBRARY into the MMP file again.

Use-Case 3: Annotate Classes as Non-sharable

Where a DEF file must be shared between components for whatever reason and none of the above use-cases can be applied, the build would fail for at least one component. An example of this may be a class MyPrivateClass that exists in the debug build (UDEV) of the OS, but not in the release build (UREL).

In such a case all classes that should not contribute to the DEF file, i.e. that are really private to the implementation of a component, must be annotated in the source as NONSHARABLE_CLASS(X) or NONSHARABLE_STRUCT(X). As a result no non-callable exports will be generated for such a class. Say for example, class MyPrivateClass is truly private to a component that must share a DEF file with another component. Then it should be declared:

    NONSHARABLE_CLASS(MyPrivateClass)
    {
    ...
    };

This will prevent the compiler from exporting non-callables for MyPrivateClass. However this means that it is not possible to DLL-derive (for the definition of DLL-derive see Terminology and Background) from MyPrivateClass and that all classes derived from MyPrivateClass must also be marked non-sharable (see Terminology and Background).

Use-Case 4: Optimisation

A consequence of the Simple Rule (see The Simple Rule – Sharable Classes) is that some components may emit entries in their DEF files which are not needed. In the worst case the overhead is 8 bytes of ROM size per class (2 DLL entry points). In the typical case an increase of 4 bytes will occur and in some cases no increase at all.

For code that is private to an implementation, i.e. where it is known that a class would never be used outside of that component, this footprint increase is unnecessary. In order to avoid the footprint increase mark all private classes (and classes derived from them) in the source as NONSHARABLE_CLASS(X) or NONSHARABLE_STRUCT(X) as described for use-case 3.

Use-Case 5: The Build Tools Automatically Ignore Non-callable Exports

For most components the build tools automatically ignore all non-callable exports. This is the case because the build tools know the situations when non-callable exports cannot be needed. Non-callable exports are only needed, if:

• The target type is either DLL, EXEDLL or EXEXP and the MMP file has no NOEXPORTLIBRARY keyword

• If the MMP file contains the DEFFILE keyword and the MMP file has no NOEXPORTLIBRARY keyword

The reason for this is that target types, such as APP, always map directly onto one of the above use-cases. For example the target type APP is an example of use-case 1, i.e. the APP is a DLL that always has the same DEF file. However no other DLL but the APP loader will ever link against this binary, unless its MMP file contains the DEFFILE keyword.

Use-Case 6: Best Practice

Note that it is good practice to avoid unnecessary footprint increases by marking private classes as non-sharable as outlined in use-case 4. Further note, that at some point in the future Symbian may add this to the Symbian Coding Standards or withdraw tools support for some of the cases described above.

Optimisation

This section discusses advantages of marking “private” classes as described in use-cases 4 and 6 in the previous section as non-sharable

Reasons for Optimisation:

• Small footprint saving

• DEF files are more “pretty”, i.e. they will have fewer entries for non-callable exports in it and may have fewer holes in them.

• When changing code that is private to a module as described in use-case 4 and not marked non-sharable, DEF file changes are required when:

  • Renaming a private class

  • Removing a private class

  • Adding a private class

  This makes it harder to maintain DEF files and will ultimately lead to less “pretty” DEF files when Binary Compatibility must be maintained.

• More non-sharable classes mean that it is less likely to have problems with shared DEF files as outlined in use-cases 1 and 2 in the previous section.

When not to use Optimisation:

It is not advisable to use optimisation in the following circumstances, as the build tools suppress non-callable exports automatically in these cases:

• The target type of the component containing private classes is neither DLL, EXEDLL nor EXEXP and no DEFFILE keyword is contained the components MMP file

• The MMP file of the component contains the NOEXPORTLIBRARY keyword.

  If the NOEXPORTLIBRARY keyword has been introduced to work around problems introduced by shared DEF files and the Simple Rule it may be better to remove the NOEXPORTLIBRARY keyword and mark private classes as non-sharable instead.