Dynamic libraries and code replacements in Swift

Dynamic library packages

I’ve already revealed an article about building static and dynamic libraries using the Swift compiler, if you do not know what’s a dynamic library or you might be merely a bit extra about how the Swift compiler works, it is best to positively check out that publish first.

This time we’ll focus a bit extra on using the Swift Bundle Supervisor to create our dynamic library merchandise. The setup goes to be similar to the one I’ve created within the loading dynamic libraries at runtime article. First we’ll create a shared library utilizing SPM.

import PackageDescription

let bundle = Bundle(
    title: "TextUI",
    merchandise: [
        .library(name: "TextUI", type: .dynamic, targets: ["TextUI"]),
    ],
    dependencies: [
        
    ],
    targets: [
        .target(name: "TextUI", swiftSettings: [
            .unsafeFlags(["-emit-module", "-emit-library"])
        ]),
    ]
)

The bundle manifest is sort of easy, though there are a number of particular issues that we had so as to add. The very very first thing is that we outlined the product sort as a dynamic library. This can make sure that the proper .dylib (or .so / .dll) binary might be created whenever you construct the goal. 🎯

The second factor is that we would wish to emit our Swift module data alongside the library, we will inform this to the compiler by some unsafe flags. Do not be afraid, these are literally not so harmful to make use of, these flags might be immediately handed to the Swift compiler, however that is it.

Now the supply code for our TextUI library goes to be quite simple.

public struct TextUI 

    public static dynamic func construct() -> String 
        "Whats up, World!"
    

It is only a struct with one static operate that returns a String worth. Fairly easy, besides one factor: the dynamic key phrase. By including the dynamic modifier to a operate (or technique) you inform the compiler that it ought to use dynamic dispatch to “resolve” the implementation when calling it.

We will make the most of the dynamic dispatch in a while, however earlier than we may transfer onto that half, now we have to construct our dynamic library and make it out there for others to make use of. 🔨

When you run swift construct (or run the mission by way of Xcode) it’s going to construct all of the required recordsdata and place them underneath the right construct folder. You too can print the construct folder by operating the swift construct -c launch --show-bin-path (-c launch is for launch builds, we’ll construct the library utilizing the discharge configuration for apparent causes… we’re releasing them). When you checklist the contents of the output listing, it is best to discover the next recordsdata there:

• TextUI.swiftdoc
• TextUI.swiftmodule
• TextUI.swiftsourceinfo
• libTextUI.dylib
• libTextUI.dylib.dSYM

So, what can we do with this construct folder and the output recordsdata? We will want them underneath a location the place the construct instruments can entry the associated recordsdata, for the sake of simplicity we’ll put all the pieces into the /usr/native/lib folder utilizing a Makefile.

PRODUCT_NAME := "TextUI"
DEST_DIR := "/usr/native/lib/"
BUILD_DIR := $(shell swift construct -c launch --show-bin-path)

set up: clear
    @swift construct -c launch
    @set up "$(BUILD_DIR)/lib$(PRODUCT_NAME).dylib" $(DEST_DIR)
    @cp -R "$(BUILD_DIR)/lib$(PRODUCT_NAME).dylib.dSYM" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftdoc" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftmodule" $(DEST_DIR)
    @set up "$(BUILD_DIR)/$(PRODUCT_NAME).swiftsourceinfo" $(DEST_DIR)
    @rm ./lib$(PRODUCT_NAME).dylib
    @rm -r ./lib$(PRODUCT_NAME).dylib.dSYM

uninstall: clear
    
    @rm $(DEST_DIR)lib$(PRODUCT_NAME).dylib
    @rm -r $(DEST_DIR)lib$(PRODUCT_NAME).dylib.dSYM
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftdoc
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftmodule
    @rm $(DEST_DIR)$(PRODUCT_NAME).swiftsourceinfo

clear:
    @swift bundle clear

Now in the event you run make or make set up all of the required recordsdata might be positioned underneath the proper location. Our dynamic library bundle is now prepared to make use of. The one query is how will we eat this shared binary library utilizing one other Swift Bundle goal? 🤔

Linking in opposition to shared libraries

We will construct a model new executable software referred to as TextApp utilizing the Swift Bundle Supervisor. This bundle will use our beforehand created and put in shared dynamic library.

import PackageDescription

let bundle = Bundle(
    title: "TextApp",
    targets: [
        .target(name: "TextApp", swiftSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ], linkerSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ]),
    ]
)

The trick is that we will add some flags to the Swift compiler and the linker, in order that they’ll know that we have ready some particular library and header (modulemap) recordsdata underneath the /usr/native/lib/ folder. We might additionally wish to hyperlink the TextUI framework with our software, so as to do that now we have to cross the title of the module as a flag. I’ve already defined these flags (-L, -I, -l) in my earlier posts so I suppose you are accustomed to them, if not please learn the linked articles. 🤓

import TextUI

print(TextUI.construct())

Our major.swift file is fairly simple, we simply print the results of the construct technique, the default implementation ought to return the well-known “Whats up, World!” textual content.

Are you prepared to exchange the construct operate utilizing native technique swizzling in Swift?

Dynamic technique alternative

After publishing my unique plugin system related article, I’ve bought an e mail from considered one of my readers. Initially thanks for letting me know in regards to the @_dynamicReplacement attribute Corey. 🙏

The factor is that Swift helps dynamic technique swizzling out of the field, though it’s by a personal attribute (begins with an underscore), which suggests it isn’t prepared for public use but (yeah… similar to @_exported, @_functionBuilder and the others), however finally will probably be finalized.

You’ll be able to learn the unique dynamic method replacement pitch on the Swift boards, there’s additionally this great little snippet that incorporates a minimal showcase in regards to the @_dynamicReplacement attribute.

Lengthy story brief, you need to use this attribute to override a customized dynamic technique with your individual implementation (even when it comes from a dynamically loaded library). In our case we have already ready a dynamic construct technique, so if we attempt we will override that the next snippet.

import TextUI

extension TextUI 

    @_dynamicReplacement(for: construct())
    static func _customBuild() -> String 
        "It simply works."
    


print(TextUI.construct()) 

When you alter the major.swift file and run the mission it is best to see that even we’re calling the construct technique, it will be dispatched dynamically and our _customBuild() technique might be referred to as underneath the hood, therefore the brand new return worth.

It really works like a attraction, however can we make this much more dynamic? Is it potential to construct yet another dynamic library and cargo that at runtime, then exchange the unique construct implementation with the dynamically loaded lib code? The reply is sure, let me present you the way to do that. 🤩

import PackageDescription

let bundle = Bundle(
    title: "TextView",
    merchandise: [
        .library(name: "TextView", type: .dynamic, targets: ["TextView"]),
    ],
    targets: [
        .target(name: "TextView", swiftSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ], linkerSettings: [
            .unsafeFlags(["-L", "/usr/local/lib/"]),
            .unsafeFlags(["-I", "/usr/local/lib/"]),
            .unsafeFlags(["-lTextUI"]),
        ]),
    ]
)

Identical SPM sample, we have simply created a dynamic library and we have used the TextUI as a shared library so we will place our TextUI extension into this library as an alternative of the TextApp goal.

To date we have created 3 separated Swift packages shared the TextUI module between the TextApp and the TextView packages as a pre-built dynamic library (utilizing unsafe construct flags). Now we’ll lengthen the TextUI struct inside our TextView bundle and construct it as a dynamic library.

import TextUI

extension TextUI 

    @_dynamicReplacement(for: construct())
    static func _customBuild() -> String 
        "It simply works."
    

We are able to use an identical makefile (to the earlier one) or just run the swift construct -c launch command and duplicate the libTextView.dylib file from the construct listing by hand.

When you run this code utilizing Linux or Home windows, the dynamic library file might be referred to as libTextView.so underneath Linux and libTextView.dll on Home windows.

So simply place this file underneath your property listing we’ll want the complete path to entry it utilizing the TextApp’s major file. We will use the dlopen name to load the dylib, this can exchange our construct technique, then we shut it utilizing dlclose (on the supported platforms, extra on this later…).

import Basis
import TextUI


print(TextUI.construct())


let dylibPath = "/Customers/tib/libTextView.dylib"
guard let dylibReference = dlopen(dylibPath, RTLD_LAZY) else 
    if let err = dlerror() 
        fatalError(String(format: "dlopen error - %s", err))
    
    else 
        fatalError("unknown dlopen error")
    

defer 
    dlclose(dylibReference)



print(TextUI.construct())

The wonderful thing about this strategy is that you do not have to fiddle with further dlsym calls and unsafe C pointers. There’s additionally a pleasant and detailed article about Swift and native technique swizzling, this focuses a bit extra on the emitted replacements code, however I discovered it a really nice learn.

Sadly there may be yet another factor that now we have to speak about…

Drawbacks & conclusion

Dynamic technique alternative works good, this strategy is behind SwiftUI reside previews (or dlsym with some pointer magic, however who is aware of this for certain..). Anyway, all the pieces appears to be like nice, till you begin involving Swift courses underneath macOS. What’s flawed with courses?

Seems that the Goal-C runtime will get concerned underneath macOS in the event you compile a local Swift class. Simply compile the next instance supply and try it utilizing the nm instrument.

class A 

Below macOS the output of nm will include traces of the Goal-C runtime and that’s greater than sufficient to trigger some troubles through the dylib shut course of. Seems in case your library incorporates the ObjC runtime you will not have the ability to really shut the dylib, it doesn’t matter what. ⚠️

Previous to Mac OS X 10.5, solely bundles may very well be unloaded. Beginning in Mac OS X 10.5, dynamic libraries can also be unloaded. There are a
couple of circumstances wherein a dynamic library won’t ever be unloaded: 1) the principle executable hyperlinks in opposition to it, 2) an API that doesn’t help
unloading (e.g. NSAddImage()) was used to load it or another dynamic library that is determined by it, 3) the dynamic library is in dyld’s
shared cache.

When you check out man 3 dlclose you may get a number of extra hints in regards to the causes, plus you can too test the source code of the Goal-C runtime, if you wish to see extra particulars.

Anyway I believed this ought to be talked about, as a result of it might probably trigger some hassle (solely on macOS), however all the pieces works simply nice underneath Linux, so if you’re planning to make use of this strategy on the server aspect, then I would say it’s going to work simply positive. It is not protected, nevertheless it ought to work. 😈

Oh, I virtually overlook the hot-reload performance. Properly, you’ll be able to add a listing or file watcher that may monitor your supply codes and if one thing adjustments you’ll be able to re-build the TextView dynamic library then load the dylib once more and name the construct technique if wanted. It is comparatively straightforward after you have tackled the dylib half, as soon as you determine the smaller particulars, it really works like magic. 🥳