Skip to content

Instantly share code, notes, and snippets.

@aSemy

aSemy/kn_cpp_compilation.md

Last active July 30, 2024 22:06
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save aSemy/0ec58b89f79702ea9822f198ccef0dc3 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save aSemy/0ec58b89f79702ea9822f198ccef0dc3 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. aSemy revised this gist Aug 18, 2023. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion kn_cpp_compilation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    Kotlin/Native has a utility, `run_konan`, that can be used to compile C/C++ code.
    Kotlin/Native has a utility, [`run_konan`](https://github.com/JetBrains/kotlin/blob/v1.9.0/kotlin-native/HACKING.md#running-clang-the-same-way-kotlinnative-compiler-does), that can be used to compile C/C++ code.

    I've written two Gradle tasks, `RunKonanClangTask` and `RunKonanClangTask`, that can be used to compile [JoltC](https://github.com/michal-z/zig-gamedev/blob/c517effe8c476c5c75ccbf735440fdabc35e42e5/libs/zphysics/libs/README.md). A lot of credit goes to https://github.com/michal-z for writing C wrappers for [Jolt](https://github.com/jrouwe/JoltPhysics)!

  2. aSemy revised this gist Aug 18, 2023. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion kn_cpp_compilation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -34,9 +34,10 @@ This Gist is very scrappy, and is missing a lot of information.
    ### Example usage

    ```kts
    // build.gradle.kts

    import org.jetbrains.kotlin.konan.util.DependencyDirectories.localKonanDir

    // build.gradle.kts
    plugins {
    kotlin("multiplatform")
    }
  3. aSemy revised this gist Aug 18, 2023. 1 changed file with 5 additions and 3 deletions.
    8 changes: 5 additions & 3 deletions kn_cpp_compilation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -7,7 +7,7 @@ This Gist is very scrappy, and is missing a lot of information.
    * Kotlin 1.9.0
    * I've only tested it on Windows.
    * The Kotlin target must be set manually - is it possible to make this automatic?
    * The `konan_run` file must be set manually - is it possible to make this automatic?
    * While the `.konan` dir can be retrieved from a KGP util function, the path to `konan_run` file must be set manually (e.g. `kotlin-native-prebuilt-windows-x86_64-1.9.0`) - is it possible to make this automatic?
    * I had to remove `#define private public` and amend the Jolt source code to make everything public, otherwise compilation failed in `sstreams`
    * I moved `JoltPhysicsC.h` into a `JoltC/` dir, and updated `#include "JoltPhysicsC.h"` to `#include <JoltC/JoltPhysicsC.h>`

    @@ -34,6 +34,8 @@ This Gist is very scrappy, and is missing a lot of information.
    ### Example usage

    ```kts
    import org.jetbrains.kotlin.konan.util.DependencyDirectories.localKonanDir

    // build.gradle.kts
    plugins {
    kotlin("multiplatform")
    @@ -84,15 +86,15 @@ val konanClangCompile by tasks.registering(RunKonanClangTask::class) {
    "-D" + "JPH_ENABLE_ASSERTS",
    )

    runKonan.set(file("C:/Users/Me/.konan/kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    runKonan.set(localKonanDir.resolve("kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    }

    val konanLink by tasks.registering(RunKonanLinkTask::class) {
    group = project.name

    libName.set("zphysics")
    objectFiles.from(konanClangCompile)
    runKonan.set(file("C:/Users/Me/.konan/kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    runKonan.set(localKonanDir.resolve("kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    }
    ```

  4. aSemy revised this gist Aug 18, 2023. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion kn_cpp_compilation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -263,7 +263,7 @@ abstract class RunKonanLinkTask @Inject constructor(
    @get:Internal
    val workingDir: DirectoryProperty =
    objects.directoryProperty().convention(
    // workaround for
    // workaround for https://github.com/gradle/gradle/issues/23708
    objects.directoryProperty().fileValue(temporaryDir)
    )

  5. aSemy revised this gist Aug 18, 2023. 1 changed file with 53 additions and 2 deletions.
    55 changes: 53 additions & 2 deletions kn_cpp_compilation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,13 +1,64 @@
    Kotlin/Native has a utility, `run_konan`, that can be used to compile C/C++ code.

    I've written two Gradle tasks, `RunKonanClangTask` and `RunKonanClangTask`, that can be used to compile Jolt.
    I've written two Gradle tasks, `RunKonanClangTask` and `RunKonanClangTask`, that can be used to compile [JoltC](https://github.com/michal-z/zig-gamedev/blob/c517effe8c476c5c75ccbf735440fdabc35e42e5/libs/zphysics/libs/README.md). A lot of credit goes to https://github.com/michal-z for writing C wrappers for [Jolt](https://github.com/jrouwe/JoltPhysics)!

    This Gist is very scrappy, and is missing a lot of information. I've only tested it on Windows.
    This Gist is very scrappy, and is missing a lot of information.

    * Kotlin 1.9.0
    * I've only tested it on Windows.
    * The Kotlin target must be set manually - is it possible to make this automatic?
    * The `konan_run` file must be set manually - is it possible to make this automatic?
    * I had to remove `#define private public` and amend the Jolt source code to make everything public, otherwise compilation failed in `sstreams`
    * I moved `JoltPhysicsC.h` into a `JoltC/` dir, and updated `#include "JoltPhysicsC.h"` to `#include <JoltC/JoltPhysicsC.h>`

    ### Project layout

    ```
    ├── buildSrc/
    │ ├── src/main/kotlin/
    │ │ ├── RunKonanClangTask.kt
    │ │ ├── RunKonanLinkTask.kt
    │ │ └── utils.kt
    │ ├── build.gradle.kts
    │ └── settings.gradle.kts
    ├── src/main/cpp/
    │ ├── Jolt/ (Jolt C++ sources)
    │ └── JoltC/ (from zphysics)
    ├── zphysics.def
    ├── build.gradle.kts
    └── settings.gradle.kts
    ```

    <!-- https://tree.nathanfriend.io/?s=(%27options!(%27fancy7~fullPath!false~trailFgSlash7~rootDot7)~4!(%274!%278SG*3kotlI5ClangH5LFkH*0utilsB*8-*923cpp*E6E%20C%2B%2B%204s%7D*EC6from%20A%7D2A.def28-29%27)~version!%271%27)*20-.gradleBs0%20%202%5Cn3sGmaI4source5*0RunKonan6%2F%20%7B7!true8build9settFgs-AzphysicsB.ktEJoltFinGrc%2FHTaskBIF%2F%01IHGFEBA987654320-* -->

    ### Example usage

    ```kts
    // build.gradle.kts
    plugins {
    kotlin("multiplatform")
    }

    description = "Kotlin/Native C-interop bindings for Jolt Physics, via zig-gamedev zphysics"

    kotlin {
    targets.withType<KotlinNativeTarget>().configureEach {
    compilations.getByName("main") {
    cinterops {
    register("zphysics") {
    defFileProperty.set(file("zphysics.def"))
    }
    }
    }
    binaries {
    staticLib()
    }
    }
    }

    //region lots of utilities for downloading JoltC
    // ...
    //endregion

    val konanClangCompile by tasks.registering(RunKonanClangTask::class) {
    group = project.name
  6. aSemy revised this gist Aug 18, 2023. 1 changed file with 51 additions and 39 deletions.
    90 changes: 51 additions & 39 deletions kn_cpp_compilation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,52 @@
    Kotlin/Native has a utility, `run_konan`, that can be used to compile C/C++ code.

    I've written two Gradle tasks, `RunKonanClangTask` and `RunKonanClangTask`, that can be used to compile Jolt.

    This Gist is very scrappy, and is missing a lot of information. I've only tested it on Windows.

    ### Example usage

    ```kts
    // build.gradle.kts

    val konanClangCompile by tasks.registering(RunKonanClangTask::class) {
    group = project.name

    kotlinTarget.set(KonanTarget.MINGW_X64)

    sourceFiles.from(
    layout.projectDirectory
    .dir("src/main/cpp/")
    .asFileTree
    .matching {
    include("**/*.cpp")
    exclude("**/*_Tests*")
    }
    )

    includeDirs.from(layout.projectDirectory.dir("src/main/cppHeaders/"))

    arguments.addAll(
    "-std=c++17",
    "-fno-sanitize=undefined",
    "-D" + "JPH_CROSS_PLATFORM_DETERMINISTIC",
    "-D" + "JPH_ENABLE_ASSERTS",
    )

    runKonan.set(file("C:/Users/Me/.konan/kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    }

    val konanLink by tasks.registering(RunKonanLinkTask::class) {
    group = project.name

    libName.set("zphysics")
    objectFiles.from(konanClangCompile)
    runKonan.set(file("C:/Users/Me/.konan/kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    }
    ```

    ### RunKonanClangTask

    ```kotlin
    import execCapture
    import org.gradle.api.DefaultTask
    @@ -114,6 +163,8 @@ abstract class RunKonanClangTask @Inject constructor(
    }
    ```

    ### RunKonanLinkTask

    ```kotlin
    import execCapture
    import org.gradle.api.DefaultTask
    @@ -204,45 +255,6 @@ abstract class RunKonanLinkTask @Inject constructor(
    }
    ```

    ```kts
    // build.gradle.kts

    val konanClangCompile by tasks.registering(RunKonanClangTask::class) {
    group = project.name

    kotlinTarget.set(KonanTarget.MINGW_X64)

    sourceFiles.from(
    layout.projectDirectory
    .dir("src/main/cpp/")
    .asFileTree
    .matching {
    include("**/*.cpp")
    exclude("**/*_Tests*")
    }
    )

    includeDirs.from(layout.projectDirectory.dir("src/main/cppHeaders/"))

    arguments.addAll(
    "-std=c++17",
    "-fno-sanitize=undefined",
    "-D" + "JPH_CROSS_PLATFORM_DETERMINISTIC",
    "-D" + "JPH_ENABLE_ASSERTS",
    )

    runKonan.set(file("C:/Users/Me/.konan/kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    }

    val konanLink by tasks.registering(RunKonanLinkTask::class) {
    group = project.name

    libName.set("zphysics")
    objectFiles.from(konanClangCompile)
    runKonan.set(file("C:/Users/Me/.konan/kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    }
    ```

    ### Utils

    ```kotlin
  7. aSemy created this gist Aug 18, 2023.
    282 changes: 282 additions & 0 deletions kn_cpp_compilation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,282 @@
    ```kotlin
    import execCapture
    import org.gradle.api.DefaultTask
    import org.gradle.api.file.*
    import org.gradle.api.model.ObjectFactory
    import org.gradle.api.provider.ListProperty
    import org.gradle.api.provider.Property
    import org.gradle.api.provider.Provider
    import org.gradle.api.tasks.*
    import org.gradle.api.tasks.PathSensitivity.NAME_ONLY
    import org.gradle.api.tasks.PathSensitivity.RELATIVE
    import org.gradle.process.ExecOperations
    import org.jetbrains.kotlin.konan.target.KonanTarget
    import org.jetbrains.kotlin.util.parseSpaceSeparatedArgs
    import javax.inject.Inject

    /**
    * Compile C/C++ source files using the
    * [`run_konan`](https://github.com/JetBrains/kotlin/blob/v1.9.0/kotlin-native/HACKING.md#running-clang-the-same-way-kotlinnative-compiler-does)
    * utility.
    */
    abstract class RunKonanClangTask @Inject constructor(
    private val exec: ExecOperations,
    private val fs: FileSystemOperations,
    private val objects: ObjectFactory,
    ) : DefaultTask() {

    /** Destination of compiled `.o` object files */
    @get:OutputDirectory
    val outputDir: Provider<Directory>
    get() = objects.directoryProperty().fileValue(temporaryDir.resolve("output"))

    /** C and C++ source files to compile to object files */
    @get:InputFiles
    @get:PathSensitive(RELATIVE)
    abstract val sourceFiles: ConfigurableFileCollection

    /** Directories that include `.h` header files */
    @get:InputFiles
    @get:PathSensitive(RELATIVE)
    abstract val includeDirs: ConfigurableFileCollection

    /** Path to the (platform specific) `run_konan` utility */
    @get:InputFile
    @get:PathSensitive(NAME_ONLY)
    abstract val runKonan: RegularFileProperty

    /** Kotlin target platform, e.g. `mingw_x64` */
    @get:Input
    abstract val kotlinTarget: Property<KonanTarget>

    @get:Input
    @get:Optional
    abstract val arguments: ListProperty<String>

    @get:Internal
    abstract val workingDir: DirectoryProperty

    @TaskAction
    fun compile() {
    val workingDir = workingDir.asFile.getOrElse(temporaryDir)
    val kotlinTarget = kotlinTarget.get()

    // prepare output dirs
    val sourcesDir = workingDir.resolve("sources")
    val headersDir = workingDir.resolve("headers")
    val compileDir = workingDir.resolve("compile")

    fs.sync {
    from(sourceFiles)
    into(sourcesDir)
    }
    fs.sync {
    from(includeDirs)
    into(headersDir)
    }
    fs.delete { delete(compileDir) }
    compileDir.mkdirs()

    // prepare args file
    val sourceFilePaths = sourcesDir.walk()
    .filter { it.extension in listOf("cpp", "c") }
    .joinToString("\n") { it.invariantSeparatorsPath }

    compileDir.resolve("args").writeText(/*language=text*/ """
    |--include-directory ${headersDir.invariantSeparatorsPath}
    |${arguments.getOrElse(emptyList()).joinToString("\n")}
    |-c $sourceFilePaths
    """.trimMargin()
    )

    // compile files
    val compileResult = exec.execCapture {
    executable(runKonan.asFile.get())
    args(parseSpaceSeparatedArgs(
    "clang clang $kotlinTarget @args"
    ))
    workingDir(compileDir)
    }

    // verify output
    val outputLog = workingDir.resolve("compileResult.log").apply { writeText(compileResult.output) }
    logger.lifecycle("compilation output log: file://${outputLog.invariantSeparatorsPath}")
    compileResult.assertNormalExitValue()

    // move compiled files to output directory
    fs.sync {
    from(compileDir) {
    include("**/*.o")
    }
    into(outputDir)
    }
    }
    }
    ```

    ```kotlin
    import execCapture
    import org.gradle.api.DefaultTask
    import org.gradle.api.file.*
    import org.gradle.api.model.ObjectFactory
    import org.gradle.api.provider.Property
    import org.gradle.api.provider.Provider
    import org.gradle.api.tasks.*
    import org.gradle.process.ExecOperations
    import org.jetbrains.kotlin.util.parseSpaceSeparatedArgs
    import javax.inject.Inject

    /**
    * Link compiled C/C++ source files using the
    * [`run_konan`](https://github.com/JetBrains/kotlin/blob/v1.9.0/kotlin-native/HACKING.md#running-clang-the-same-way-kotlinnative-compiler-does)
    * utility.
    */
    abstract class RunKonanLinkTask @Inject constructor(
    private val exec: ExecOperations,
    private val fs: FileSystemOperations,
    objects: ObjectFactory,
    ) : DefaultTask() {

    /** The linked file */
    @get:OutputFile
    val compiledLib: Provider<RegularFile>
    get() = workingDir.file(libFileName)

    /** All `.o` object files that will be linked */
    @get:InputFiles
    @get:PathSensitive(PathSensitivity.NAME_ONLY)
    abstract val objectFiles: ConfigurableFileCollection

    /** Path to the (platform specific) `run_konan` utility */
    @get:InputFile
    @get:PathSensitive(PathSensitivity.NAME_ONLY)
    abstract val runKonan: RegularFileProperty

    @get:Input
    abstract val libName: Property<String>

    private val libFileName: Provider<String>
    get() = libName.map { "lib${it}.a" }

    @get:Internal
    val workingDir: DirectoryProperty =
    objects.directoryProperty().convention(
    // workaround for
    objects.directoryProperty().fileValue(temporaryDir)
    )

    @TaskAction
    fun compile() {
    val workingDir = workingDir.asFile.get()
    val libFileName = libFileName.get()

    // prepare output dir
    fs.delete { delete(workingDir) }
    workingDir.mkdirs()

    // prepare args file
    val sourceFilePaths = objectFiles
    .asFileTree
    .matching { include("**/*.o") }
    .joinToString("\n") { it.invariantSeparatorsPath }

    workingDir.resolve("args").writeText(/*language=text*/ """
    |-rv
    |$libFileName
    |$sourceFilePaths
    """.trimMargin()
    )

    // compile files
    val linkResult = exec.execCapture {
    executable(runKonan.asFile.get())
    args(parseSpaceSeparatedArgs(
    "llvm llvm-ar @args"
    ))
    workingDir(workingDir)
    }

    val outputLog = workingDir.resolve("linkResult.log").apply { writeText(linkResult.output) }
    logger.lifecycle("compilation output log: file://${outputLog.invariantSeparatorsPath}")

    linkResult.assertNormalExitValue()
    }
    }
    ```

    ```kts
    // build.gradle.kts

    val konanClangCompile by tasks.registering(RunKonanClangTask::class) {
    group = project.name

    kotlinTarget.set(KonanTarget.MINGW_X64)

    sourceFiles.from(
    layout.projectDirectory
    .dir("src/main/cpp/")
    .asFileTree
    .matching {
    include("**/*.cpp")
    exclude("**/*_Tests*")
    }
    )

    includeDirs.from(layout.projectDirectory.dir("src/main/cppHeaders/"))

    arguments.addAll(
    "-std=c++17",
    "-fno-sanitize=undefined",
    "-D" + "JPH_CROSS_PLATFORM_DETERMINISTIC",
    "-D" + "JPH_ENABLE_ASSERTS",
    )

    runKonan.set(file("C:/Users/Me/.konan/kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    }

    val konanLink by tasks.registering(RunKonanLinkTask::class) {
    group = project.name

    libName.set("zphysics")
    objectFiles.from(konanClangCompile)
    runKonan.set(file("C:/Users/Me/.konan/kotlin-native-prebuilt-windows-x86_64-1.9.0/bin/run_konan.bat"))
    }
    ```

    ### Utils

    ```kotlin
    import org.gradle.process.ExecOperations
    import org.gradle.process.ExecResult
    import org.gradle.process.ExecSpec
    import java.io.ByteArrayOutputStream

    fun ExecOperations.execCapture(
    configure: ExecSpec.() -> Unit,
    ): ExecCaptureResult {

    val (result, output) = ByteArrayOutputStream().use { os ->
    exec {
    isIgnoreExitValue = true
    standardOutput = os
    errorOutput = os
    configure()
    } to os.toString()
    }

    return if (result.exitValue != 0) {
    ExecCaptureResult.Error(output, result)
    } else {
    ExecCaptureResult.Success(output, result)
    }
    }


    sealed class ExecCaptureResult(
    val output: String,
    private val result: ExecResult,
    ) : ExecResult by result {
    class Success(output: String, result: ExecResult) : ExecCaptureResult(output, result)
    class Error(output: String, result: ExecResult) : ExecCaptureResult(output, result)
    }
    ```