Nokee Labs Blog

Devlog #9

Tue, 18 Apr 2023 00:00:00 +0000

Hello, Nokee users! We hope you all had a fantastic New Year and are off to a great start in 2023. We apologize for the radio silence over the past few months. Rest assured; our team has been hard at work making significant improvements to the Nokee plugins. This devlog will provide an overview of the progress made since the last devlog. Our primary focus was the Xcode build adapter.

Improved correctness

We’ve considerably enhanced the correctness of build input detection from Xcode projects. This improvement ensures that the build tasks are more accurately detected when out-of-date, resulting in a more reliable build process and reducing the likelihood of issues caused by stale build outputs.

Configurable build settings

To work around the slow performance of querying xcodebuild for effective build settings and overcome the temporary limitation of the inability to extract all build settings from Xcode projects, we’ve added the ability to configure build settings for each Xcode target build task. This feature allows you to fine-tune your build process and helps to streamline build configuration times.

Preliminary build caching

We’ve enabled build caching on all Xcode target build tasks, which improves build times. However, this feature is still in its preliminary stages. The build cache is not relocatable, meaning we can only share the build cache when performing the build in the exact same working directory. We’ll continue refining the build input detection in upcoming updates.

Expanded testing to Signal and Wikipedia iOS applications

To ensure the reliability and effectiveness of our improvements, we’ve expanded our testing to cover the Signal and Wikipedia iOS applications. This broadened scope of testing will enable us to catch and fix potential issues more effectively, leading to a more stable and robust build adapter between Xcode and Gradle.

Reduced reliance on `xcodebuild`

Lastly, we’ve made efforts to improve the overall performance of the Xcode build adapter by reducing our dependence on xcodebuild. This change should result in faster build times for your projects.

We appreciate your patience during this period of growth and development. Our team is dedicated to continuously improving the Nokee plugins, and we’re excited about the progress made so far. As always, we welcome your feedback and suggestions to help make Nokee even better. Here’s to a productive and successful 2023!

Devlog #8

Wed, 14 Dec 2022 00:00:00 +0000

This week’s development log (devlog) includes work and experiments done since the last devlog. Our primary focus was the correctness of Xcode build adapter.

Xcode Build Adapter Correctness

Before tackling support for Xcode scheme, we want to improve the correctness of our build adapter.

But what does correctness imply? In short, our build adapter should delegate to Xcode when it deems the previous work to be out-of-date. It means any meaningful change to the build configuration and input/output files.

It raises the question, what is a meaningful change? Gradle is efficient at detecting changes to input/output files. The main issue is finding out what those files are. As for the build configuration, we want to do better than simply snapshotting the pbxproj file. For this reason, we are distilling the information found inside Xcode project so Gradle can understand them and perform its outstanding up-to-date checking. Thanks to the great modelling we did a couple of months ago, we can reuse the same infrastructure to encode an Xcode build spec containing all the Gradle annotation required.

At the moment, we merged the obvious out-of-date scenario around non-declared inputs/outputs for shell script build phases. We are working on improved correctness.

Devlog #7

Tue, 25 Oct 2022 00:00:00 +0000

This week’s development log (devlog) includes work and experiments done since the last devlog. We split our primary development time between improvement to the Xcode build adapter and our internal testing infrastructure.

Xcode Build Adapter Improvements

While preparing the work for Xcode scheme support, we retouched some of the internal implementations to make it more testable, a big subject in the Gradle world. We are finishing the reimplementation of the Xcode models, the backbone of the build adapter. Performance bottlenecks were showing up during the configuration time. The refactor will unblock future work such as helping users to compare and merge Xcode projects, e.g. the dreadful pbxproj merge conflict. It will also help with malformed or badly crafted projects. The new implementation keeps track of already assigned Global IDs. Sometimes, we need to edit an existing Xcode project without breaking any potential reference to its elements. For this reason, we need to keep track of the PBXObject’s Global ID to reuse it when serializing the project again. Finally, properties that we believe do not affect the build process are generally not mapped in our models. However, we are skeptical that this may cause unintended side effects when editing projects as we remove those properties. The new implementation preserves all properties and their declared order, minimizing potential issues with Xcodebuild.

Internal Testing Infrastructure Improvements

Testing Gradle plugins is a complicated subject. We already documented a possible solution at the beginning of the year but never got to perform actual experimentation. Since the last devlog, we finally did some exploration on that very solution. Unfortunately, we didn’t finalize a complete solution due to time constraints. However, we can successfully say the outlined solution is up-and-coming. The following steps are to split the work into mergeable PRs that would alleviate some of our pain, mainly tool selection and coverage assertion.

Through this work, we worked on our sample tests - our longest and most impactful tests - which we converted to JUnit 5 (from Spock). We cut down the execution time in half for those tests, thanks to JUnit parallel execution.

Devlog #6

Tue, 20 Sep 2022 00:00:00 +0000

This week’s development log (devlog) is a three-for-one special and includes work done since the last devlog, published a while ago. We split our primary development time between improvement to the language source set model, and the Xcode build adapter.

Language Source Set Improvements

For a while now, we had two competing LanguageSourceSet implementations. When we first introduced the concept, our chosen design focused on declaring the sources and headers. We borrow some ideas from the software model core plugins. The models were single bags of files that didn’t correctly represent reality and didn’t allow for ad-hoc declaration of source sets. The second concept initially introduced in the JNI library plugin focuses on software composition. In short, declaring a LanguageSourceSet implied the sources, their headers, a compile task, and consumable dependencies. We can use the source set alone to create object files or as part of a more complex hiearchy such as an executable binary. Thanks to our universal model, we can clearly define the owner and simple rules that will automatically complete native components with the right source set. The last part is much more complicated than it seems.

Xcode Build Adapter Improvements

As we onboarded more real-life projects with our Xcode build adapter plugin, it became clear that support for cross-project dependencies was necessary. Previously, we only supported implicit dependencies via Xcode workspace. We took a step back and made a deeper analysis of our Xcode model which led to the following graph ^[1]:

Figure 1. PBXProj models and relationships

One thing that stands out from our research is how simple yet powerful the Xcode build system is. We need a stellar understanding of Xcode to ensure proper up-to-date checking, caching and parallelization.

The first step was to deserialize cross-project reference from the PBXProj properly. It allows us to understand cross-project dependencies and remote product references (e.g. file references declared in another project). Then, we perform an additional discovery step during configuration to find the effective list of Xcode projects participating in the build. Finally, we use the Gradle dependency engine to process project dependencies (implicit, explicit and cross-project).

This small change to how we handle Xcode projects allows for deeper insight into the build process via build scan:

Figure 2. Build scan of vlc-ios project

As we can see, all targets are independent, even if they come from the same project. Previously all targets of a single project were treated as a single cluster. To successfully break down target clusters, we preprocess the Xcode project before delegating the build to xcodebuild. Thanks to the simplicity of the Xcode build system, we can trick Xcode into consuming our previously built products as part of a different and independent invocation.

Our next focus will be improving the configuration of Gradle’s superior up-to-date check by extracting more accurate information from Xcode projects.

Ad-hoc Artifact Repositories

Our plugins still rely on core native support in Gradle to discover and locate native toolchains. It isn’t ideal for us, given our lack of control over such critical infrastructure. For a long while, we had a plan to select native toolchains (and system libraries) using the Gradle dependency engine. Recently, we pushed the first part of this plan which introduces the concept of ad-hoc artifact repositories. They are on-disk Maven repositories with a twist! The repository provides callbacks to generate the repository’s content before completing the resolution. We did countless experiments, and this solution was by far the simplest. We still rely on internal Gradle APIs, so we don’t expect users to interact directly with those repositories. However, it marks the first step toward better native dependency management (including Conan support). We already migrated our macOS framework dependency support from the dreadful embedded Jetty server to an ad-hoc repository.

1. We are developing the graph organically based on the development need.

Devlog #5

Tue, 19 Jul 2022 00:00:00 +0000

This development log (devlog) includes work done since the last devlog, which mainly focuses on the Universal Model, more specifically on dependency buckets (aka Gradle Configuration). The progress was primarily technical; hence we think sharing the major roadblocks we overcame around using Configuration in a nested hierarchy would be appropriate for this devlog.

When we talk about nested hierarchy, we mean several layers of domain object ownership. For example, a component, variant or artifact can own a dependency bucket, which in turn, a component or variant can own artifacts, and finally, a component can own variants. This kind of hierarchy causes a lot of headaches in terms of discovery. In a fully lazy build, how does Gradle know it collected every outgoing Configuration for a proper dependency resolution? The short answer is it can’t! The focus today is on how to configure Configuration without realizing the world properly. Stay tuned for more discussion on the discovery in the future.

Misbehaving plugins

Our first surprise was a misbehaving plugin. In our example, the Koltin Gradle plugin realizes the dependencies of a Configuration too early preventing a pull behaviour from our model using addAllLater trick. Thankfully, we have an internal concept of finalized state, which we can use to propagate the dependencies from our buckets to their matching Configuration.

Lesson: Plugin authors should avoid realizing DomainObjectCollection too early.

Completing the `Configuration`

To ensure we can finalize a Configuration before Gradle performs its dependency resolution, we need some ways to receive a notification when Gradle is about to use the Configuration. The only hook available to us is ConfigurationInternal#beforeLocking. It would be wrong to think Configuration#beforeResolve is the public API equivalent; the Configuration is already immutable. Thanks to the internal hook, we can perform additional configuration avoidance.

Resolving gap of the `Configuration`

Now that we can adequately avoid early configuration and finalize the Configuration itself, are we done? Unfortunately, we face our final surprise, Configuration.s are not locked when Gradle visit their task dependencies. A Configuration behave much like a FileCollection. We can iterate the resolved files or visit the task dependencies that produce those files. In practice, visiting the task dependencies looks at ProjectDependency as it’s the only kind of Dependency that contains meaningful TaskDependency. Sadly for us, visiting the task dependencies does not lock the Configuration and only looks at the current DependencySet. Because of misbehaving plugins, we couldn’t lazily attach our dependencies via addAllLater. The result is a bit confusing. The tasks using the Configuration won’t have the correct task dependencies but will still end up with the expected resolved files, which points to missing files. In our Universal Model, we work around this issue by finalizing the dependency bucket upon collecting the incoming files. However, in our JNI library plugin, we had to force the java plugin’s Configuration to lock at key locations.

Note that, as of Gradle 7.4, Gradle will ignore task dependencies provided by ProjectDependency when added via Configuration#withDependencies and Configuration#defaultDependencies actions as those actions execute during the Configuration locking.

Devlog #4

Tue, 21 Jun 2022 00:00:00 +0000

This development log (devlog) includes work done since the last devlog, which mainly focuses on iOS support and Nokee integration with Gradle.

Xcode Build Adapter

We continue working on the Xcode build adapter plugin, which enables iOS support. We pushed the required changes to support implicit dependencies (e.g. Xcode’s inter-project dependencies). We can now build a project’s main target and dependencies without issues.

We still don’t support cross-project references, usually present when developers nest Xcode projects. Those type of reference requires improvement to our Xcode modelling. We also need to improve our strategy for delegating to Xcode as we would need to cluster these cross-project references together, defeating the goal of using Gradle/Nokee. The situation is very similar to Xcode target clustering. We have some experiments that should allow us to break down these clusters and improve our already excellent performance results.

Documentation Pipeline

We fixed the final two stages of our documentation pipeline (e.g. merged Javadoc and DSL reference). We should be able to update our documentation which includes multiple changes such as moving all samples to use the latest Gradle wrapper, using the correct Nokee version, and removing broken links.

Nokee Version Management plugin

We finished the implementation of the Nokee Version Management plugin. The last stretch of development required changes to our functional testing (to allow a wider spread of Gradle version to tests) and fixing our documentation pipeline. We expect to release this plugin soon.

Devlog #3

Mon, 6 Jun 2022 00:00:00 +0000

This development log (devlog) includes work done since the last devlog which mostly focus on iOS support.

Xcode Build Adapter

Building Xcode Scheme

Xcode schemes are the glue that ties actions (i.e. build, test, archive, etc.) and targets together. We map the scheme in two ways: globally, using the workspace, or locally, per project. The difference is subtle, but we will favour the local mapping over time as it should be more performant. The global mapping is a simple substitute for invoking the xcodebuild command line directly, as shown in the following cast:

We can build any shared scheme from Gradle’s root project.

Execution Avoidance for Xcode Targets

We are investigating how Gradle can provide execution avoidance directly out-of-the-box. Execution avoidance includes up-to-date detection (e.g. inputs/outputs snapshotting) and build cache (local or remote). Xcode is significantly lacking in the up-to-date detection capability. From our research, it seems Xcode heavily relies on modification time. In scenarios where only the modification time changes (not the content), i.e. on fresh GitHub clone or on pod install, xcodebuild will perform a full rebuild. The following demonstrates this behaviour:

However, with Gradle, we can detect the inputs and outputs of a target through analytics, allowing for some execution avoidance:

In this scenario, we used the HackerNews repository, a trivial application. The important takeaway here is the ability to split an Xcode workspace into multiple target clusters where we can cache each one individually. Some targets cluster can execute for as long as 7 minutes! It should be possible to avoid those lengthy executions using Gradle/Nokee.

Xcode error out on the side of rebuilding a target; hence proper inputs/outputs declaration isn’t as important. We will most likely need user intervention to identify the inputs/outputs of target clusters correctly.

Miscellaneous

The following are other creative explorations we did around the Xcode Build Adapter:

Nokee Version Management plugin

To provide a first-class integration between Nokee and Gradle, we are developing a settings plugin to manage Nokee versions and repositories. We initially intended for the plugin to have a narrow scope for the first version. Still, we decided to include more features for those odd projects (i.e. air-gapped, firewalled, source dependencies, etc.). Nokee configuration will simplify to a single plugin apply instead of the current complicated pluginManagement block:

settings.gradle[.kts]

plugins {
    id("dev.nokee.nokee-version-management") version("<version>")
}

Devlog #2

Thu, 19 May 2022 00:00:00 +0000

This development log (devlog) includes work done since the last devlog.

Nokee Universal Model (UM)

Since the last update, we finally removed an older implementation of the UM. We moved the language source set entities under the binary hierarchy. We introduce the UM implementation of the static library, executable and bundle binaries. Unfortunately, we didn’t migrate from the previous implementation due to missing modelling around exported headers and legacy source set implementation usage. We also optimized the tag component (no-data component) as we plan to move away from projection as a first-class concept in the UM.

Performance Regression

One of the blockers for the next Nokee version is a performance regression that we accepted when moving forward with the Universal Model (UM) migration. The UM allow a clean migration from Software Model to Nokee plugins in gradle/native-platform. We initially brought the regression down to about 1800ms which we then cut in half (to 900ms) with some internal optimization. To eliminate the regression, we will focus on discovery avoidance in the UM.

Documentation

We are slowly ramping up a documentation effort to prepare for the next Nokee version. You can follow the work using this GitHub query.

Devlog #1

Wed, 20 Apr 2022 00:00:00 +0000

This development log (devlog) is more extensive than expected in the following entries. It includes most work done during February and March in no particular order.

Performance Testing

Performance tests are essential for Nokee plugins to turn 1.0. We focused on performance comparison and repeatable profiling. We developed a new runner, similar to our Gradle runner, to perform build experiments. At the moment, we are tracking the performance of a JNI library plugin implemented using the Software Model and our Nokee plugin as part of the gradle/native-platform project.

iOS Support

First-class support for building iOS applications and libraries requires a lot of upfront work. Instead, we can shortcut the work required by adapting the Xcode build system to Gradle and progress backward. We named this solution build adapters as it adapts a complete build system to Gradle, similar to an included build. We already have an experimental CMake build adapter that we can use to build Google Test as a source dependency. The goal of build adapters is to reduce the migration cost while maximizing maintenance and performance.

We developed most of the infrastructure for handling Xcode workspace and projects. Gradle/Nokee can build targets within a single Xcode project. We successfully validated this first step using the VLC iOS application as shown below:

The published build scan is available here.

Nokee Universal Model

The Universal Model (UM) is a data-oriented design to configure Gradle. The typical configuration model (often referred to as vanilla Gradle) lacks support for deeply nested domain objects. The Universal Model focuses on breaking the boundaries between Gradle containers allowing a universal space for all domain objects. The model follows a plain Entity-Component-System architecture with the ability to project Entity into vanilla domain objects. You can follow our technical rambling about the Universal Model in our PRs.

All of our platform plugins use the Universal Model, i.e. c-library, jni-library, etc. The latest work focus on performance and clean architecture. We identified hot spots that we will investigate before releasing v0.5. We also untangled capabilities that should build on top of the Universal Model, i.e. classic Action execution, component elements tracking (similar to Gradle’s domain object container), etc. Finally, we prepare the architecture to support near-future capabilities such as reporting and custom repositories like Conan and vcpkg.

JBake plugin

All of our documentation is statically compiled using JBake. Our needs and complexity far outweigh the official plugin’s capability, mainly around multi-repository cooperation. We wrote our plugin to better support our documentation development. This plugin will allow splitting documentation to accelerate its growth by improving test performance.

FreeBSD/ARM support

Nokee officially supports building on FreeBSD and ARM systems. To successfully test on FreeBSD system, we developed a set of GitHub actions using Vagrant. We have ARM infrastructure coming soon, enabling ARM architecture testing.

Gradle build script model

As part of our test fixtures and documentation samples, we have to maintain many Gradle build scripts. We often compose those scripts from multiple snippets. Gradle’s multi-DSL language support (Groovy/Kotlin) makes this task challenging. For this reason, we wrote a model over Gradle build script. This model is crucial for our upcoming documentation samples and user onboarding stories, e.g. build init templates.

Nokee distribution management plugin experimentation

The effortless nature of the build adapter plugins is a great way to bridge other build systems with Gradle. However, managing which Nokee distribution to use and aligning the versions between projects is tricky. We want Nokee to feel like a core Gradle plugin despite being a 3rd party plugin. We tried solving the issue via an init plugin. This solution had many moving pieces, making it inadequate for users. Instead, we experimented with a settings plugin that provides a much easier user interface.

Gradle Enterprise server

The Gradle team provided us with a Gradle Enterprise instance for Nokee. Although we are Gradle experts, the onboarding helped us identify a few issues with our build. As part of this effort, we took the time to upgrade our Gradle version to 7.4. We also worked on our documentation pipeline, which had a lot of caching issues.

Introducing Development Logs

Wed, 20 Apr 2022 00:00:00 +0000

Today we are introducing development logs (also known as devlog) to help keep our users and sponsors informed and connected. Those short-form, semi-formal posts focus on where we spent our efforts since the last entry. Feel free to reach out to us if you have a particular request to be included in those updates.

The first devlog is already available.

License Clarification

Wed, 22 Sep 2021 00:00:00 +0000

Thus far, we developed Nokee in the open without ever having a precise Open Source Software (OSS) license. It can be a legal blocker for some teams when adopting Nokee for native development with Gradle. Today, we are officially clarifying Nokee source code under Apache 2.0 license. We believe this change will help Nokee grow further alongside Gradle.

Why Apache 2.0?

The Gradle team already licenses Gradle Build Tool under Apache 2.0. Our license choice needs to be compatible with the Gradle Build Tool itself. To maximize Nokee’s adoption, settling on Apache 2.0 was a natural choice. It will also reduce friction with your legal department.

What about previous versions?

We won’t be making any changes to the previous Nokee versions. However, if your team needs an Apache 2.0 friendly license for those versions, please send your request to support@nokee.dev. We will work together to find a solution for your needs.

If you have any questions, always feel free to reach out to me via GitHub discussion, sending an email to hello@nokee.dev, or joining the Gradle Community Slack.

Farewell Bintray…

Tue, 16 Feb 2021 00:00:00 +0000

On February 3rd, JFrog announced the sunset of Bintray and JCenter services. This news is particularly devastating as we depended on those services to release our hard work to you, our fantastic users. We are committed to offering our plugins and libraries without interruption, which is why solving this issue became our top priority.

We maintain two major artifact groups:

dev.nokee: a suite of Gradle plugins for painless native development
dev.gradleplugins: a suite of Gradle plugins and libraries for accelerated Gradle plugin development.

Moving forward, we will host the artifacts for both groups on our servers. Unfortunately, we won’t release the subsequent versions simultaneously into the central plugin portal until the Gradle team provide more guidance to the plugin development community. See the repositories landing page for more information on how to configure your Gradle build.

The following is a rundown of the replacement URLs for our Bintray repositories:

https://repo.nokee.dev/release contains the artifacts from the following Bintray repositories:
- hxxps://dl.bintray.com/nokeedev/distributions
- hxxps://dl.bintray.com/nokeedev/distributions-initialization
- hxxps://dl.bintray.com/gradle-plugins/distributions
https://repo.nokee.dev/snapshot contains the artifacts from the following Bintray repository:
- hxxps://dl.bintray.com/nokeedev/distributions-snapshots

In general, the migration process is as simple as updating the repositories to their new locations. However, some of our plugins have internal dependencies to Bintray, e.g. they configure repositories for the projects. We recommend updating the following plugins to the minimum specified version:

dev.gradleplugins.java-gradle-plugin → 1.2.1 (with repository in pluginManagement)
dev.gradleplugins.groovy-gradle-pluign → 1.2.1 (with repository in pluginManagement)
dev.gradleplugins.gradle-plugin-development → 1.2.1 (with repository in pluginManagement)
dev.gradleplugins.gradle-plugin-unit-test → 1.2.1 (with repository in pluginManagement)
dev.gradleplugins.gradle-plugin-functional-test → 1.2.1 (with repository in pluginManagement)
Nokee init plugin → use the latest version according to the usage documentation.

There are still more than two months before the service is wholly decommissioned; however, we urge our users to take the time now and proceed with a migration to the new repository locations. All new versions of our plugins and libraries will be released only in our repositories. If you have any questions regarding this migration, please sends us an email to bintray-migration@nokee.dev.

We are working on deeper integration with Gradle removing your need to manage those repositories manually. Stay tuned.

What’s the latest Nokee version?

Mon, 7 Sep 2020 00:00:00 +0000

As more users adopt Nokee for building their native codebases, we often get asked what’s the latest version they can try. As part of an effort to answer this question, we introduce a web service available at services.nokee.dev. The service aims at gathering useful information about Nokee. It also lay the groundwork for tighter integration with Gradle, which we will announce shortly.

At the moment, we expose version information about the latest snapshot, latest release and all available versions. We update the data daily. If you have neat ideas or want to improve this web service, feel free to open issues or contribute to the GitHub repository.

Stay tuned for more information about the upcoming Nokee release.

Release 0.4 Announcement

Mon, 29 Jun 2020 00:00:00 +0000

Join us to celebrate the fourth version of the Nokee plugins.

Longer than expected release cycle helped leap Nokee forward. In this release, we change the configuration model of Nokee plugins, which may potentially break current users. We made ten new plugins available for our users to support building native applications/libraries implemented in C, C++, Objective-C, Objective-C++ and Swift. We have also included five additional plugins to enhance the development of Gradle plugins. Let’s explore the release together.

Configuration model changes

The configuration model of Gradle was a talking subject for many years. It started with the zero configuration effort, aimed at minimizing the time spent processing the build script. Then, it was the software model, a different configuration model with ambitious goals. Now, it’s all about the configuration cache. When completed, it will allow near-instant execution of your Gradle builds. As the Gradle’s configuration cache near its completion, thanks to the fantastic work by the Gradle team, it’s essential to enable forward compatibility.

This release removes our dependency on obsolete Gradle core configuration modelling. It gives us more control for our future work required to push the iOS support forward. We also took a different direction than the native plugins developed by the Gradle team. The Gradle core plugins never instantiate unbuildable variants, making them impossible to access during the configuration phase. It’s a struggle point when adopting these new plugins for intermediate use cases. As part of this release, the Nokee plugins will instantiate all variants as required. It makes use cases like using previously built binaries when the local environment cannot produce the binaries possible.

New native plugins

This release contains ten new plugins for building native! They support building native applications and native libraries implemented in C, C++, Objective-C, Objective-C++, and Swift. These new plugins are similar in features and will improve them over the next releases. They will provide parity with all the native language supported by the core Gradle plugins. See their respective plugin reference chapters to learn more.

Gradle Plugin Toolbox version 1.0 announcement

We recently announced the Gradle Plugin Toolbox to help plugin authors. As part of building the Nokee project, we experience every issue of the Gradle plugin development process. For this reason, we decided to share our development plugins with everyone. It allows testing against multiple Gradle versions as well as targeting specific Gradle APIs. Head over to the user manual to learn more about the Gradle Plugin Toolbox features.

New Swift iOS application support

The iOS support got a new plugin to build applications written in Swift. Due to the lack of cross-compilation support in the core Gradle infrastructure, dependencies on libraries didn’t make it in this release. However, we are hard at work in improving our supporting infrastructure.

What’s Next?

With release 0.4, we leaped forward. Over the next releases, the focus will be on testing, publishing and toolchain support. Testing is an important story often overlooked by build systems. Our goal is to allow for the most straightforward testing setup as possible. Finally, publishing is hard to get right in native. We are discussing various scenarios to solve the problem for everyone using Nokee plugins. For developers relying on building from sources, we will soon start working on build wrapping stories.

Leave a comment below or reach out on the Gradle community Slack. If you want to work with us, email us at hello@nokee.dev.

Introducing Gradle Plugin Toolbox

Mon, 22 Jun 2020 00:00:00 +0000

The Gradle plugin development process is tedious. Writing a simple plugin is somewhat acceptable; however, very painful when developing complex plugins. The Nokee team is glade to officially unveil a toolbox to enhance the task at hand.

Releasing new Gradle plugins

Today, we are announcing five plugins to improve the Gradle plugin development process:

dev.gradleplugins.java-gradle-plugin: Provides support for developing Gradle plugin implemented in Java.
dev.gradleplugins.groovy-gradle-plugin: Provides support for developing Gradle plugin implemented in Groovy.
dev.gradleplugins.gradle-plugin-development: Provides additional support for developing Gradle plugin-related projects.
dev.gradleplugins.gradle-plugin-unit-test: Provides support for unit testing Gradle plugins.
dev.gradleplugins.gradle-plugin-functional-test: Provides support for functional testing Gradle plugins.

The first two plugins are a welcome alternative for the java-gradle-plugin core plugin and focus on Java and Groovy language, respectively. The Kotlin ecosystem is a bit particular due to some workarounds(hacks) in Gradle itself.

The third plugin provides conveniences to projects supporting the development of Gradle plugins. It’s a settings plugin and decorates all projects. The plugin provides conveniences for adding Gradle API dependencies of a specific version, querying compatibility information for released Gradle distribution, and adding dependencies on Gradle fixtures for testing maintained by Nokee. Have a look at the Gradle runtime compatibility sample to learn more.

The final two plugins formalize unit and functional testing for Gradle plugin development. Typically, unit tests focus on individual classes using the ProjectBuilder testing fixtures. On the other hand, functional tests focus on execution behaviours using the Gradle TestKit testing fixtures.

Why replace the core Gradle development plugins?

The core Gradle development plugins, e.g. java-gradle-plugin released as part of the Gradle distribution, are unfavorable to a fast iterative process. We have seen little improvement catering to the Gradle plugin development experience despite a need for key features to aid high-quality plugin development.

Features

Developing under a specific Gradle version while building with another

Modelling the minimum Gradle version supported by the plugins is an essential use case solved by the toolbox; instead of runtime errors Gradle will automatically detect the errors at compile-time by using the correct Gradle API JAR regardless of the running Gradle version.

Example 1. Configuring minimum supported Gradle

build.gradle

gradlePlugin {
   compatibility {
      minimumGradleVersion = '4.9'
   }
}

build.gradle.kts

gradlePlugin {
   compatibility {
      minimumGradleVersion.set("4.9")
   }
}

Nokee published the generated Gradle API’s JAR from each distribution together with its matching sources. Additionally, we included it’s correct Groovy/Kotlin runtime version as dependencies. We’ve written a utility class to map the different Gradle versions to it’s appropriate JVM runtimes; Java, Groovy, Kotlin.

Improving compatibility defaults

Our plugins select Gradle’s source and target compatibility based on the minimum supported Gradle version of your plugins. The plugins still allows you to overwrite Java compatibility as you typically would.

The same process applies to the Groovy runtime dependency as shown by the build scan.

By applying the dev.gradleplugins.gradle-plugin-development plugin in your settings script, you will have access to the GradleCompatibility class, used by our plugins' automatic compatibility selection, from every project.

Example 2. Querying Gradle compatibility information

settings.gradle

plugins {
    id 'dev.gradleplugins.gradle-plugin-development' version '1.0.0'
}

build.gradle

import static dev.gradleplugins.GradleCompatibility.*

tasks.register('gradleCompatibility') {
    def gradleVersion = '4.9'
    doLast {
      println "=== Gradle ${gradleVersion} Compatibility Information ==="
      println "Minimum Java version: ${minimumJavaVersionFor(gradleVersion)}"
      println "Groovy version: ${groovyVersionOf(gradleVersion)}"
      println "Kotlin version: ${kotlinVersionOf(gradleVersion).orElse('N/A')}"
    }
}

settings.gradle.kts

plugins {
    id("dev.gradleplugins.gradle-plugin-development") version("1.0.0")
}

build.gradle.kts

import dev.gradleplugins.GradleCompatibility.*

tasks.register("gradleCompatibility") {
    val gradleVersion = "4.9"
    doLast {
      println("=== Gradle ${gradleVersion} Compatibility Information ===")
      println("Minimum Java version: ${minimumJavaVersionFor(gradleVersion)}")
      println("Groovy version: ${groovyVersionOf(gradleVersion)}")
      println("Kotlin version: ${kotlinVersionOf(gradleVersion).orElse("N/A")}")
    }
}

Testing strategies

Plugin testing is the most important shortcoming in Gradle plugin development. Unfortunately, writing tests are the only way to ensure high-quality plugins. Our plugins support test coverage across several Gradle versions. As part of this release, we include three common testing strategies: minimum supported Gradle version, latest released Gradle, and latest nightly. Have a look at the testing strategies sample to learn more.

You can select the testing strategy from IntelliJ.

Extracted from the Gradle codebase, our plugins offer complementary test fixtures for common assertions, e.g. content of archives, files, Maven repositories. These test fixtures work in tandem with those provided by the Gradle team and already available: ProjectBuilder and Gradle TestKit. An early preview is now available for trial, and your feedback will help us prioritize future release features.

Generate Groovydoc JAR

Our Groovy development plugin enables the creation and publication of the Groovydoc JAR via the groovy extension on gradlePlugin. It automatically registers the Groovydoc JAR as an outgoing artifact for the project. Gradle will then publish the JAR side-by-side with your other plugin artifacts.

Example 3. Creating Groovydoc JAR for Gradle plugins

build.gradle

plugins {
    id 'dev.gradleplugins.groovy-gradle-plugin' version '1.0.0'
}

gradlePlugin {
    groovy {
        withGroovydocJar()
    }
}

build.gradle.kts

plugins {
    id("dev.gradleplugins.groovy-gradle-plugin") version("1.0.0")
}

gradlePlugin {
    groovy {
        withGroovydocJar()
    }
}

What’s next?

Try, share, exchange your ideas and suggestion. The Nokee team is eagerly awaiting your feedback!

We have a lot more features to come soon! Please vote for your favourite to help us prioritize which feature to deliver first in our next release.

Add compatibility information to the JAR plugin: Help diagnose incompatible plugins in future releases.
Helpful runtime message for incompatible environments: Improve user experience and promote plugin adoptions.
Improve the modularization of the Gradle fixtures: Use the Gradle fixtures with any testing framework.
Allow testing strategy for specific Gradle version: Offer more coverage options to plugin authors.
Add cross-JVM testing strategies: Improve the plugin testing coverage across JDKs.
Formalize well-behaving test suite for tasks and plugins: Ensure a baseline quality for all plugins.
Support low-ceremony plugin development: Quick and easy creation of plugins by removing the majority of a plugin project setup steps.

Leave a comment below, write us on the Gradle community Slack or email us at hello@nokee.dev.

Release 0.3 Announcement

Tue, 12 May 2020 00:00:00 +0000

Join me in celebrating the third release of the Nokee plugins. The most significant announcement of this release is the early iOS support for Gradle. Let’s explore this announcement as well as the other improvement accomplished as part of this release.

iOS Support for Gradle

Despite the requirements express in the previous announcement road map for iOS support, I decided to work around them and deliver the first milestone. I’m proud to show the fruit of that labour and deliver support for building iOS applications with XCTest for Gradle. It’s early support, which means there are limitations imposed by the plugin. For example, the application will only execute inside the simulator running iOS 13.2. The implementation language is also limited to Objective-C. I purposefully limited the scope for this first release so I could focus on the core experience from the developer’s perspective. There are two main scenarios I wanted to cover.

The first scenario is building and launching the application from Xcode. I carefully crafted the Xcode integration to delegate the build to Gradle while allowing Xcode to launch the application inside the simulator. The experience aims to be as vanilla as possible. It means you can build via Product > Build (Cmd+B) and launch the application via the main launch button, as shown in the video below.

The second scenario is testing from Xcode. This scenario is probably the most tricky to get right. The integration allows tests to be executed via the main test button in the top left corner, from the test navigator or the side dot in the editor. You can see it in action in the video below.

For both scenarios, the integration needed to allow indexing to work as expected. Developers can jump to symbol definitions, jump to files from an #import directive and perform searches inside the symbol navigator.

Feel free to try out one of the samples. If you find any issues, please open an issue. Don’t hesitate to contact me at hello@nokee.dev if you want to help sponsor the effort. We can work together and deliver the use case that would have the highest impact on your developer productivity first.

Java Native Interface (JNI) development improvement

The JNI library plugin kept improving during this release. I worked with the team behind Darklaf to showcase the JNI library plugin on their project. I identified gaps in the API resolving mostly around accessing variants, binaries and their tasks. I took this opportunity to add new APIs to allow a smoother transition.

A special thanks to Javen Wang for his help in enabling to use Kotlin as a JVM language with the JNI library plugin.

Finally, I also rolled out an entire chapter on explaining building JNI projects with Gradle and the Nokee plugins.

Documentation improvement

On the documentation front, I wrote additional chapters to explain the anatomy of Nokee plugins, building JNI projects and developing with Xcode IDE. I also wrote reference chapters for all the new plugins as well as new samples covering additional use cases raised by the community.

When I started my Gradle journey in 2012, I found the build language documentation (also known as DSL reference) precious to my learning. To honour this great resource, I wrote the automation required to generate the build language reference for all public types of the Nokee plugins.

As I take documentation very seriously, it is essential to note that I consider unclear documentation as a bug. If you encounter any issue with the documentation, please open an issue, so we can improve it for everyone.

Testing improvement

On the testing front, I added automated tests to keep the Nokee plugins in sync with Gradle release cycles. With this new addition, we are closer to the 1.0 milestone. I notice some gaps in the testing coverage during the last development cycle. They will be closed in the next release so we can move forward with the 1.0 release!

Open polls

Sometimes I don’t have enough context to make a decision that would have the most impact. For this reason, I like working with teams to help drive the requirements forward. The key point is about gathering information to make clear technical decisions. I started to create poll issues that ask open-ended questions for everyone to participate. The result of those polls will help drive the Nokee plugins forward with features that you care about. I scattered links to the polls across the documentation contextualize them.

Feel completely free to provide your opinion on each one of them. I will be using those issues as guidelines for what to work on next. For this reason, specific and tangible use cases are the key point I’m looking for.

What’s Next?

I’m scheduling improvements to toolchain support, in particular, the toolchain selection for the next release. This work is a fundamental feature that will help the cross-compiling stories with Gradle. It will be beneficial for iOS support as well as open opportunities for native Android support. It’s also essential to support dependencies and iOS libraries. CocoaPods is a must for the success of the iOS development support in Gradle. Luckily, the flexibility of Gradle will enable all these features over the next few releases of the Nokee plugins.

Don’t hesitate to leave a comment below or ping me on the Gradle community Slack. If you want to work with me and drive the iOS support for Gradle forward with your team, send me an email at hello@nokee.dev.

Introducing Leaf Bird

Thu, 7 May 2020 00:00:00 +0000

While I’m putting the final touch to the next release packed with impressive features, let’s take a quick break to welcome our happy leaf bird.

It’s no surprise that graphic design isn’t my strong point. The first logo, if we can even call it that, was simply the GitHub identicon automatically generated when I created the organization. It was a placeholder until I find a worthy replacement. All this time, it was painful to see how horrible my initial design work was.

I finally decided to reach out to Jason Thompson, an awesome designer and a good friend, to help me out on finding the perfect logo for Nokee. It didn’t take long before we narrowed down to this lovely leaf-like bird. It gives an excellent narrative to the project, but most importantly, a great emotion.

Stay tuned for the next release. If you can’t wait and want to spoil the surprise, head over to the nightly documentation to learn more about the next release.

Release 0.2 Announcement

Mon, 20 Apr 2020 00:00:00 +0000

Join me today in celebrating the second release of the Nokee plugins with some oatmeal cookies. ^[1] Some great features are being released like the support for macOS framework as normal dependencies. I will also go through the outcomes of various experimentation that happen since the first release.

macOS Framework Dependencies

It is common for macOS and iOS developers to rely on frameworks as dependencies. Historically, developers had to rely on the compiler and linker flags to support them. Why does it have to be this way? This release introduces the first step in enabling macOS framework bundles as standard dependencies. There are some limitations to the current implementation, and I’m going to work on removing them in future releases.

The following code snippet shows two interesting aspects of framework dependencies. Head over to the sample page for the complete code.

Example 1. Adding dependencies on macOS frameworks

build.gradle

library {
   dependencies {
      nativeImplementation 'dev.nokee.framework:Cocoa:10.15'       (1)

      nativeImplementation('dev.nokee.framework:JavaVM:10.15') {
         capabilities {
            requireCapability 'JavaVM:JavaNativeFoundation:10.15'  (2)
         }
      }
   }
}

build.gradle.kts

library {
   dependencies {
      nativeImplementation("dev.nokee.framework:Cocoa:10.15")      (1)

      nativeImplementation("dev.nokee.framework:JavaVM:10.15") {
         capabilities {
            requireCapability("JavaVM:JavaNativeFoundation:10.15") (2)
         }
      }
   }
}

1	Adding dependencies on a system framework
2	Adding dependencies on a subframework of a system framework

As shown in the previous snippet, a framework dependency is as simple as any other dependencies. In fact, the framework dependency resolution uses the Gradle dependency engine. It uses a magic group name (e.g. dev.nokee.framework) so the resolution engine can direct the request to the right resolver. Despite Apple’s recommendation to avoid referencing subframework directly ^[2], there is still a need to reference them for older deprecated frameworks like JavaVM and its subframework, JavaNativeFoundation. The dependency on subframework support is modeled using the Gradle capability support. The second example in the above snippet shows a dependency on the JavaNativeFoundation subframework. You can see the complete sample inside the documentation.

An interesting side-effect of using the Gradle dependency engine is the ability to customize your dependency just like any other dependencies.

You may be wondering how do the Nokee plugins hook into the dependency engine? One of the highest requested feature for native during my time at Gradle was support for Conan packages. Adam and I even experimented ^[3] with adding support for Homebrew using a custom resolver for consuming native artifacts. The issue with this approach is how tangled it is with the internal of Gradle. The spike we developed doesn’t work with the latest Gradle. It just shows how fragile such an approach is, and it’s not something I can maintain. There are also other artifact types the Nokee plugins should support out-of-the-box (e.g. Nuget, system libraries, Windows SDK/DDK, and toolchains). Gradle’s dependency engine is powerful. Relying on the Gradle’s dependency engine for the artifact resolution has positive side-effects. For starter, we can benefit from the rich version matching or gracefully resolve dependency conflict. What about variant selection? Native artifacts have the same variant selection problems.

The big problem is how we can feed custom dependency data inside the dependency engine? Nokee’s solution is an embedded HTTP server that serves as a customizable resolver for finding and providing all the required artifacts to your build. It uses all public APIs, which makes the solution forward compatible. The challenge is to ensure harmony between Gradle and Nokee without impacting the performance of the build. Thankfully, the Gradle team has been releasing helpful features such as the build service that enable such a solution. The consumers request artifacts as a standard group, artifact, version (GAV) triplet, optionally using the rich Gradle metadata for more advanced selection. Nokee’s embedded server aggregate all the artifacts found by his resolvers and let Gradle perform the selection and conflict resolution. It effectively creates a translation layer between Gradle and the chaotic world of native.

New Objective-C and Objective-C++ Language Plugins

A language plugin’s sole job is to provide a specific implementation language capability to the applied project. With this release, I’m introducing two new language plugins to support JNI libraries written in Objective-C and Objective-C++. They work just like any of the other Nokee language plugins.

JNI Library Resource Path Customization

The previous version had incomplete support for multi-variant JNI libraries. When assembling the JAR of the final JNI library, there was no way to customize the resource path where to put the native libraries would reside. Some JNI projects already have an opinion on where the native libraries are located inside the JAR so they can be unpacked and loaded at runtime. This release allows users to customize the resource path per-variant. A default mapping is provided based on the project group and ambiguous dimension values (e.g. when several supported operating systems).

Testing Coverage

Before I learned about the upcoming Gradle support for distributed testing, I did some experimentation on spawning Gradle workers on remote machines. In a previous job, I wrote a remote native testing capability for Gradle. I can’t begin to tell you how useful it was, not for performance, but for the convenience of being able to test the code on specific systems by any developers. I recently had a great discussion with a community member regarding his bare metal x86 development with Gradle. At such a low level, the issue is rarely the build time. Usually, it’s all the intermediate steps that are required to accurately tests the software. Often those steps are so troublesome that testing isn’t automated if you are even lucky to have testing in place.

In my case, I wanted to simplify the testing infrastructure by avoiding the classic dance of fanning out the pipeline at the testing phase. My experiment had some success on Linux but hit some roadblock on Windows. I decided to shelve the idea for the time being. However, rest assured that remote orchestration of Gradle workers would be hugely beneficial for the native ecosystem.

What’s Next?

The next versions will focus on adding support for Swift implementation language, the last native language needed before attacking full on iOS support in Gradle. There will be improvements to toolchain support, in particular the toolchain selection. At the moment, toolchain selection is preventing FreeBSD official support. It’s also preventing cross-compilation on all major operating system which in turn prevent iOS support in Gradle.

There will also be improvements to the testing coverage, particularly across multiple toolchains and multiple Gradle. Both of these are being worked on as part of the Gradle Development Plugins, a project aimed at improving plugin authoring. The goal is to provide near zero-configuration for Gradle plugin development. It also extends the core plugin development by modelling minimum Gradle version support and fixtures to improve your testing coverage. The next versions will focus on adding testing strategies for the plugin functional tests across multiple Gradle version.

Don’t hesitate to leave a comment below or ping me on the Gradle community Slack.

1. I added semi-sweet chocolat chips to the cookie recipe.

2. Subframeworks are frameworks embedded in other frameworks. In a nutshell, they should be seen as implementation details of the target framework and, like everything, there are exceptions

3. Mostly just Adam; I simply made the code merge ready

Maven POM DSL

Wed, 1 Apr 2020 00:00:00 +0000

After 8 delusional years of thinking that Gradle can become the best build system on the planet I realized that Maven is the way to go. It has a simple DSL that everyone can gasp within minutes, unlike Gradle, with its fancy Groovy and Kotlin DSL. A full Turing language for your build script is overrated. Ant and Maven worked perfectly fine, why reinvent the wheel? I’m proud to announce that our next big project at Nokee will be a Maven Project Object Model (POM) DSL ^[1] support for Gradle. It will ease your transition off Gradle toward Maven. If you are a native developer, have a look at NAR plugin for Maven. It even supports cross-compilation, imagine that!

How does it work?

Pretty much how you would expect it to work. The plugin reads a standard pom.xml file and configures your Gradle build accordingly. It’s that simple. Forget about the Groovy DSL or even the new, and fancy, Kotlin DSL. Those are things of the past, long live the POM DSL.

Don’t go on thinking that Gradle executes a full Maven instance inside the daemon… that would be pure madness.

I get it, you are a smart developer and want to know how Gradle will behave with this new (old?) DSL. The following shows the pom.xml concept mapping to Gradle:

packaging: The values jar, ear, and war map to the Gradle core plugins java, ear, and war respectively. The plugin ignores all other packaging values.
modules: The pom.xml files discovery starts from the root POM file, and they all participate in the Gradle multi-project configuration.
properties: The plugin maps the key/value pairs into the Gradle project’s extra properties.
dependencies: It maps each dependency scope to their equivalent Gradle configuration, i.e. runtime maps to runtimeOnly, provided maps to compileOnly, etc. The plugin will also wire local project dependency as you would expect. Finally, each dependency will be subject to the constraint imposed by the dependencyManagement tag.
parent: During the configuration phase, it creates an effective POM by merging all the parent POM information for each project.

The other informational tags, i.e. name, inceptionYear, contributors, etc., are simply copied over to the published POM.

How to get started?

You can get started with those three easy steps:

Save the init script to your ~/.gradle/init.d. ^[2]
Huh, that is it… Only one step. Head over to the next section to see it in action.
I don’t know, profit? Why didn’t you stop at the first step like everyone else? I should have just said a single step… Sight, I not good at this.

From the command line, inside a POM DSL enabled project, simply execute Gradle as you would normally do:

$ gradle assemble

BUILD SUCCESSFUL
2 actionable tasks: 2 executed

What is supported?

I’m glad you asked, and the short answer is everything! ^[3] What? Don’t you believe me? Fair enough, let’s go through some examples together. Feel free to follow along. We will be using these samples. First things first, let’s clone the repository:

$ git clone https://github.com/nokeedev/gradle-pom-dsl
...
$ cd gradle-pom-dsl/src/docs/samples
$ ls -1
1.basic-maven-example
2.intermediate-maven-example
3.multi-module-maven-example
README.adoc

Basic configuration? Check.

All the basic models, artifactId, groupId, description, modules, etc. are used for configuring the equivalent Gradle model. We can see it in action using the projects tasks, as shown below:

$ cd 3.multi-module-maven-example
$ gradle projects

> Task :projects

------------------------------------------------------------
Root project - This pom acts as the parent pom for the entire project.
------------------------------------------------------------

Root project 'parent' - This pom acts as the parent pom for the entire project.
+--- Project ':business' - Business Layer Project.
|    +--- Project ':business:api' - Business Layer API.
|    \--- Project ':business:impl' - Business Implementation.
+--- Project ':data' - Sample Implementation of the Business Layer.
|    +--- Project ':data:api' - Data Access API.
|    \--- Project ':data:impl' - Data Access Implementation.
\--- Project ':model' - Model Layer Project.

To see a list of the tasks of a project, run gradle <project-path>:tasks
For example, try running gradle :business:tasks

BUILD SUCCESSFUL
1 actionable task: 1 executed

Did you see that? The structure of the project, as well as the description, are pulled directly from the pom.xml files. Unbelievable! What’s next…

Dependency with exclusions? Check.

Do you need to exclude transitive dependencies? Picture the following pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <!-- ... -->

   <dependencies>
      <!-- ... -->

      <dependency>
         <groupId>org.hibernate</groupId>
         <artifactId>hibernate</artifactId>
         <version>3.2.5.ga</version>
         <exclusions>
            <exclusion>
               <groupId>javax.transaction</groupId>
               <artifactId>jta</artifactId>
            </exclusion>
         </exclusions>
      </dependency>
   </dependencies>

    <!-- ... -->
</project>

Pretty typical, right? Now, let’s see it in action:

$ cd ../2.intermediate-maven-example
$ gradle build --scan

BUILD SUCCESSFUL
4 actionable tasks: 4 executed

Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? [yes, no] yes

Gradle Terms of Service accepted.

Publishing build scan...
https://gradle.com/s/hj73vryn73y74

Did I mention that build scans also work? Speaking of which, let’s look at the dependency insight report. We can see the artifact javax.transaction:jta wasn’t pulled from Maven Central.

Multi module project with parent POM? Check.

So far, the features aren’t that impressive. How about a multi-module project with parent POM configuration? The following example is exactly that:

$ cd ../3.multi-module-maven-example
$ gradle check --scan

BUILD SUCCESSFUL
11 actionable tasks: 11 executed

Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? [yes, no] yes

Gradle Terms of Service accepted.

Publishing build scan...
https://gradle.com/s/tbul7lnihirto

Gradle reports all those tests in the build scan. If you prefer, you can also look at the local reports generated by Gradle.

Are you impressed yet? No? What about…

Publishing? Check.

Publishing! Everyone’s favourite topic. The POM DSL plugin automatically applies and configure the maven-publish plugin free of charge:

It publishes everything, see for yourself:

$ tree ~/.m2/repository
~/.m2/repository
└── com
    └── example
        └── maven
            └── layering
                ├── example-multi-module-business
                │   ├── 0.0.1-SNAPSHOT
                │   │   ├── example-multi-module-business-0.0.1-SNAPSHOT.pom
                │   │   └── maven-metadata-local.xml
                │   └── maven-metadata-local.xml
                ├── example-multi-module-business-api
                │   ├── 0.0.1-SNAPSHOT
                │   │   ├── example-multi-module-business-api-0.0.1-SNAPSHOT.jar
                │   │   ├── example-multi-module-business-api-0.0.1-SNAPSHOT.module
                │   │   ├── example-multi-module-business-api-0.0.1-SNAPSHOT.pom
                │   │   └── maven-metadata-local.xml
                │   └── maven-metadata-local.xml
                ├── example-multi-module-business-impl
                │   ├── 0.0.1-SNAPSHOT
                │   │   ├── example-multi-module-business-impl-0.0.1-SNAPSHOT.jar
                │   │   ├── example-multi-module-business-impl-0.0.1-SNAPSHOT.module
                │   │   ├── example-multi-module-business-impl-0.0.1-SNAPSHOT.pom
                │   │   └── maven-metadata-local.xml
                │   └── maven-metadata-local.xml
                ├── example-multi-module-data
                │   ├── 0.0.1-SNAPSHOT
                │   │   ├── example-multi-module-data-0.0.1-SNAPSHOT.pom
                │   │   └── maven-metadata-local.xml
                │   └── maven-metadata-local.xml
                ├── example-multi-module-data-api
                │   ├── 0.0.1-SNAPSHOT
                │   │   ├── example-multi-module-data-api-0.0.1-SNAPSHOT.jar
                │   │   ├── example-multi-module-data-api-0.0.1-SNAPSHOT.module
                │   │   ├── example-multi-module-data-api-0.0.1-SNAPSHOT.pom
                │   │   └── maven-metadata-local.xml
                │   └── maven-metadata-local.xml
                ├── example-multi-module-data-impl
                │   ├── 0.0.1-SNAPSHOT
                │   │   ├── example-multi-module-data-impl-0.0.1-SNAPSHOT.jar
                │   │   ├── example-multi-module-data-impl-0.0.1-SNAPSHOT.module
                │   │   ├── example-multi-module-data-impl-0.0.1-SNAPSHOT.pom
                │   │   └── maven-metadata-local.xml
                │   └── maven-metadata-local.xml
                ├── example-multi-module-model
                │   ├── 0.0.1-SNAPSHOT
                │   │   ├── example-multi-module-model-0.0.1-SNAPSHOT.jar
                │   │   ├── example-multi-module-model-0.0.1-SNAPSHOT.module
                │   │   ├── example-multi-module-model-0.0.1-SNAPSHOT.pom
                │   │   └── maven-metadata-local.xml
                │   └── maven-metadata-local.xml
                └── parent
                    ├── 0.0.1-SNAPSHOT
                    │   ├── maven-metadata-local.xml
                    │   └── parent-0.0.1-SNAPSHOT.pom
                    └── maven-metadata-local.xml

20 directories, 34 files

But, why?

According to our in-depth study, the POM DSL is so simple to understand that it will give back over 9000 hours of R&D to your organization each week. ^[4] You heard me right. Your developers can focus on what matters, like sending cat videos or brag about their Nyan score to each other.

Wait! Wat?

That’s not even a question. Have you been following? Maybe this cat picture using the POM DSL plugin will put you at ease.

Convince me harder

I can surely try. I had my team prepare a visual for your next meeting to help convince everyone:

Based on a study we did over the weekend, 1 out of 10 participants recommends the POM DSL over the current Groovy and Kotlin DSL. Equally, we made the breakthrough discovery that you can’t trust 9 people out of 10 that participated in the study, go figure.

Conclusion

All kidding aside, Gradle is exceptionally flexible. It always gets me when I hear "Gradle can’t do X, Y or Z". The truth of the matter being Gradle is the most flexible build system and possibly the only one you will ever need.

In the native ecosystem, the majority of developers strongly prefer Python for their scripting language. Similarly, in the Web ecosystem, the developers strongly prefer Javascript (or it’s variant). Some developers go through great lengths for these preferences and create build systems with DSL that feels like there preferred language. However, I beg the question, which is more complicated:

Creating a build system for a DSL; or
Creating DSL for a build system?

I realized I could have faked the entire post; after all, it’s April Fool day if you didn’t know. However, I built the entire POM DSL plugin and released the source code on GitHub. All samples shown in this post are real examples and work as demonstrated. Feel free to toy around with the project and keep me posted on what you manage to accomplish. This plugin is a demonstration, and you shouldn’t use the plugin in production. ^[5]

Stay safe and don’t hesitate to leave a comment below or ping me on the Gradle community Slack.

1. DSL stands for Domain Specific Language and, in the current context, refers to the language used for configuring your build.

2. It is preferred to specify the init script using the --init-script command line flag.

3. Some limitations apply. Batteries not included. Not recommended to use when it surpasses ten on the Richter scale.

4. Disclaimer: Don’t believe everything you read online.

5. I don’t guarantee its efficiency and don’t believe it’s bug-free.

Release 0.1 Announcement

Mon, 16 Mar 2020 00:00:00 +0000

I’m proud to announce the first version of the Nokee plugins.

Building Java Native Interface (JNI) Library

If you are like me, building native software has always painful regardless of the build system being used. Eight years ago, I showed that it doesn’t have to be that way, and Gradle has the potential of being a game-changer in that ecosystem. The JNI libraries are often the first encounter with native development for Java developers. This encounter isn’t always pleasant. As a first step in the ongoing native support in Gradle, I’m proud to announce the release of the JNI Library Plugin. It allows the users to build the JVM and native code side-by-side within the same project. It also support migration use cases where each components of the JNI library are built individually.

Everything you would expect from a JVM library is supported: publishing, testing, and developing within an IDE using the IntelliJ native importer or Eclipse Buildship.

Documentation Portal

The documentation is probably more important than the plugins themselves. For this reason, I spent quite a bit of time ensuring it’s of high quality. I was attached to rolling out quality samples first. They are goal-oriented, easy to write, and provide immediate value to you, the users. As Gradle is a very sophisticated build system, I’m often resorting to Sample Driven Development (SDD) before modelling or writing feature code. For this reason, everyone should feel free to open issues for use cases that are important to you but aren’t obvious to solve.

As for the user manual, the focus was on getting the core concept and direction of the plugins across. I suggest reading the terminology chapter to get a good understanding of the core concept. The JNI Library Plugin reference chapter is also a good place to start to learn about highlighted plugin of this release.

More will be coming in the future release.

Supporting Infrastructure

The infrastructure is the invisible syndicate that ensure the users are always presented with quality work. One of the perks of working within the Gradle codebase was all the infrastructure already available. Sadly, lots of it is couple to Gradle, which prevents you from sharing the pain plugin authors are going through when developing high-quality plugins. Some of my supporting infrastructures are shared over on Gradle Plugin Development repositories. The goal is to fill the repository with all the tools plugin authors may require to be successful at his job. It includes fixtures for executing Gradle (i.e. using the TestKit Runner or the Gradle Wrapper), making sense of the Gradle output (i.e. comparing two builds, asserting execution events) and composing source samples for testing. You can look at Nokee’s repository for all the creative ways I’m using those fixtures for assembling the sample templates and automating the sample tests to name a few.

The infrastructure also support nightly release of the work in progress. The testing coverage is also an essential part of the infrastructure puzzle. I’m committed to providing high-quality plugins to Gradle users. As the great Titus Winter once said: if you like a feature, you should put a test on it. It is also known as The Beyoncé Rule.

Roadmap to 1.0

It’s always hard to identify a clear roadmap as the software domain is forever changing. Despite this, I feel confident about the steps that will lead to version 1.0. I’m committed to providing high-quality plugins, which assume requires good test coverage. The features of this release are already providing a lot of values to the users. However, the test coverage for individual platforms is lacking. It’s a typical chicken and egg problem. Until version 1.0, I will split the R&D time between improving test coverage and adding valuable features for the users. Because of this decision, the 0.x versions may be a bit bumpy as the test coverage improve to lock the features in place and avoid bugs on a different platform. If you discover any issues with the plugins, please open an issue. I will make sure to prioritize a fix for those issues quickly for easy adoption.

So what does this all mean? It means there will be a heavy focus on adding test agents that support various toolchains, operating systems, and architecture. As the coverage improves, I will keep everyone informed via the blog and Twitter. Feel free to follow the most convenient, so you don’t miss any announcement.

iOS Development Poll

Lastly, I want to take a moment to thanks everyone who took the time to fill the iOS development poll. If you haven’t filled it, you can still do so. The next phase of the native development by the Nokee team will focus on cross-compilation for iOS inside Gradle.

iOS Development Poll

Thu, 13 Feb 2020 00:00:00 +0000

I want to thanks everyone for the positive response following the announcement last week. As discussed briefly in conversations with some of you, the focus is on bridging the gap around cross-compilation, more specifically on iOS support. This first step is critical in improving the native support in Gradle. It’s also a significant milestone for mobile development.

To focus on what is truly important, I created the this short, anonymous poll for users and build authors regardless of their current environment. Gathering this information is essential to you and me, so the Gradle plugins released can bring happiness into your build. It’s important to remember that Gradle is highly configurable. A common practice in native is to resolve pain points slowing down your development by using Gradle as an orchestrator. It can be tedious to migrate from another build system to Gradle. However, it could be trivial to wrap another build system inside Gradle to benefit from all of its star features, such as a build cache, a powerful dependency engine or an active community.

For the ones that would love to have a more active voice during the development process, you can leave your contact information at the end of the poll, and I will contact you for a more in-depth interview.

If you have any questions, always feel free to reach out to us via GitHub discussion, sending an email to hello@nokee.dev, or joining the Gradle Community Slack.

Hello Native

Tue, 4 Feb 2020 00:00:00 +0000

My name is Daniel Lacasse, and I’m the world’s most experience build engineer in native (C/C++/Assembly) with Gradle. I started my interest in Gradle for native long before I joined Gradle inc. It was in May of 2012 when I was researching how to replace a massive native frankenbuild system in a previous job. In January 2013, I prepared a proof-of-concept (POC) as part of an internal workshop supporting the idea of using Gradle to build our native monolith. The catch? Gradle didn’t have any of the native features we needed. Despite this shortcoming, it was accepted unanimously as the step forward to alleviate our pain. Yeah, my POC was that impressive! It was the start of my long journey in becoming a world-class Gradle engineer advocating for native support.

Quick history lesson

Following the success of my POC, I wrote and directed the statement of work that became the native support introduced in the Gradle 1.x releases (see the release notes for 1.7, 1.8, 1.9, 1.10, 1.11, and 1.12). The final result was presented at the Gradle Summit 2014. It was also the beginning of what is known today as the software model. Spoiler alert, the software model is phasing out. I extended the core native plugins in incredible ways adding support for remote testing, artifact publication, artifact promotion, unattended provisioning of the Windows SDK, Protocol Buffers, Doxygen, and many more supporting technologies for native. The massive success I got out of Gradle was just the tip of the potential iceberg Gradle had to offer. In 2016, I joined Gradle inc. to help improve the native support. Shortly after, I was the bearer of bad news for the Gradle community announcing the phase-out of the software model. I felt personally responsible for the disruption this news brought to the build engineers that, just like me, were praising and centralizing their development around it. I transitioned to the C++ and Swift revival of the core native support development. The approach and feature set were extremely promising (see here, here and here). Unfortunately, the effort was short-lived.

What is Nokee?

Nokee is a blank slate, a fresh start, lessons learned from mistakes and, most importantly, a path forward to bring a grand vision to reality. It is, first and foremost, a continuous effort for native support in Gradle where I will dedicate the majority of my time nurturing the native ecosystem of Gradle. Second, it will be a set of high-quality Gradle plugins to enhance and work in tandem with the core native plugins. Fear not, the plugins will be compatible with the Gradle core plugins, and a clear migration path will be communicated where needed. The focus will be around improving the shortcoming of the current core plugins.

How can you help?

You may be as excited as me and wonder how you can help. You can provide feedback by commenting on posts like this one, send an email to hello@nokee.dev, tag your native questions on the Gradle user forum and join the discussion in the #native channel over on the Gradle Community Slack.

How can you keep up-to-date?

Maybe the excitement hasn’t grown on you yet, and you want to keep informed. I suggest you watch the project’s GitHub organization, subscribe to the blog’s atom feed, and join the newsletter for future announcements.

Let’s make 2020 a Gradle native year!

Nokee Labs Blog

Devlog #9

Devlog #8

Devlog #7

Devlog #6

Devlog #5

Devlog #4

Devlog #3

Devlog #2

Devlog #1

Introducing Development Logs

License Clarification

Why Apache 2.0?

What about previous versions?

Farewell Bintray…​

What’s the latest Nokee version?

Release 0.4 Announcement

Introducing Gradle Plugin Toolbox

Release 0.3 Announcement

Introducing Leaf Bird

Release 0.2 Announcement

Maven POM DSL

Release 0.1 Announcement

iOS Development Poll

Hello Native

Farewell Bintray…