Skip to content

Instantly share code, notes, and snippets.

@pvillega

pvillega/readme.md

Forked from laughedelic/sbt-dependency-management-guide.md
Created April 16, 2019 19:20
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save pvillega/6823e3732d11e8c78573cc62db31bc3f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save pvillega/6823e3732d11e8c78573cc62db31bc3f to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @laughedelic laughedelic created this gist Jan 4, 2019.
    51 changes: 51 additions & 0 deletions readme.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,51 @@
    Some of these practices might be based on wrong assumptions and I'm not aware of it, so I would appreciate any feedback.

    1. avoiding _some_ dependency conflicts:
    + add [sbt-explicit-dependencies](https://github.com/cb372/sbt-explicit-dependencies) to the `project/plugins.sbt`
    + run `undeclaredCompileDependencies` and add any missing explicit dependencies to `libraryDependencies` of each sub-project
    + (optionally) run `unusedCompileDependencies` and remove some _obvious_ unused libraries. This has false positives, so `; reload; Test/compile` after each change and ultimately run all tests to see that it didn't break anything
    + (optionally) add `undeclaredCompileDependenciesTest` to the CI pipeline, so that it will fail if you have some undeclared dependencies

    2. keeping dependencies up to date and resolving conflicts:
    + install [sbt-updates](https://github.com/rtimush/sbt-updates) globally in your `~/.sbt/{0.13,1.0}/plugins/plugins.sbt`
    + run `dependencyUpdates` and bump all non-major versions. Major versions updates should be done one by one with care ~~and love~~ and testing.
    + include all explicit `libraryDependencies` in `dependencyOverrides` to force their versions. This is supposed to have the same effect as applying [`_ force()`](https://www.scala-sbt.org/release/docs/Library-Management.html#Forcing+a+revision+%28Not+recommended%29) on all `libraryDependencies`, but isn't ivy-specific. The point is to prevent conflict manager choosing automatically some version required by a transitive dependency instead of the one you wrote explicitly.
    This is done in `project/Dependencies.scala`

    3. using [sbt-assembly](https://github.com/sbt/sbt-assembly):
    + try running `assembly` for each project (starting from the independent ones) and see if there are any merge conflicts
    + if there are two different libraries that contain conflicting class files (same path, different content), use [shading](https://github.com/sbt/sbt-assembly#shading) to rename one of them:
    - don't use `.inAll` because it will rename classes in both of the libraries and it will be the same situation. Instead use `inLibrary` or `inProject`
    + avoid using `exclude` or `excludeDependencies` because you may throw away some library which is needed by one of the transitive dependencies and it will fail in runtime with `MethodNotFoundException` or something like that
    + avoid overriding [merge strategy](https://github.com/sbt/sbt-assembly#merge-strategy) on class files (using `first`/`last`/`discard` strategies), because it's the same as excluding some classes. Use merge strategy overrides only for some trivial conflicts or non-class files, e.g. to merge two `.properties` files with `concat` or `filterDistinctLines` strategy

    4. resolving more conflicts:
    + run `evicted` for each project and inspect the list of automatically resolved conflicts
    + try to minimize the number of lines marked as `[warn]`, those are conflicts with _potentially_ binary incompatible versions
    + if some of the libraries introducing the conflict are yours (company-owned), go and update its dependencies to solve the conflict. Unfortunately, it's more often another way around: your libraries are more up to date than some external ones that you don't have access to. Anyway, consider updating those external libraries.
    + use [sbt-dependency-graph](https://github.com/jrudolph/sbt-dependency-graph) installed globally to untangle the dependencies and understand the origin of the conflicts. The `whatDependsOn` task is very useful for that.

    5. Rinse and repeat. I numbered these steps because IMO it's better to do them in this order, but after each step it might be useful to go through the previous steps again.


    Some useful links:

    * ["What are your tips/tricks/tools for dealing with dependency hell?"](https://twitter.com/codenoodle/status/1067107851410259968)
    - https://github.com/cb372/sbt-explicit-dependencies
    - good point (human factor):
    > Do a better job of **articulating WHY certain dependencies are used** rather than dumping everything in `common` and then wondering why SBT is so complex.
    * [advices](https://twitter.com/eed3si9n/status/1067139186623475713) from [Eugene Yokota](https://github.com/eed3si9n):
    > - [x] avoid -SNAPSHOT outside of local testing
    > - [x] avoid version range
    > - [x] `evicted` for eviction report
    > - [ ] sbt -no-colors update > update.txt
    > - [x] jrudolph/sbt-dependency-graph
    > - [x] rtimush/sbt-updates
    * sbt docs:
    + [library management](https://www.scala-sbt.org/release/docs/Library-Management.html)
    + [dependency management flow](https://www.scala-sbt.org/release/docs/Dependency-Management-Flow.html)
    + [strict conflict manager](https://www.scala-sbt.org/1.x/docs/Library-Management.html#Conflict+Management)
    - I decided not to use it, because it works only in the cases when you have a limited number of dependencies, most of them are explicit and _controlled by you_ (i.e. minimum external dependencies, or just NIH syndrome). I can expand my arguments here if needed.