starts the effort to simplify artifact release #995

arron-green · 2018-10-06T23:11:21Z

In the spirit of hacktoberfest, this PR is a start to simplify the effort required to release finch artifacts.

@vkostyukov I'm going to need your help with finishing this PR as i'm sure you have the keys to publish these artifacts.

as suggested i used https://www.theguardian.com/info/developer-blog/2014/sep/16/shipping-from-github-to-maven-central-and-s3-using-travis-ci as a reference

Requires the following files to be added to the ./build directory

File	Description
deploy_key.pem.enc	Encrypted github deploy key to trigger commits, when non PR branches are merged into the `master` branch when `version.sbt` does not contain `SNAPSHOT`
secring.gpg.enc	Encrypted `gpg` secret ring
pubring.gpg.enc	Encrypted `gpg` public ring
credentials.sbt.enc	Encrypted sbt file containing credentials to sonatype

// example credentials.sbt
import com.typesafe.sbt.SbtPgp._

credentials += Credentials(
  "Sonatype Nexus Repository Manager",
  "oss.sonatype.org",
  "<SONATYPE_USERNAME>",
  "<SONATYPE_PASSWORD>"
)

pgpPassphrase := Some("<PGP_PASSPHRASE>".toCharArray)

I also wasn't sure on what to add for the github commits for releasing so i figured we could take advantage of environment variables set through travis. I defined CI_DEPLOY_USERNAME and CI_DEPLOY_EMAIL

Also worth noting that the matrix style building of 2.11 and 2.12 might not make sense with this PR. I don't think the sbt release command used in the build.sh makes sense if it kicks off two separate travis jobs. Therefore i left the execution of build.sh commented out for now

We might need to iterate on this a few times to find the right balance of auto release vs semi-automatic release

Fixes #977

codecov-io · 2018-10-06T23:20:36Z

Codecov Report

Merging #995 into auto-releases will increase coverage by 0.58%.
The diff coverage is n/a.

@@                Coverage Diff                @@
##           auto-releases     #995      +/-   ##
=================================================
+ Coverage          83.41%   83.99%   +0.58%     
=================================================
  Files                 50       50              
  Lines                856      856              
  Branches              51       54       +3     
=================================================
+ Hits                 714      719       +5     
+ Misses               142      137       -5

Impacted Files	Coverage Δ
core/src/main/scala/io/finch/Input.scala	`100% <0%> (+9.61%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3079e72...87c2df4. Read the comment docs.

vkostyukov · 2018-10-08T15:27:41Z

Thank, @arron-green! I will take a look into this today.

arron-green · 2018-10-08T19:29:24Z

@vkostyukov no problem. glad to help on next steps to find the right balance in what you're looking for :)

vkostyukov · 2018-10-08T22:50:23Z

build.sbt

+    )
+    val pattern = """"com.github.finagle" %% "finch-.*" % "(.*)"""".r
+
+    filesToUpdate.foreach{ fileName =>


This needs a space after foreach

vkostyukov · 2018-10-08T22:52:54Z

build/build.sh

+        chmod 600 local.deploy_key.pem
+        ssh-add local.deploy_key.pem
+        git config --global user.name "${CI_DEPLOY_USERNAME}"
+        git config --global user.email "${CI_DEPLOY_EMAIL}"


What are those CI_DEPLOY values by default? Should we set them into something?

Perhaps I should somehow use the same email/user name while generating a github deploy key: https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/#generating-a-new-ssh-key

I think I understand now. Maybe we should just hard-code these into something? Let me think about this for a bit.

As we don't really have a Finch domain yet, I made an email alias from my personal domain: [email protected]. Let's just use it for now as a hard-coded value.

Okay cool. How about Finch CI for the username so it's easy to distinguish automated CI deploys vs deploys made by you?

borrowing ideas from https://github.com/traneio/arrows/blob/master/build/build.sh#L21

Yep. Finch CI sounds good!

vkostyukov · 2018-10-08T22:54:29Z

build/build.sh

+    openssl aes-256-cbc -pass env:ENCRYPTION_PASSWORD -in ./build/credentials.sbt.enc -out local.credentials.sbt -d
+    openssl aes-256-cbc -pass env:ENCRYPTION_PASSWORD -in ./build/deploy_key.pem.enc -out local.deploy_key.pem -d
+
+    if [[ "${TRAVIS_BRANCH}" == "master" && $(cat version.sbt) != *"SNAPSHOT"* ]]; then


What does commit the new version back? Or maybe we aren't doing it yet?

The magic seems to be part of the sbt-release functionality defined here
https://github.com/sbt/sbt-release/blob/v1.0.9/src/main/scala/ReleaseExtra.scala#L77-L93

Used as part of the release process defined here https://github.com/finagle/finch/pull/995/files#diff-fdc3abdfd754eeb24090dbd90aeec2ceR131

TBH, i haven't tested the release process yet. I have an internal jfrog account where i can test with a sandbox repo. Maybe i can create a dummy repo to test out the versioning. I also need to read up a bit more on sbt-release and understand how the next version gets promoted. In theory it should be 0.26.0-SNAPSHOT when a new release is performed. Maybe setting to 1.0.0-SNAPSHOT is a manual step before committing to the 1.0.0 milestone

Wasn't super familiar with this plugin but seems like it can do a bit more than what is currently being done. Seems like it can apparently automate release notes?

vkostyukov · 2018-10-08T22:57:13Z

.travis.yml

@@ -14,5 +17,6 @@ cache:
    - $HOME/.sbt/boot/
 script:
  - sbt ++$TRAVIS_SCALA_VERSION validate
+# - "./build/build.sh"


Maybe we should just give up on built matrix and start using build.sh? It seems to be what Flavio is doing for Arrows (https://github.com/traneio/arrows/blob/master/.travis.yml).

makes sense to me!

@fwbrasil i see you went through a couple of different iterations on doing cross builds to finally hardcoding 2.12 in your build script. Any helpful gotchas that you discovered in the process?

vkostyukov · 2018-10-09T05:37:49Z

build.sbt

@@ -75,6 +103,10 @@ lazy val publishSettings = Seq(
      Some("releases"  at nexus + "service/local/staging/deploy/maven2")
  },
  publishArtifact in Test := false,
+  pgpSecretRing := file("local.secring.gpg"),


Should we add local.* into .gitignore?

Also, what's interesting is I have gpg 2.2.10 installed locally and it doesn't use the secring.gpg file (according to this doc):

A secret keyring as used by GnuPG versions before 2.1. It is not used by GnuPG 2.1 and later.

I'm not sure which version of gpg utility sbt-gpg depends on so perhaps I should downgrade to make sure I generate both files.

Nevermind. I'll modify the .gitignore in my separate branch.

Adding to the .gitignore makes sense to me to prevent accidental commits. As to the GPG situation i can look into the sbt-release plugin to see what version that it supports. Maybe that's no longer a required setting for newer versions of GPG.

I decided to use the older version of GPG I installed via brew. I'm pretty sure that version works just fine with sbt-gpg plugin:

$ gpg --version gpg (GnuPG) 1.4.18 Copyright (C) 2014 Free Software Foundation, Inc.

vkostyukov · 2018-10-09T07:02:48Z

@arron-green, I think I did everything right: #999 (although I haven't really checked anything).

vkostyukov

I will merge this into auto-releases when Travis is green (just changed the base branch).

vkostyukov · 2018-10-09T06:35:56Z

build.sbt

@@ -75,6 +103,10 @@ lazy val publishSettings = Seq(
      Some("releases"  at nexus + "service/local/staging/deploy/maven2")
  },
  publishArtifact in Test := false,
+  pgpSecretRing := file("local.secring.gpg"),


Nevermind. I'll modify the .gitignore in my separate branch.

* starts the effort to simplify artifact release * lint fix * hardcoded name and email

starts the effort to simplify artifact release

adc2a08

vkostyukov reviewed Oct 8, 2018

View reviewed changes

arron-green added 2 commits October 8, 2018 23:30

lint fix

dd62ab3

hardcoded name and email

1f0268c

vkostyukov reviewed Oct 9, 2018

View reviewed changes

vkostyukov mentioned this pull request Oct 9, 2018

Add encoded keys/credentials/etc #999

Merged

vkostyukov changed the base branch from master to auto-releases October 9, 2018 17:16

vkostyukov approved these changes Oct 9, 2018

View reviewed changes

Merge branch 'auto-releases' into fix-977

87c2df4

vkostyukov merged commit e951831 into finagle:auto-releases Oct 9, 2018

vkostyukov pushed a commit that referenced this pull request Oct 9, 2018

starts the effort to simplify artifact release (#995)

3706e9e

* starts the effort to simplify artifact release * lint fix * hardcoded name and email

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

starts the effort to simplify artifact release #995

starts the effort to simplify artifact release #995

arron-green commented Oct 6, 2018

codecov-io commented Oct 6, 2018 •

edited

Loading

vkostyukov commented Oct 8, 2018

arron-green commented Oct 8, 2018

vkostyukov Oct 8, 2018

vkostyukov Oct 8, 2018

vkostyukov Oct 8, 2018

vkostyukov Oct 8, 2018

vkostyukov Oct 8, 2018

arron-green Oct 9, 2018

arron-green Oct 9, 2018

vkostyukov Oct 9, 2018

vkostyukov Oct 8, 2018

arron-green Oct 9, 2018

vkostyukov Oct 8, 2018

arron-green Oct 9, 2018

arron-green Oct 9, 2018

vkostyukov Oct 9, 2018

vkostyukov Oct 9, 2018 •

edited

Loading

vkostyukov Oct 9, 2018

arron-green Oct 9, 2018

vkostyukov Oct 9, 2018

vkostyukov commented Oct 9, 2018 •

edited

Loading

vkostyukov left a comment

vkostyukov Oct 9, 2018

starts the effort to simplify artifact release #995

starts the effort to simplify artifact release #995

Conversation

arron-green commented Oct 6, 2018

codecov-io commented Oct 6, 2018 • edited Loading

Codecov Report

vkostyukov commented Oct 8, 2018

arron-green commented Oct 8, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vkostyukov Oct 9, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vkostyukov commented Oct 9, 2018 • edited Loading

vkostyukov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Oct 6, 2018 •

edited

Loading

vkostyukov Oct 9, 2018 •

edited

Loading

vkostyukov commented Oct 9, 2018 •

edited

Loading