Мы недавно заметили, что наши сборки Jenkins перестали запускаться автоматически. После дальнейшего расследования возникли многочисленные проблемы.
Веб-подключения Github не увенчались успехом, так как Github сообщал «Не удалось подключиться к серверу» в пользовательском интерфейсе конфигурации веб-крючка Github. Я могу подтвердить, что наш хостинг Jenkins для экземпляров ELB и EC2 жив и здоров. Изменений DNS здесь не было.
Журналы Jenkins сообщают о различных сбоях:
Неверные учетные данные, несмотря на наличие действительных учетных данных Jenkins для имени пользователя и пароля (пароль - это личный токен API):
There is no credentials with admin access to manage hooks on GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED]
Не удалось удалить ловушку после фиксации:
ALPN callback dropped: SPDY and HTTP/2 are disabled. Is alpn-boot on the boot class path?
Apr 12, 2019 6:15:43 PM WARNING org.jenkinsci.plugins.github.webhook.WebhookManager$2 applyNullSafe
Failed to add GitHub webhook for GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED]
java.io.FileNotFoundException: https://api.github.com/repos/REDACTED/REDACTED/hooks/101704125
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
at org.kohsuke.github.Requester.parse(Requester.java:617)
at org.kohsuke.github.Requester.parse(Requester.java:599)
at org.kohsuke.github.Requester._to(Requester.java:277)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/repos/hooks/#delete-a-hook"}
at org.kohsuke.github.Requester.handleApiError(Requester.java:691)
at org.kohsuke.github.Requester._to(Requester.java:298)
at org.kohsuke.github.Requester.to(Requester.java:239)
at org.kohsuke.github.Requester.to(Requester.java:227)
at org.kohsuke.github.GHHook.delete(GHHook.java:56)
at org.jenkinsci.plugins.github.webhook.WebhookManager$10.applyNullSafe(WebhookManager.java:344)
Caused: org.kohsuke.github.GHException: Failed to delete post-commit hook
at org.jenkinsci.plugins.github.webhook.WebhookManager$10.applyNullSafe(WebhookManager.java:347)
at org.jenkinsci.plugins.github.webhook.WebhookManager$10.applyNullSafe(WebhookManager.java:341)
at org.jenkinsci.plugins.github.util.misc.NullSafePredicate.apply(NullSafePredicate.java:19)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:649)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:647)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Lists.newArrayList(Lists.java:138)
at com.google.common.collect.Lists.newArrayList(Lists.java:119)
at org.jenkinsci.plugins.github.util.FluentIterableWrapper.toList(FluentIterableWrapper.java:147)
at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:202)
at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:175)
at org.jenkinsci.plugins.github.util.misc.NullSafeFunction.apply(NullSafeFunction.java:18)
at com.google.common.collect.Iterators$8.next(Iterators.java:812)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:648)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:647)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Lists.newArrayList(Lists.java:138)
at com.google.common.collect.Lists.newArrayList(Lists.java:119)
at org.jenkinsci.plugins.github.util.FluentIterableWrapper.toList(FluentIterableWrapper.java:147)
at org.jenkinsci.plugins.github.webhook.WebhookManager$1.run(WebhookManager.java:127)
at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Не удалось создать хук:
Apr 12, 2019 6:15:44 PM WARNING org.jenkinsci.plugins.github.webhook.WebhookManager$2 applyNullSafe
Failed to add GitHub webhook for GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED]
java.io.FileNotFoundException: https://api.github.com/repos/REDACTED/REDACTED/hooks
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
at org.kohsuke.github.Requester.parse(Requester.java:617)
at org.kohsuke.github.Requester.parse(Requester.java:599)
at org.kohsuke.github.Requester._to(Requester.java:277)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Validation Failed","errors":[{"resource":"Hook","code":"custom","message":"Hook already exists on this repository"}],"documentation_url":"https://developer.github.com/v3/repos/hooks/#create-a-hook"}
at org.kohsuke.github.Requester.handleApiError(Requester.java:691)
at org.kohsuke.github.Requester._to(Requester.java:298)
at org.kohsuke.github.Requester.to(Requester.java:239)
at org.kohsuke.github.GHHooks$Context.createHook(GHHooks.java:49)
at org.kohsuke.github.GHRepository.createHook(GHRepository.java:1206)
at org.jenkinsci.plugins.github.webhook.WebhookManager$9.applyNullSafe(WebhookManager.java:329)
Caused: org.kohsuke.github.GHException: Failed to create hook
at org.jenkinsci.plugins.github.webhook.WebhookManager$9.applyNullSafe(WebhookManager.java:331)
at org.jenkinsci.plugins.github.webhook.WebhookManager$9.applyNullSafe(WebhookManager.java:316)
at org.jenkinsci.plugins.github.util.misc.NullSafeFunction.apply(NullSafeFunction.java:18)
at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:204)
at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:175)
at org.jenkinsci.plugins.github.util.misc.NullSafeFunction.apply(NullSafeFunction.java:18)
at com.google.common.collect.Iterators$8.next(Iterators.java:812)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:648)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:647)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Lists.newArrayList(Lists.java:138)
at com.google.common.collect.Lists.newArrayList(Lists.java:119)
at org.jenkinsci.plugins.github.util.FluentIterableWrapper.toList(FluentIterableWrapper.java:147)
at org.jenkinsci.plugins.github.webhook.WebhookManager$1.run(WebhookManager.java:127)
at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Это странно, потому что я также вижу положительные журналы, такие как:
GitHub webhooks activated for job REDACTED_ORG_NAME/REDACTED_REPO with [GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED_REPO]] (events: [PULL_REQUEST, PUSH])
Apr 12, 2019 6:15:43 PM INFO org.jenkinsci.plugins.github.webhook.WebhookManager$1 run
GitHub webhooks activated for job REDACTED_ORG_NAME/REDACTED_REPO/develop with [GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED_REPO]] (events: [PULL_REQUEST, PUSH])
Apr 12, 2019 6:15:43 PM INFO org.jenkinsci.plugins.github.webhook.WebhookManager$1 run
GitHub webhooks activated for job REDACTED_ORG_NAME/REDACTED_REPO/master with [] (events: [PULL_REQUEST, PUSH])
Apr 12, 2019 6:15:43 PM INFO org.jenkinsci.plugins.github.webhook.WebhookManager$1 run
GitHub webhooks activated for job REDACTED_ORG_NAME/REDACTED_REPO/release%2F0.x with [] (events: [PULL_REQUEST, PUSH]
Мы настроили следующие плагины Jenkins:
Я выполнил все шаги по устранению неполадок в https://support.cloudbees.com/hc/en-us/articles/224621648-GitHub-webhook-troubleshooting, но я не пойду куда. Неудачные шаги:
- А.2. Я попытался восстановить полезную нагрузку webhook, но получил
Couldn't connect to server
- B.4 Отображает списки
Failed to delete post-commit hook:, Failed to create hook
и There is no credentials with admin access to manage hooks on GitHubRepositoryName[host=github.com,username=REDACTED,repository=REDACTED]
Что работает
- Возможность сканирования организации
- Возможность запуска сборок вручную с помощью Build Now
Кажется, есть так много проблем, что я не знаю, где искать дальше. Заранее спасибо за помощь.