[gnutls-devel] libtasn1 | GitLab config: Job 'x86' is retried in case of failures (#23)

[Development of GNU's TLS library]

Sebastian P_ created an issue:


  To fully benefit from the advantages of CI/CD, developers need to follow certain principles. Many of these principles have been introduced in the landmark book [Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation](https://www.oreilly.com/library/view/continuous-delivery-reliable/9780321670250/) and are nowadays widely accepted. One of these principles is:

> The build process has to be deterministic. Flaky behavior, e.g., tests that sometimes fail, should be avoided at all cost, because this causes maintenance issues. However, addressing this issue through retries after failures might not only hide the underlying problem, but makes issues also harder to debug when they only occur sometimes.

**Problem**: We analyzed your project and found that the file [.gitlab-ci.yml](https://gitlab.com/gnutls/libtasn1/blob/master/.gitlab-ci.yml#L34) (line 34) violates this principle. The job `x86` (in stage `test`) is set to be retried in case of failures:

```
x86:
    ...
    retry: 1
    ...
```

**Suggested Fix:** Remove ```retry: 1``` from the job definition or consider adding the option ```when``` to specify failures cases (e.g., ```runner_system_failure```) in which the job can be retried without hiding flaky tests.

**Disclaimer:**
This issue has been automatically reported by [CD-Linter](https://bitbucket.org/sealuzh/cd-linter/), a tool developed at the [University of Zurich](https://www.uzh.ch/) that detects CI/CD violations in the GitLab CI/CD pipeline configuration.
We are currently evaluating the effectiveness of our tool and we are monitoring this issue.

*Please up/downvote this issue to indicate whether you agree/disagree with the report.*

-- 
Reply to this email directly or view it on GitLab: https://gitlab.com/gnutls/libtasn1/issues/23
You're receiving this email because of your account on gitlab.com.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnupg.org/pipermail/gnutls-devel/attachments/20190816/311313fb/attachment.html>