G
YouTube TikTok Games
Skip to content

[Feature][framework] CI/CD Domain Layer Entities Design & Development #2537

New issue
New issue
Closed
Closed
[Feature][framework] CI/CD Domain Layer Entities Design & Development#2537
Assignees
mappjzcwarren830abeizn
Labels
type/feature-requestThis issue is a proposal for something new
Milestone
v0.13.0

Description

@klesh
klesh
opened on Jul 19, 2022

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

Where we at

The following status is how we support the CI/CD data analysis :

  1. We collect Gitlab Pipeline data into Tool Layer as _tool_gitlab_pipeline
  2. We collect Jenkins data into both Tool Layer and Domain Layer as _tool_jenkins_jobs _tool_jenkins_builds jobs and builds

In another word, we modeled the CI/CD Domain Layer Tables based on Jenkins, and the Gitlab Pipeline data never get into Domain Layer because they are quite different in terms of modeling. This is also true for other kinds of CI/CD systems.

Why we need it

However, CI/CD plays an important role during the R&D process, metrics like Build Success Rate, etc (@Startrekzky please add more examples) depend on it.

What is the problem

CI/CD modeling varies in different platforms, there is no de facto modeling. It is hard to design a sensible model for CI/CD Domain Layer Data.

How should we approach

I propose that we address the problem by the following step:

  1. Set up a couple of designated charts depending on CI/CD data that we would like to have @Startrekzky
  2. Assign a Veteran Developer to investigate the modeling of the most popular platforms, such as Github/Gitlab/Jenkins/ArgoCD, and propose a CI/CD Domain Layer Table design
  3. The design shall be evaluated by the PPMC members in 3 workdays while all Committers could share their opinions
  4. A Data-Oriented PoC (a set of tables with data from different platforms) should be presented for the Analyst (@Startrekzky @leglar @hezyin) to evaluate
  5. And then we implement

Use case

  1. We may create Standard CI/CD charts based on Unified Domain Layer Tables
  2. Users may exploit the Unified CI/CD Domain Layer Tables to create customized charts

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. klesh
    added
    type/feature-requestThis issue is a proposal for something new
    on Jul 19, 2022
  2. klesh
    added this to the v0.13.0 milestone on Jul 19, 2022

  3. Startrekzky commented on Jul 19, 2022

    @Startrekzky
    Startrekzky
    on Jul 19, 2022
    Contributor

    CI/CD Metrics：

    1. No. of Builds (Similar to Deployment Frequency- D in DORA, which can be interpreted as the total No. of builds that meet a specific condition)
    2. Build success rate %
    3. Build result distribution %
    4. Build duration
    5. Mean Lead Time for Changes (Depending on the definition of a change, it may be a “build” that meets specific conditions.)
    6. Mean Time to Recovery (MTTR) (Depending on the definition of recovery, it may start when a “bug” is created, and end when it's fixed.)
    7. Change Failure Rate (Depending on the definition of a change and failure, it may be the failing percentage of “build” that meets specific conditions.)

    Analysis Dimensions

    1. time: e.g. measure the average build duration over the calendar date
    2. job: e.g. compare the average build duration under different jobs
    3. repo (code base): e.g. compare the average build duration triggered by different repos
  4. klesh
    assigned
    warren830
    on Jul 22, 2022

  5. warren830 commented on Jul 22, 2022

    @warren830
    warren830
    on Jul 22, 2022
    Contributor

    I created three issues for separate CI/CD tools.
    We will investigate those tools first, and after a new discussion, we will implement the collector and extractor of the plugins.

  6. warren830 commented on Jul 25, 2022

    @warren830
    warren830
    on Jul 25, 2022
    Contributor

    According to our discussion from mailing list, Metrics to be considered:

    1. No. of Builds (Similar to Deployment Frequencyhttps://linearb.io/blog/dora-engineering-metrics/#deployment-frequency- D in DORA, which can be interpreted as the total No. of builds that meet a specific condition)
    2. Build success rate %
    3. Build result distribution %
    4. Build duration in hour -- should this be minute? @Startrekzky
    5. Mean Lead Time for Changeshttps://linearb.io/blog/dora-engineering-metrics/#mean-lead-time-for-changes (Depending on the definition of a change, it may be a “build” that meets specific conditions. measure it beginning from merging the code to the main branch.)
    6. Mean Time to Recovery (MTTR)https://linearb.io/blog/dora-engineering-metrics/#mean-time-to-recovery-mttr (Depending on the definition of recovery, it may start when a “bug” is created, and end when it's fixed.)
    7. Change Failure Ratehttps://linearb.io/blog/dora-engineering-metrics/#change-failure-rate (Depending on the definition of a change and failure, it may be the failing percentage of “build” that meets specific conditions. Normally, this metric is calculated by counting the number of times a deployment results in a failure and dividing by the number of total deployments to get an average number. A lower average is better.)

    Analysis Dimensions to be considered:

    1. time: e.g. measure the average build duration over the calendar date
    2. job: e.g. compare the average build duration under different jobs
    3. repo (code base): e.g. compare the average build duration triggered by different repos

    In addition, the naming of tables should avoid confusion for users of any CI/CD tools, such as Jenkins, GitLabCI, GitHub Actions or CircleCI. For example, job means different entity in GitLabCI and Jenkins, thus we should consider not to use it in the domain layer.

    We also need to consider Azure DevOps/CircleCI/Travis CI/AWS CodePipeline

  7. mappjzc
    self-assigned this
    on Jul 26, 2022
  8. abeizn
    self-assigned this
    on Aug 9, 2022

  9. warren830 commented on Aug 25, 2022

    @warren830
    warren830
    on Aug 25, 2022
    Contributor

    This ticket can be closed for now

  10. warren830
    closed this as completedon Aug 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

type/feature-requestThis issue is a proposal for something new

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions