mirror of
https://github.com/saltstack/salt.git
synced 2025-04-17 10:10:20 +00:00
Thomas S Hatch
commit
a628778e3c
2 changed files with 294 additions and 0 deletions
|
|
@ -283,3 +283,10 @@ Run the test suite with following command:
|
|||
./setup.py test
|
||||
|
||||
See :doc:`here <tests/index>` for more information regarding the test suite.
|
||||
|
||||
Issue and Pull Request Labeling System
|
||||
--------------------------------------
|
||||
|
||||
SaltStack uses several labeling schemes to help facilitate code contributions
|
||||
and bug resolution. See the :doc:`<labels-and-milestones>` documentation for
|
||||
more information.
|
||||
|
|
|
|||
287
doc/topics/development/labels.rst
Normal file
287
doc/topics/development/labels.rst
Normal file
|
|
@ -0,0 +1,287 @@
|
|||
.. _labels-and-milestones:
|
||||
|
||||
============================
|
||||
GitHub Labels and Milestones
|
||||
============================
|
||||
|
||||
SaltStack uses several labeling schemes, as well as applying milestones, to triage incoming issues and pull requests in
|
||||
the GitHub Issue Tracker. Most of the labels and milestones are used for internal tracking, but the following
|
||||
definitions might prove useful for the community to discover the best issues to help resolve.
|
||||
|
||||
Milestones
|
||||
==========
|
||||
|
||||
Milestones are most often applied to issues, as a milestone is assigned to every issue that has been triaged. However,
|
||||
milestones can also be applied to pull requests. SaltStack uses milestones to track bugs or features that should be
|
||||
included in the next major feature release, or even the next bug-fix release, as well as what issues are ready to be
|
||||
worked on or what might be blocked. All incoming issues must have a milestone associated with them.
|
||||
|
||||
Approved
|
||||
Used to indicate that this issue has all of the needed information and is ready to be worked on.
|
||||
|
||||
Blocked
|
||||
Used to indicate that the issue is not ready to be worked on yet. This typically applies to issues that have been
|
||||
labeled with “Info Needed”, “Question”, “Expected Behavior”, “Won’t Fix for Now”, etc.
|
||||
|
||||
Dot or Bug-fix Release
|
||||
Used to help filter/identify what issues must be fixed before the release such as 2014.7.4 or 2015.2.3. This
|
||||
milestone is often used in conjunction with the ``Blocker`` label, but not always.
|
||||
|
||||
Feature Release
|
||||
Similar to the Dot or Bug-fix Release milestone, but for upcoming feature releases such as Boron, Carbon, etc.
|
||||
This milestone is often used in conjunction with the ``Blocker`` label, but not always.
|
||||
|
||||
Labels
|
||||
======
|
||||
|
||||
Labels are used to facilitate the resolution of new pull requests and open issues. Most labels are confined to being
|
||||
applied to either issues or pull requests, though some labels may be applied to both.
|
||||
|
||||
Issue Labels
|
||||
------------
|
||||
|
||||
All incoming issues should be triaged with at least one label and a milestone. When a new issue comes in, it should be
|
||||
determined if the issue is a bug or a feature request, and either of those labels should be applied accordingly. Bugs
|
||||
and Feature Requests have differing labeling schemes, detailed below, where other labels are applied to them to further
|
||||
help contributors find issues to fix or implement.
|
||||
|
||||
There are some labels, such as ``Question`` or some of the "Status" labels that may be applied as "stand alone" labels
|
||||
in which more information may be needed or a decision must be reached on how to proceed. (See the "Bug Status Labels"
|
||||
section below.)
|
||||
|
||||
Features
|
||||
~~~~~~~~
|
||||
|
||||
The ``Feature`` label should be applied when a user is requesting entirely new functionality. This can include new
|
||||
functions, modules, states, modular systems, flags for existing functions, etc. Features *do not* receive severity
|
||||
or priority labels, as those labels are only used for bugs. However, they may receive "Functional Area" labels or "ZD".
|
||||
|
||||
Feature request issues will be prioritized on an "as-needed" basis using milestones during SaltStack's feature release
|
||||
and sprint planning processes.
|
||||
|
||||
Bugs
|
||||
~~~~
|
||||
|
||||
All bugs should have the ``Bug`` label as well as a severity, priority, functional area, and a status, as applicable.
|
||||
|
||||
Severity
|
||||
^^^^^^^^
|
||||
|
||||
How severe is the bug? SaltStack uses four labels to determine the severity of a bug: ``Blocker``, ``Critical``,
|
||||
``High``, and ``Medium``. This scale is intended to make the bug-triage process as objective as possible.
|
||||
|
||||
Blocker
|
||||
Should be used sparingly to indicate must-have fixes for the impending release.
|
||||
|
||||
Critical
|
||||
Applied to bugs that have data loss, crashes, hanging, unresponsive system, etc.
|
||||
|
||||
High Severity
|
||||
Any bug report that contains incorrect functionality, bad functionality, a confusing user experience, etc.
|
||||
|
||||
Medium Severity
|
||||
Applied to bugs that are about cosmetic items, spelling, spacing, colors, etc.
|
||||
|
||||
Priority
|
||||
^^^^^^^^
|
||||
|
||||
In addition to using a bug severity to classify issues, a priority is also assigned to each bug to give further
|
||||
granularity in searching for bugs to fix. In this way, a bug's priority is defined as follows:
|
||||
|
||||
P1
|
||||
Very likely. Everyone will see the bug.
|
||||
|
||||
P2
|
||||
Somewhat likely. Most will see the bug, but a few will not.
|
||||
|
||||
P3
|
||||
Half will see the bug, about half will not.
|
||||
|
||||
P4
|
||||
Most will not see the bug. Usually a very specific use case or corner case.
|
||||
|
||||
.. note::
|
||||
|
||||
A bug's priority is relative to its functional area. If a bug report, for example, about ``gitfs`` includes details
|
||||
indicating that everyone who ``gitfs`` will run into this bug, then a ``P1`` label will be applied, even though
|
||||
Salt users who are not enabling ``gitfs`` will see the bug.
|
||||
|
||||
Functional Areas
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
All bugs should receive a "Functional Area" label to indicate what region of Salt the bug is mainly seen in. This will
|
||||
help internal developers as well as community members identify areas of expertise to find issues that can be fixed more
|
||||
easily. Functional Area labels can also be applied to Feature Requests.
|
||||
|
||||
Functional Area Labels, in alphabetical order, include:
|
||||
|
||||
* Core
|
||||
* Documentation
|
||||
* Execution Module
|
||||
* File Servers
|
||||
* Multi-Master
|
||||
* Packaging
|
||||
* Pillar
|
||||
* Platform Mgmt.
|
||||
* RAET
|
||||
* Returners
|
||||
* Salt-API
|
||||
* Salt-Cloud
|
||||
* Salt-SSH
|
||||
* Salt-Syndic
|
||||
* State Module
|
||||
* Windows
|
||||
* ZMQ
|
||||
|
||||
Bug Status Labels
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
Status lables are used to define and track the state a bug is in at any given time. Not all bugs will have a status
|
||||
label, but if a SaltStack employee is able to apply a status label, he or she will. Status labels are somewhat unique
|
||||
in the fact that they might be the only label on an issue, such as ``Pending Discussion``, ``Info Needed``, or
|
||||
``Expected Behavior`` until further action can be taken.
|
||||
|
||||
Cannot Reproduce
|
||||
Someone from the SaltStack team has tried to reproduce the bug with the given information but they are unable to
|
||||
replicate the problem. More information will need to be provided from the original issue-filer before proceeding.
|
||||
|
||||
Confirmed
|
||||
A SaltStack engineer has confirmed the reported bug and provided a simple way to reproduce the failure.
|
||||
|
||||
Duplicate
|
||||
The issue has been reported already in another report. A link to the other bug report must be provided. At that
|
||||
point the new issue can be closed. Usually, the earliest bug on file is kept as that typically has the most
|
||||
discussion revolving around the issue, though not always. (This can be a "stand-alone" label.)
|
||||
|
||||
Expected Behavior
|
||||
The issue reported is expected behavior and nothing needs to be fixed. (This can be a "stand-alone" label.)
|
||||
|
||||
Fixed Pending Verification
|
||||
The bug has been fixed and a link to the applicable pull request(s) has been provided, but confirmation is being
|
||||
sought from the community member(s) involved in the bug to test and confirm the fix.
|
||||
|
||||
Info Needed
|
||||
More information about the issue is needed before proceeding such as a versions report, a sample state, the command
|
||||
the user was running, or the operating system the error was occurring on, etc. (This can be a "stand-alone" label.)
|
||||
|
||||
Upstream Bug
|
||||
The reported bug is something that cannot be fixed in the Salt code base but is instead a bug in another library
|
||||
such a bug in ZMQ or Python. When an issue is labeled with ``Upstream Bug`` then a bug report in the upstream
|
||||
project must be filed (or found if a report already exists) and a link to the report must be provided to the issue
|
||||
in Salt for tracking purposes. (This can be a stand-alone label.)
|
||||
|
||||
Won't Fix for Now
|
||||
The SaltStack team has acknowledged the issue at hand is legitimate, but made the call that it’s not something
|
||||
they’re able or willing to fix at this time. These issues may be revisited in the future.
|
||||
|
||||
Other
|
||||
~~~~~
|
||||
|
||||
There are a couple of other labels that are helpful in categorizing bugs that are not included in the categories above.
|
||||
These labels can either stand on their own such as ``Question`` or can be applied to bugs or feature requests as
|
||||
applicable.
|
||||
|
||||
Low Hanging Fruit
|
||||
Applied to bugs that should be easy to fix. This is useful for new contributors to know where some simple things
|
||||
are to get involved in contributing to salt.
|
||||
|
||||
Question
|
||||
Used when the issue isn’t a bug nor a feature, but the user has a question about expected behavior, how something
|
||||
works, is misunderstanding a concept, etc. This label is typically applied on its own with ``Blocked`` milestone.
|
||||
|
||||
Regression
|
||||
Helps with additional filtering for bug fixing. If something previously worked and now does not work, as opposed to
|
||||
something that never worked in the first place, the issue should be treated with greater urgency.
|
||||
|
||||
ZD
|
||||
Stands for “Zendesk” and is used to help track bugs that customers are seeing as well as community members. Bugs
|
||||
with this label should be treated with greater urgency.
|
||||
|
||||
Pull Request Labels
|
||||
-------------------
|
||||
|
||||
SaltStack also applies various labels to incoming pull requests. These are mainly used to help SaltStack engineers
|
||||
easily identify the nature the changes presented in a pull request and whether or not that pull request is ready to be
|
||||
reviewed and merged into the Salt codebase.
|
||||
|
||||
Type of Change
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
A "* Change" label is applied to each incoming pull request. The type of change label that is applied to a pull request
|
||||
is based on a scale that encompasses the number of lines affected by the change in conjunction with the area of code
|
||||
the change touches (i.e. core code areas vs. execution or state modules).
|
||||
|
||||
The conditions given for these labels are recommendations, as the pull request reviewer will also consult their
|
||||
intuition and experience regarding the magnitude of the impact of the proposed changes in the pull request.
|
||||
|
||||
Core code areas include: state compiler, crypto engine, master and minion, transport, pillar rendering, loader,
|
||||
transport layer, event system, salt.utils, client, cli, logging, netapi, runner engine, templating engine, top file
|
||||
compilation, file client, file server, mine, salt-ssh, test runner, etc.
|
||||
|
||||
* Minor Change
|
||||
* Less than 64 lines changed, or
|
||||
* Less than 8 core lines changed
|
||||
* Medium Change
|
||||
* Less than 256 lines changed, or
|
||||
* Less than 64 core lines changed
|
||||
* Master Change
|
||||
* More than 256 lines changed, or
|
||||
* More than 64 core lines changed
|
||||
* Expert Change
|
||||
* Needs specialized, in-depth review
|
||||
|
||||
Back-port Labels
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
There are two labels that are used to keep track of what pull requests need to be back-ported to an older release branch
|
||||
and which pull requests have already been back-ported.
|
||||
|
||||
Bugfix - back-port
|
||||
Indicates a pull request that needs to be back-ported. Once the back-port is completed, the back-porting pull request
|
||||
is linked to the original pull request and this label is removed.
|
||||
|
||||
Bugfix - [Done] back-ported
|
||||
Indicates a pull request that has been back-ported to another branch. The pull request that is responsible for the
|
||||
backport should be linked to this original pull request.
|
||||
|
||||
Testing Labels
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
There are a couple of labels that the QA team uses to indicate the mergability of a pull request. If the pull request is
|
||||
legitimately passing or failing tests, then one or more of these labels may be applied.
|
||||
|
||||
Lint
|
||||
If a pull request fails the test run, but the only failures are related pylint errors, this label will be applied to
|
||||
indicate that pylint needs to be fixed before proceeding.
|
||||
|
||||
Pending Changes
|
||||
Indicates that additional commits should be added to the original pull request before the pull request is merged
|
||||
into the codebase. These changes are unrelated to fixing tests and are generally needed to round out any unfinished
|
||||
pull requests.
|
||||
|
||||
Tests Passed
|
||||
Sometimes the Jenkins test run encounters problems, either tests that are known to have reliability issues or a
|
||||
test VM failed to build, but the problems are not related to the code changed in the pull request. This label is
|
||||
used to indicate that someone has reviewed the test failures and has deemed the failures to be non-pertinent.
|
||||
|
||||
Other Pull Request Labels
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Awesome
|
||||
Applied to pull requests that implemented a cool new feature or fixed a bug in an excellent way.
|
||||
|
||||
Labels that Bridge Issues and Pull Requests
|
||||
===========================================
|
||||
|
||||
Needs Testcase
|
||||
Used by SaltStack's QA team to realize where pain points are and to bring special attention to where some test
|
||||
coverage needs to occur, especially in areas that have regressed. This label can apply to issues or pull requests,
|
||||
which can also be open or closed. Once tests are written, the pull request containing the tests should be linked to
|
||||
the issue or pull request that originally had the ``Needs Testcase`` label. At this point, the ``Needs Testcase``
|
||||
label must be removed to indicate that tests no longer need to be written.
|
||||
|
||||
Pending Discussion
|
||||
If this label is applied to an issue, the issue may or may not be a bug. Enough information was provided about the
|
||||
issue, but some other opinions on the issue are desirable before proceeding. (This can be a "stand-alone" label.)
|
||||
If the label is applied to a pull request, this is used to signal that further discussion must occur before a
|
||||
decision is made to either merge the pull request into the code base or to close it all together.
|
||||
Loading…
Add table
Reference in a new issue