Parser
Cibyl provides two sources of user input, the configuration file and the command line arguments. The configuration file details the ci environment that the user wants to query, while the command line arguments tell Cibyl what the user wants to query.
Cibyl’s cli is divided in several subcommands. The parser is the component
responsible for bringing all the subcommands together and ensuring the
corresponding arguments are added. In the case of the features
subcommands
that is simple, since it only has one argument. The case of the query
sucommand is different, since the cli arguments are extended dynamically depending on the
contents of the configuration.
Note
The rest of this page is relevant only for the query
subcommand.
When running cibyl query -h
only the arguments that are relevant to the user,
according to its configuration, will be shown. If there is no configuration
file, Cibyl will just print a few general arguments when calling cibyl query -h
.
If the configuration is populated then arguments will be added depending on its contents.
The parser is extended using a hierarchy of CI models. This hierarchy is
Cibyl’s internal representation of the CI environments. The models are created after reading the
configuration and the hierarchy is implicitely defined in the API attribute of
said models. For example, one environment might include a Jenkins instance as
CI system, and have it also as source for information, in addition to an
ElasticSearch instance as a second source. With this environment, if the user
runs cibyl query -h
, it will show arguments that are relevant to a Jenkins
system, like --jobs
, --builds
or --build-status
. In such a case it will
not show arguments like --pipelines
which would be useful if the CI system
was a Zuul instance.
The API of a CI model is a dictionary with the following structure (extracted from the System API):
API = {
'name': {
'attr_type': str,
'arguments': []
},
'sources': {
'attr_type': Source,
'attribute_value_class': AttributeListValue,
'arguments': [Argument(name='--sources', arg_type=str,
nargs="*",
description="Source name")]
},
'jobs': {'attr_type': Job,
'attribute_value_class': AttributeDictValue,
'arguments': [Argument(name='--jobs', arg_type=str,
nargs='*',
description="System jobs",
func='get_jobs')]}
}
each key corresponds to the name of an attribute, and the value is another
dictionary with attribute-related information. At this point we need to
distinguish between arguments and attributes. In Cibyl an Argument
is the object
that is obtained from parsing the user input. The values passed to each option
like --debug
or --jobs
are stored in an Argument
. Attributes correspond to the actual
key-value pairs in the API. An attribute has an attribute_value_class
which
by default is AttributeValue
, but can also be AttributeDictValue
and AttributeListValue
.
The difference between the three is the how they store the arguments. The first
is intended to hold a single option (things like name, type, etc.). While the
other two hold a collection of values either in a dictionary or a list (hence
the name). The information provided by the user is accessible throgh the
value
field of any Attribute
class.
Each API element has also an attr_type, which describes what kind of object will it hold. In the example above name will hold a string, while jobs will hold a dictonary of Job objects. This allows us to establish the hierarchy mentioned previously, by checking if the attr_type field is not a builtin type. Finally, there is an arguments field, which associates the actual options that will be shown in the cli with an attribute. An attribute may have no arguments, one argument or multiple arguments associated with it.
Argument
objects have a set of options to configure the behavior of the
cli. The name determines the option that will be shown, arg_type specifies
the type used to store the user input (str, int, etc.), nargs and
description have the same meaning as they do in the arparse module.
The level argument, measures how deep in the hierarchy
a given model is. Finally, we see the func argument, which points to the
method a source must implement in order to provide information about a certain
model. In the example shown here, only jobs has an argument with func
defined, as it is the only CI model present. If the user runs a query like:
cibyl query --jobs
then Cibyl will look at the sources defined and check whether any has a method
get_jobs
, and if it finds one it will use it to get all the jobs available
in that source.
Arguments are added to the application parser in the extend_parser
method
of the Orchestrator
class. This method loops through the API of a model
(in the first call it will be an Environment
model) and adds its arguments. If any
of the API elements is a CI model, the element’s API is recursively used to
augment the parser. As the extend_parser
method iterates through the model
hierarchy, it creates a graph of the relationships between query methods (the
sources’ methods that are added to the arguments’ func attribute). The edges
of the graph are created when a new recursive call is made. As an example, when
exploring the API for the Job model, we know that the arguments will call
get_jobs
, so when a new call is made for the Build API, a new edge wil be
created from get_jobs
to all the new query methods that are found, in this
case it will be get_builds
.
For each recursive call, the level is increased. The level parameter is key to identify the source of information for the query that the user sends. In the Jenkins environment example mentioned before, we may have a hierarchy like:
Environment => System => Job => Build
where each at each step we increase the level by 1. We can then parse the cli
arguments and sort by decreasing level. To select which query method should be
called, cibyl relies on the graph constructed during the call to
extend_parser
. It iterates over the sorted list of arguments and for each
of them constructs a path to the root of the graph. The intermediate nodes in
this path are removed from the list of arguments to query, since by the
hierarchical nature of the relationship between the models, calling an
argument’s func makes the call to the argument’s parent func redundant.
In the example above, Build is the model with the largest level. If we assume that
user has made a call like cibyl --jobs --builds
, we want to query the sources for builds,
but we known that each build will be associated with a job, and each job will be associated with
a system, etc. We also know that after calling get_builds
, we will not need
to call get_jobs
. Thus we get a sorted list of arguments, which is [builds, jobs].
We create a path from builds to the root of the graph, which in the case of
a Jenkins systems is jobs (for a zuul system this would be more complex).
After iterating over the path, we remove jobs from the list of arguments to
query, since builds already will provide the jobs information.