Tutorial¶
python-args
installs the arg
module and comes with three main
decorators:
@arg.validators
- Run validators against function arguments.@arg.defaults
- Process and coerce default values for arguments.@arg.contexts
- Run context managers around functions using arguments.@arg.parametrize
- For parametrizing an argument of a function.
python-args
provides several utilities for lazily-evaluating code within
the decorators:
arg.func
- Run a lazy function.arg.init
- Lazily initialize a class.arg.val
- A shortcut to return the value of an argument to a function.arg.first
- A shortcut to return the value of the first loadable lazy object.
As we will discuss later, the combination of these utilities offers the ability to construct functions in a more digestible and testable manner.
We will cover these decorators and lazy utilities in the following, along with
covering some additional features such as arg.s
. We start
off with @arg.validators
.
Using @arg.validators
¶
A common design when writing python functions is validating the parameters to a function. For example, consider the following function that sends an email to an address:
def send_email(email_address, message):
if not _validate_email(email_address):
raise ValueError(f'Email address "{email_address}" is invalid.')
if not message:
raise ValueError('Must provide message')
# Contact the SMTP server to send the email.
_send_email(email_address, message)
# Create a record that the email was sent
EmailRecord.objects.create(email_address, message)
In this example, we:
Check that a valid email address was supplied.
Ensure the user passed a message that wasn’t empty.
Call the actual code to do the request that sends the email.
Create a record in our database about the email being successfully sent (we are using Django in this example, but that’s not important).
When writing code in this way, it is more difficult to construct unit tests
for some of the core business logic of the send_email
function. For
example, in order to test that we are creating an appropriate email record
when we send an email, we have to be sure to set up the proper state so
that our validations hold true (or mock out the validations).
For this particular example, this may be trivial, but the problem can easily grow in complexity when validating arguments against database state or other conditions.
Enter @arg.validators
. The validators
decorator allows you
to construct a function and apply validators to appropriate arguments.
Let’s take the previous example:
import arg
def validate_email(email_address):
if not _validate_email(email_address):
raise ValueError(f'Email address "{email_address}" is invalid.')
def validate_message(message):
if not message:
raise ValueError('Must provide message')
@arg.validators(validate_email, validate_message)
def send_email(email_address, message):
message = message.strip()
email_address = email_address.lower()
with transaction.atomic():
# Create a record that we sent the email
EmailRecord.objects.create(email_address, message)
# Hit our email server and send it. If an error happens,
# our transaction will be rolled back and we won't store the record
send_email_via_smtp(email_address, message)
The function above has the same behavior as calling:
def send_email(email_address, message):
validate_email(email_address)
validate_message(message)
message = message.strip()
email_address = email_address.lower()
with transaction.atomic():
...
python-args
knows how to orchestrate this by inspecting the argument
names to the original function and calling validators with matching
argument names.
Note
Validators can take any subset of the arguments of the calling function
based on the argument names. If the argument names don’t match, keep
reading about @arg.defaults
later for ways around this.
When structuring your code with @arg.validators
,
you get the following:
Validation functions that are completely separate and can be tested in isolation.
The ability to run only the validators of your function. Using our example above, one can call
send_email.pre_func(email_address, message)
to run all code that is executed before the main function (in this case, the validators).The ability to run the function without validators. Using our example above, one can call
send_email.func(email_address, message)
to only run the wrapped function.
With these characteristics in mind, one now has more tools at their disposal for constructing unit tests that focus on core business logic instead of setting up state (or mocking it out).
Along with this, higher-level tools can more seamlessly integrate with
python-args
functions. For example, it is possible to integrate the
validators of this function with a Django form that calls the function
with user-supplied arguments, all while keeping the validation close to
the core logic.
Note
python-args
currently only supports validators that throw exceptions
when failing. We are considering some extensions that allow validators
that return bool
values.
Using @arg.defaults
¶
Similar to validators, another common pattern is to process the default value for an argument into something usable for the function. For example, consider the classic case of avoiding using mutable keyword argument defaults:
def my_kwarg_func(my_kwarg=None):
my_kwarg = my_kwarg or []
...
Another common use case is stripping string values:
def my_str_func(str_arg):
str_arg = str_arg.strip()
...
The @arg.defaults
decorator allows one to apply default
processing to arguments before the function is called. Let’s take our
two examples above and convert them to use @arg.defaults
:
@arg.defaults(my_kwarg=lambda: my_kwarg or [])
def my_kwarg_func(my_kwarg=None)
...
@arg.defaults(str_arg=lambda: str_arg.strip())
def my_str_func(str_arg):
...
@arg.defaults
takes the argument name and its associated logic
for processing it. Although we are using a lambda
here,
@arg.defaults
values can take functions and other lazy
utilities offered by python-args
(more on this later).
Similar to @arg.validators
, the same principles apply here -
One can write and test default processors more elegantly in isolation while
keeping focus on core business logic.
@arg.defaults
also allows us to preprocess default values
before validators run. For example, take our previous example using
@arg.validators
:
@arg.validators(validate_email, validate_message)
def send_email(email_address, message):
message = message.strip()
email_address = email_address.lower()
...
In the above, the validate_email
and validate_message
validators
also have to preprocess the arguments before running validation. This can
be solved with stacking the decorators in the order in which they should
be applied:
@arg.defaults(email_address=lambda email_address: email_address.lower(),
message=lambda message: message.strip())
@arg.validators(validate_email, validate_message)
def send_email(email_address, message):
...
Using @arg.contexts
¶
Sometimes resources need to be created before a function and destroyed after
its execution or instrumentation needs to be put in place. Context managers
are the preferred python design pattern for this, and python-args
comes with the @arg.contexts
decorator to enter and leave
context managers.
For example, the following context manager logs a message before and after execution:
import contextlib
import logging
import args
@contextlib.contextmanager
def log_func():
logging.info('Starting')
yield
logging.info('Finishing')
@arg.contexts(log_func)
def my_func(arg):
...
Similar to other python-args
decorators, context managers can take named
arguments that are named the same as the underlying function.
Need to attach a value from a context manager to an argument name before
the execution of the function? Similar to @arg.defaults
,
@arg.contexts
can re-assign the argument before
execution.
For example, consider the pattern of a function that can either take in a file name or an already-open file object:
import os
def read_file_contents(file_obj):
"""Read the file contents of the file object.
If the file object is a string, open the file and read it.
"""
if isinstance(file_obj, str):
with open(file_obj, 'r') as f:
return f.read()
else:
return f.read()
By using @arg.contexts
with a label for the context manager,
the result of the context manager will be used for the argument. For example,
import os
@contextlib.contextmanager
def ensure_file_obj(file_obj):
if isinstance(file_obj, str):
with open(file_obj, 'r') as file_obj:
yield file_obj
else:
yield file_obj
@arg.contexts(file_obj=ensure_file_obj)
def read_file_contents(file_obj):
return f.read()
With this pattern, the ensure_file_obj
context manager can be re-used
for this particularly ugly scenario of handling a file name or file-like object.
Using @arg.parametrize
¶
Similar to the parametrization in
pytest, python-args
allows
one to parametrize the input to a function using
@arg.parametrize
. Here’s an example of a function
that doubles a number and an associated parametrization:
@arg.parametrize(number=arg.val('numbers'))
def double(number):
return val * 2
assert double(numbers=[1, 3, 4, 5]) == [2, 6, 8, 10]
Similar to @arg.defaults
, @arg.parametrize
can bind an argument from another value. In the case of
@arg.parametrize
, the value must be an iterable.
When used, the resulting function returns a list of all parametrized results.
Note
@arg.parametrize
can only parametrize one argument
at a time. Nesting @arg.parametrize
will result
in a list that contains other lists.
Accessing properties of the current call¶
Each run of a function decorated with python-args
stores global
state about the current call. This information can be gathered with
arg.call
and can be called inside of any function supplied to the
primary python-args
decorators.
Below is a code example of a context manager that is used as
a python-args
context. The docs for the code elaborate on what
various properties mean.
import arg
@contextlib.context
def my_args_context():
# Get the current python-args call. This will raise an error if
# called outside of a python-args function
c = arg.call()
# When this flag is true, we are running in partial mode and
# are only running python-args decorators that can be bound
# to the calling arguments
c.is_partial
# When this flag is true, we are running in pre_func mode.
# This means we are only running everything up to the main function.
# Sometimes context managers might want to use this mode to
# alter their run-time characteristics
c.is_pre_func
# All of these arguments are set when running under a parametrization
# of an argument.
c.parametrized_arg # The argument name being parametrized
c.parametrized_arg_val # The value of the argument
c.parametrized_arg_index # The index with respect to all values
c.parametrized_arg_vals # All values that are being parametrized
@arg.contexts(my_args_context)
def my_args_func():
...
Arg naming limitations and work-arounds¶
python-args
decorators work well when argument names are consistent.
As long as argument names match in contexts, validators, and defaults,
things will work as expected. However, in order to more easily share contexts,
validators, and defaults among code, it does require that all code uses the
same argument names. When argument names do not match, a lazy binding error
will be raised when calling the decorated function.
This is a known limitation. Here we offer a few work-arounds and some future plans for easing this burden.
For now, stacking @arg.defaults
is the best way to ensure
that argument names match. For example, let’s use our ensure_file_obj
context manager from before that uses a file_obj
argument. In the
following example, we declare a function that should take a file object, but
the argument is not named file_obj
:
def parse_file(my_file):
# Do file parsing on the file object.
In order to take advantage of our previously-declared ensure_file_obj
context, we need to process the default values before passing them into
the context:
@arg.defaults(file_obj=lambda my_file: my_file)
@arg.contexts(my_file=ensure_file_obj)
def parse_file(my_file):
# Do file parsing. my_file will be a file object
In the above, the file_obj
argument is created from the my_file
argument and passed down through the chain. Since the ensure_file_obj
expects a file_obj
argument, it will succeed and assign the proper
value to the my_file
argument before it is passed to parse_file
.
Although it is possible to create argument processing chains like this, it
is not recommended for the sake of readability. python-args
plans to
address the obvious limitation in future releases by allowing one to more
clearly specify how validators, contexts, and defaults can be called
from arguments that don’t have matching argument names.
python-args
has some shortcuts for lazy loading to
help reduce the boilerplate of writing lambda
functions.
We cover these in the next section.
Args lazy-loading shortcuts¶
For simple @arg.defaults
processing or more digestible
argument renaming, python-args
comes with a few utilities to help
the user avoid writing lambda
expressions or declaring new functions for
trivial operations.
Using arg.val
¶
arg.val
is used to retrieve the value of an argument. For example,
@arg.defaults(arg1=lambda other_arg: other_arg)
def my_func(arg1):
...
The above assigns the arg1
argument the value of other_arg
when
my_func
is called with other_arg
. This expression can be
shortened with:
@arg.defaults(arg1=arg.val('other_arg'))
def my_func(arg1):
...
arg.val
can be chained. Let’s go back to a previous example where we
ensure that a string message is stripped before executing a function:
@arg.defaults(my_str_arg=arg.val('my_str_arg').strip())
def my_func(my_str_arg):
...
Using arg.init
¶
arg.init
is a shortcut to initialize a class. Instead of:
@arg.defaults(arg1=lambda: MyClass(kwarg=arg1))
def my_func(arg1):
...
One can use arg.init
:
@arg.defaults(arg1=arg.init(MyClass, kwarg=arg.val('arg1')))
def my_func(arg1):
...
Using arg.func
¶
All functions passed to @arg.defaults
,
@arg.validators
, and @arg.contexts
are wrapped in an arg.func
call. arg.func
takes a function and
lazily binds arguments to it. This is how python-args
is able to
dynamically bind function arguments to the various decorators.
Although it is not necessary to use this utility with any
current python-args
decorators, users are able to inherit arg.func
and create other lazy utilities to use with python-args
decorators.
Using arg.first
¶
arg.first
is a shortcut to obtain the first value that can be lazily
loaded. It takes an arbitrary amount of lazy objects (such as arg.val
or arg.func
). For example:
@arg.defaults(arg1=arg.first(arg.val('b'), arg.val('c')))
def my_func(arg1):
return arg1
assert my_func(b=1, c=3) == 1
assert my_func(c=3) == 3
Similar to arg.val
, a default
keyword argument can be used to
return a default value if no arguments can be binded.
Users may also pass in strings as a shorthand
for arg.val
or callables as a shorthand for arg.func
. For example,
this is equivalent to our previous example:
@arg.defaults(arg1=arg.first('b', 'c'))
def my_func(arg1):
return arg1
Partial and pre_func run modes¶
As briefly described earlier, python-args
allows decorated functions
to be executed in various modes. One mode, the pre_func
mode, ensures
that only the code before the primary function is executed. This allows
other tools to seamlessly only run validators and other pre-processing
logic without running the underlying function. As mentioned earlier,
this interface is accessed with the pre_func
attribute on the decorated
function.
Along with the pre_func
mode, python-args
can also go a step further
and only run pre_func
routines based on the arguments provided. This is done
with the partial
attribute of the decorated function.
Partially running the pre_func
routines of the decorated function allows
us to verify that individual arguments are in good shape before running
the function.
For example, imagine one is writing a command line interface for a function
and wishes to individually validate arguments and provide useful error
messages. Assuming the function
is called my_func
, this can be done by calling
my_func.pre_func.partial(arg_name=value)
to only run the pre_func
routines associated with arg_name
.
Creating aggregate decorators with arg.s
¶
python-args
decorators such as @arg.validators
@arg.contexts
are meant to be stacked on top of one
another to create chains of processing for arguments. However,
repeating the same chains of decorators across similar functions
can become unwieldy over time.
The arg.s
utility can be used to address this and combine chains into
one single decorator. For example, let’s say that you have a function decorated
like so:
@arg.validators(validate_object)
@arg.contexts(track_object_changes, trap_errors)
def my_func(...):
...
If this pattern needs to be applied to many functions, it can be useful
to make a single decorator. This is where arg.s
comes in handy:
validate_object_and_track_changes = arg.s(
arg.validators(validate_object),
arg.contexts(track_object_changes, trap_errors),
)
@validate_object_and_track_changes
def my_func(...):
...
Note
One can similarly use validate_object_and_track_changes
as a
function runner or orchestrator by calling
validate_object_and_track_changes(function_to_run)
.