The Goals Of Argument Clinic

Argument Clinic's primary goal

is to take over responsibility for all argument parsing code

inside CPython. This means that, when you convert a function

to work with Argument Clinic, that function should no longer

do any of its own argument parsing--the code generated by

Argument Clinic should be a "black box" to you, where CPython

calls in at the top, and your code gets called at the bottom,

with ``PyObject *args`` (and maybe ``PyObject *kwargs``)

magically converted into the C variables and types you need.

In order for Argument Clinic to accomplish its primary goal,

it must be easy to use. Currently, working with CPython's

argument parsing library is a chore, requiring maintaining

redundant information in a surprising number of places.

When you use Argument Clinic, you don't have to repeat yourself.

Obviously, no one would want to use Argument Clinic unless

it's solving a their problem without creating problems of

its own.

So Argument Clinic should generate correct code, and its

code should preferably be slower, and definitely should

not introduce a major speed regression. (Eventually Argument

Clinic should enable a major speedup--we should be able

to rewrite its code generator so it produces tailor-made

parsing code, rather than using the general-purpose functions

from the CPython code base, which would make for the fastest

argument parsing possible.)

Additionally, Argument Clinic must be flexible enough to

work with any approach to argument parsing. Python has

some functions with some very strange parsing behaviors;

Argument Clinic's goal is to support all of them.

Finally, the original motivation for Argument Clinic was

to provide introspection "signatures" for CPython builtins.

It used to be, the introspection query functions would throw

an exception if you passed in a builtin. With Argument

Clinic, that's a thing of the past!

One idea you should keep in mind, as you work with

Argument Clinic: the more information you give it, the

better job it'll be able to do.

Argument Clinic is admittedly relatively simple right

now. But as it evolves it will get more sophisticated,

and it should be able to do many interesting and smart

things with all the information you give it.

convert a function to work with it. Here, then, are the bare

minimum steps you'd need to follow to convert a function to

work with Argument Clinic. Note that for code you plan to

check in to CPython, you really should take the conversion farther,

using some of the advanced concepts you'll see later on in

the document (like "return converters" and "self converters").

But we'll keep it simple for this walkthrough so you can learn.

Let's dive in!

Changing and redirecting Clinic's output

It can be inconvenient to have Clinic's output interspersed with

your conventional hand-edited C code. Luckily, Clinic is configurable:

you can buffer up its output for printing later (or earlier!), or write

its output to a separate file. You can also add a prefix or suffix to

every line of Clinic's generated output.

While changing Clinic's output in this manner can be a boon to readability,

it may result in Clinic code using types before they are defined, or

your code attempting to use Clinic-generated code befire it is defined.

These problems can be easily solved by rearranging the declarations in your file,

or moving where Clinic's generated code goes. (This is why the default behavior

of Clinic is to output everything into the current block; while many people

consider this hampers readability, it will never require rearranging your

code to fix definition-before-use problems.)

Let's start with defining some terminology:

*field*

A field, in this context, is a subsection of Clinic's output.

For example, the ``#define`` for the ``PyMethodDef`` structure

is a field, called ``methoddef_define``. Clinic has seven

different fields it can output per function definition::

All the names are of the form ``"<a>_<b>"``,

where ``"<a>"`` is the semantic object represented (the parsing function,

the impl function, the docstring, or the methoddef structure) and ``"<b>"``

represents what kind of statement the field is. Field names that end in

``"_prototype"``

represent forward declarations of that thing, without the actual body/data

of the thing; field names that end in ``"_definition"`` represent the actual

definition of the thing, with the body/data of the thing. (``"methoddef"``

is special, it's the only one that ends with ``"_define"``, representing that

it's a preprocessor #define.)

*destination*

A destination is a place Clinic can write output to. There are

five built-in destinations:

``block``

The default destination: printed in the output section of

the current Clinic block.

``buffer``

A text buffer where you can save text for later. Text sent

here is appended to the end of any exsiting text. It's an

error to have any text left in the buffer when Clinic finishes

processing a file.

``file``

A separate "clinic file" that will be created automatically by Clinic.

The filename chosen for the file is ``{basename}.clinic{extension}``,

where ``basename`` and ``extension`` were assigned the output

from ``os.path.splitext()`` run on the current file. (Example:

the ``file`` destination for ``_pickle.c`` would be written to

``_pickle.clinic.c``.)

**Important: When using a** ``file`` **destination, you**

*must check in* **the generated file!**

``two-pass``

A buffer like ``buffer``. However, a two-pass buffer can only

be written once, and it prints out all text sent to it during

all of processing, even from Clinic blocks *after* the

``suppress``

The text is suppressed--thrown away.

Clinic defines five new directives that let you reconfigure its output.

The first new directive is ``dump``::

This dumps the current contents of the named destination into the output of

the current block, and empties it. This only works with ``buffer`` and

``two-pass`` destinations.

The second new directive is ``output``. The most basic form of ``output``

is like this::

This tells Clinic to output *field* to *destination*. ``output`` also

supports a special meta-destination, called ``everything``, which tells

Clinic to output *all* fields to that *destination*.

``output`` has a number of other functions::

``output push`` and ``output pop`` allow you to push and pop

configurations on an internal configuration stack, so that you

can temporarily modify the output configuration, then easily restore

the previous configuration. Simply push before your change to save

the current configuration, then pop when you wish to restore the

previous configuration.

``output preset`` sets Clinic's output to one of several built-in

preset configurations, as follows:

``original``

Clinic's starting configuration.

Suppress the ``parser_prototype``

and ``docstring_prototype``, write everything else to ``block``.

``file``

Designed to write everything to the "clinic file" that it can.

You then ``#include`` this file near the top of your file.

You may need to rearrange your file to make this work, though

usually this just means creating forward declarations for various

``typedef`` and ``PyTypeObject`` definitions.

Suppress the ``parser_prototype``

and ``docstring_prototype``, write the ``impl_definition`` to

``block``, and write everything else to ``file``.

``buffer``

Save up all most of the output from Clinic, to be written into

your file near the end. For Python files implementing modules

or builtin types, it's recommended that you dump the buffer

just above the static structures for your module or

builtin type; these are normally very near the end. Using

``buffer`` may require even more editing than ``file``, if

your file has static ``PyMethodDef`` arrays defined in the

middle of the file.

Suppress the ``parser_prototype``, ``impl_prototype``,

and ``docstring_prototype``, write the ``impl_definition`` to

``block``, and write everything else to ``file``.

``two-pass``

Similar to the ``buffer`` preset, but writes forward declarations to

the ``two-pass`` buffer, and definitions to the ``buffer``.

This is similar to the ``buffer`` preset, but may require

less editing than ``buffer``. Dump the ``two-pass`` buffer

near the top of your file, and dump the ``buffer`` near

the end just like you would when using the ``buffer`` preset.

Suppresses the ``impl_prototype``, write the ``impl_definition``

to ``block``, write ``docstring_prototype``, ``methoddef_define``,

and ``parser_prototype`` to ``two-pass``, write everything else

to ``buffer``.

``partial-buffer``

Similar to the ``buffer`` preset, but writes more things to ``block``,

only writing the really big chunks of generated code to ``buffer``.

This avoids the definition-before-use problem of ``buffer`` completely,

at the small cost of having slightly more stuff in the block's output.

Dump the ``buffer`` near the end, just like you would when using

the ``buffer`` preset.

Suppresses the ``impl_prototype``, write the ``docstring_definition``

and ``parser_defintion`` to ``buffer``, write everything else to ``block``.

The third new directive is ``destination``::

This performs an operation on the destination named ``name``.

There are two defined subcommands: ``new`` and ``clear``.

The ``new`` subcommand works like this::

This creates a new destination with name ``<name>`` and type ``<type>``.

There are five destination types::

The ``clear`` subcommand works like this::

It removes all the accumulated text up to this point in the destination.

(I don't know what you'd need this for, but I thought maybe it'd be

useful while someone's experimenting.)

The fourth new directive is ``set``::

``set`` lets you set two internal variables in Clinic.

``line_prefix`` is a string that will be prepended to every line of Clinic's output;

``line_suffix`` is a string that will be appended to every line of Clinic's output.

Both of these suport two format strings:

``{block comment start}``

Turns into the string ``/*``, the start-comment text sequence for C files.

``{block comment end}``

Turns into the string ``*/``, the end-comment text sequence for C files.

The final new directive is one you shouldn't need to use directly,

called ``preserve``::

This tells Clinic that the current contents of the output should be kept, unmodifed.

This is used internally by Clinic when dumping output into ``file`` files; wrapping

it in a Clinic block lets Clinic use its existing checksum functionality to ensure

the file was not modified by hand before it gets overwritten.

Using Argument Clinic in Python files