From 7ff1d0006fa7f3b0204a10af14cc5141dc47fd47 Mon Sep 17 00:00:00 2001 From: Tanya Schlusser Date: Sat, 17 Jan 2015 11:55:26 -0600 Subject: [PATCH 1/6] First attemt at a section on logging. --- docs/writing/logging.rst | 180 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 180 insertions(+) create mode 100644 docs/writing/logging.rst diff --git a/docs/writing/logging.rst b/docs/writing/logging.rst new file mode 100644 index 000000000..e9e8bd169 --- /dev/null +++ b/docs/writing/logging.rst @@ -0,0 +1,180 @@ +Logging +======= + +The :mod:`logging` module has been a part of Python's Standard Library since +version 2.3. It is succinctly described in :pep:`282`. The documentation +is notoriously hard to read, except for the `basic logging tutorial`_, +and often less useful than simply reading the source code. + +Logging serves two purposes: + +- **Diagnostic logging** records events related to the application's + operation. If a user calls in to report an error, for example, the logs + can be searched for context. +- **Audit logging** records events for business analysis. A user's + transactions can be extracted and combined with other user details for + reports or to optimize a business goal. + + +... or Print Statements? +------------------------ + +The only time that ``print`` is a better option than logging is when +the goal is to display a help statement for a command line application. +Other reasons why logging is better than ``print``: + +- The `log record`_, which is created with every logging event, contains + readily available diagnostic information such as the file name, + full path, function, and line number of the logging event. +- Events logged in included modules are automatically accessible via the + root logger + to your application's logging stream, unless you filter them out. +- Logging can be selectively silenced or disabled by using the method + :meth:`logging.Logger.setLevel` or setting the attribute + :attr:`logging.Logger.disabled` to ``True``. + + +Logging in a Library +-------------------- + +Notes for `configuring logging for a library`_ are in the +`basic logging tutorial`_. Because the *user*, not the library, should +dictate what happens when a logging event occurs, One admonition bears +repeating: + +.. note:: + It is strongly advised that you do not add any handlers other than + NullHandler to your library’s loggers. + + +Best practice when instantiating loggers in a library is to only create them +using the ``__name__`` global variable: the :mod:`logging` module creates a +hierarchy of loggers using dot notation, so using ``__name__`` ensures +no name collisions. + +Here is an example of best practice from the `requests source`_ -- place +this in your ``__init__.py`` + +.. code-block:: python + + # Set default logging handler to avoid "No handler found" warnings. + import logging + try: # Python 2.7+ + from logging import NullHandler + except ImportError: + class NullHandler(logging.Handler): + def emit(self, record): + pass + + logging.getLogger(__name__).addHandler(NullHandler()) + + + +Logging in an Application +------------------------- + +The `twelve factor app's `_, an authoritative reference +for good practice in application development, contains a section on +`logging best practice `_. It emphatically +advocates for treating log events as an event stream, and for +sending that event stream to standard output to be handled by the +application environment. Do that. + + +There are at least three ways to configure a logger: + +- using a file (recommended) +- using a dictionary +- using code + +Here is how with a file -- let us say it is named ``logging_config.txt``: + +.. code-block:: none + + [loggers] + keys=root + + [handlers] + keys=stream_handler + + [formatters] + keys=formatter + + [logger_root] + level=DEBUG + handlers=stream_handler + + [handler_stream_handler] + class=StreamHandler + level=DEBUG + formatter=formatter + args=(sys.stderr,) + + [formatter_formatter] + format=%(asctime)s %(name)-12s %(levelname)-8s %(message)s + + +Then use :meth:`logging.config.fileConfig` in the code: + +.. code-block:: python + + import logging + from logging.config import fileConfig + + fileConfig('logging_config.txt') + logger = logging.getLogger() + logger.debug('often makes a very good meal of %s', 'visiting tourists') + + +.. + As of Python 2.7, you can use a dictionary with configuration details: + + .. code-block:: python + + import logging + from logging.config import dictConfig + + logging_config = dict( + version = 1, + formatters = { + 'f': {'format': + '%(asctime)s %(name)-12s %(levelname)-8s %(message)s'} + }, + handlers = { + 'h': {'class': 'logging.StreamHandler', + 'formatter': 'f', + 'level': logging.DEBUG} + }, + loggers = { + root : {'handlers': ['h'], + 'level': logging.DEBUG} + } + ) + + dictConfig(logging_config) + + logger = logging.getLogger() + logger.debug('often makes a very good meal of %s', 'visiting tourists') + + + Or instantiate the logger directly in code: + + .. code-block:: python + + import logging + + logger = logging.getLogger() + handler = logging.StreamHandler() + formatter = logging.Formatter( + '%(asctime)s %(name)-12s %(levelname)-8s %(message)s') + handler.setFormatter(formatter) + logger.addHandler(handler) + logger.setLevel(logging.DEBUG) + + logger.debug('often makes a very good meal of %s', 'visiting tourists') + + +.. _basic logging tutorial: http://docs.python.org/howto/logging.html#logging-basic-tutorial +.. _configuring logging for a library: https://docs.python.org/howto/logging.html#configuring-logging-for-a-library +.. _log record: https://docs.python.org/library/logging.html#logrecord-attributes +.. _requests source: https://github.com/kennethreitz/requests From 95ecb66c9631960624687f1c77d341d629113493 Mon Sep 17 00:00:00 2001 From: Tanya Schlusser Date: Sat, 17 Jan 2015 11:56:16 -0600 Subject: [PATCH 2/6] Added the logging section after the testing section in the Writing page --- docs/contents.rst.inc | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/contents.rst.inc b/docs/contents.rst.inc index cc385325c..b312ee3e6 100644 --- a/docs/contents.rst.inc +++ b/docs/contents.rst.inc @@ -32,6 +32,7 @@ This part of the guide focuses on best practices for writing Python code. writing/reading writing/documentation writing/tests + writing/logging writing/gotchas writing/license From 7c37ec8396b2487cc1dc38b86fa78f9d361163e5 Mon Sep 17 00:00:00 2001 From: Tanya Schlusser Date: Sat, 17 Jan 2015 20:59:53 -0600 Subject: [PATCH 3/6] Modified the new writing/logging.rst according to comments on the initial pull request --- docs/writing/logging.rst | 121 +++++++++++++++++++++------------------ 1 file changed, 66 insertions(+), 55 deletions(-) diff --git a/docs/writing/logging.rst b/docs/writing/logging.rst index e9e8bd169..e138ad995 100644 --- a/docs/writing/logging.rst +++ b/docs/writing/logging.rst @@ -3,8 +3,7 @@ Logging The :mod:`logging` module has been a part of Python's Standard Library since version 2.3. It is succinctly described in :pep:`282`. The documentation -is notoriously hard to read, except for the `basic logging tutorial`_, -and often less useful than simply reading the source code. +is notoriously hard to read, except for the `basic logging tutorial`_. Logging serves two purposes: @@ -16,8 +15,8 @@ Logging serves two purposes: reports or to optimize a business goal. -... or Print Statements? ------------------------- +... or Print? +------------- The only time that ``print`` is a better option than logging is when the goal is to display a help statement for a command line application. @@ -73,23 +72,28 @@ this in your ``__init__.py`` Logging in an Application ------------------------- -The `twelve factor app's `_, an authoritative reference +The `twelve factor app `_, an authoritative reference for good practice in application development, contains a section on `logging best practice `_. It emphatically advocates for treating log events as an event stream, and for sending that event stream to standard output to be handled by the -application environment. Do that. +application environment. There are at least three ways to configure a logger: -- using a file (recommended) +- using a file - using a dictionary - using code -Here is how with a file -- let us say it is named ``logging_config.txt``: +Example Configuration via an INI File +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. code-block:: none +Let us say the file is named ``logging_config.ini``. +More details for the file format are in the `logging configuration`_ +section of the `logging tutorial`_. + +.. code-block:: ini [loggers] keys=root @@ -126,55 +130,62 @@ Then use :meth:`logging.config.fileConfig` in the code: logger.debug('often makes a very good meal of %s', 'visiting tourists') -.. - As of Python 2.7, you can use a dictionary with configuration details: - - .. code-block:: python - - import logging - from logging.config import dictConfig - - logging_config = dict( - version = 1, - formatters = { - 'f': {'format': - '%(asctime)s %(name)-12s %(levelname)-8s %(message)s'} - }, - handlers = { - 'h': {'class': 'logging.StreamHandler', - 'formatter': 'f', - 'level': logging.DEBUG} - }, - loggers = { - root : {'handlers': ['h'], - 'level': logging.DEBUG} - } - ) - - dictConfig(logging_config) - - logger = logging.getLogger() - logger.debug('often makes a very good meal of %s', 'visiting tourists') - - - Or instantiate the logger directly in code: - - .. code-block:: python - - import logging - - logger = logging.getLogger() - handler = logging.StreamHandler() - formatter = logging.Formatter( - '%(asctime)s %(name)-12s %(levelname)-8s %(message)s') - handler.setFormatter(formatter) - logger.addHandler(handler) - logger.setLevel(logging.DEBUG) - - logger.debug('often makes a very good meal of %s', 'visiting tourists') +Example Configuration via a Dictionary +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As of Python 2.7, you can use a dictionary with configuration details. +:pep:`319` contains a list of the mandatory and optional elements in +the configuration dictionary. + +.. code-block:: python + + import logging + from logging.config import dictConfig + + logging_config = dict( + version = 1, + formatters = { + 'f': {'format': + '%(asctime)s %(name)-12s %(levelname)-8s %(message)s'} + }, + handlers = { + 'h': {'class': 'logging.StreamHandler', + 'formatter': 'f', + 'level': logging.DEBUG} + }, + loggers = { + root : {'handlers': ['h'], + 'level': logging.DEBUG} + } + ) + + dictConfig(logging_config) + + logger = logging.getLogger() + logger.debug('often makes a very good meal of %s', 'visiting tourists') + + +Example Configuration Directly in Code +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: python + + import logging + + logger = logging.getLogger() + handler = logging.StreamHandler() + formatter = logging.Formatter( + '%(asctime)s %(name)-12s %(levelname)-8s %(message)s') + handler.setFormatter(formatter) + logger.addHandler(handler) + logger.setLevel(logging.DEBUG) + + logger.debug('often makes a very good meal of %s', 'visiting tourists') .. _basic logging tutorial: http://docs.python.org/howto/logging.html#logging-basic-tutorial +.. _logging configuration: https://docs.python.org/howto/logging.html#configuring-logging +.. _logging tutorial: http://docs.python.org/howto/logging.html .. _configuring logging for a library: https://docs.python.org/howto/logging.html#configuring-logging-for-a-library .. _log record: https://docs.python.org/library/logging.html#logrecord-attributes .. _requests source: https://github.com/kennethreitz/requests From 1cfb637cb1babcc7f0ecb1d48b6f3bb4d093e3a5 Mon Sep 17 00:00:00 2001 From: Tanya Schlusser Date: Sat, 17 Jan 2015 21:05:56 -0600 Subject: [PATCH 4/6] Removed a stray capitalization and improved the grammar in the 'print' vs 'logging' section --- docs/writing/logging.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/writing/logging.rst b/docs/writing/logging.rst index e138ad995..7faa55363 100644 --- a/docs/writing/logging.rst +++ b/docs/writing/logging.rst @@ -28,8 +28,8 @@ Other reasons why logging is better than ``print``: - Events logged in included modules are automatically accessible via the root logger to your application's logging stream, unless you filter them out. -- Logging can be selectively silenced or disabled by using the method - :meth:`logging.Logger.setLevel` or setting the attribute +- Logging can be selectively silenced by using the method + :meth:`logging.Logger.setLevel` or disabled by setting the attribute :attr:`logging.Logger.disabled` to ``True``. @@ -37,8 +37,8 @@ Logging in a Library -------------------- Notes for `configuring logging for a library`_ are in the -`basic logging tutorial`_. Because the *user*, not the library, should -dictate what happens when a logging event occurs, One admonition bears +`logging tutorial`_. Because the *user*, not the library, should +dictate what happens when a logging event occurs, one admonition bears repeating: .. note:: @@ -86,6 +86,7 @@ There are at least three ways to configure a logger: - using a dictionary - using code + Example Configuration via an INI File ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 0c1551ae85940d9f75a79865356da02a72abad7b Mon Sep 17 00:00:00 2001 From: Tanya Schlusser Date: Sat, 17 Jan 2015 21:09:52 -0600 Subject: [PATCH 5/6] Added pro / con bullets to the list of ways to configure a logger --- docs/writing/logging.rst | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/docs/writing/logging.rst b/docs/writing/logging.rst index 7faa55363..0cee15d1c 100644 --- a/docs/writing/logging.rst +++ b/docs/writing/logging.rst @@ -82,9 +82,20 @@ application environment. There are at least three ways to configure a logger: -- using a file -- using a dictionary -- using code +- Using an INI-formatted file: + - *Pro* -- possible to update configuration while running + using the function :func:`logging.config.listen` to listen + on a socket. + - *Con* -- less control (*e.g.* custom subclassed filters or loggers) + than possible when configuring a logger in code. +- Using a dictionary or a JSON-formatted file: + - *Pro* -- in addition to updating while running, it is possible to + load from a file using the :mod:`json` module, in the standard + library since Python 2.6. + - *Con* -- less control than when configuring a logger in code. +- Using code: + - *Pro* -- complete control over the configuration. + - *Con* -- modifications require a change to source code. Example Configuration via an INI File From df7a22568bc4e944fd37a41ebf1f2740ff54976d Mon Sep 17 00:00:00 2001 From: Tanya Schlusser Date: Sat, 17 Jan 2015 21:43:22 -0600 Subject: [PATCH 6/6] Italic pro/con to bold, and '--' to colon after the pro/con --- docs/writing/logging.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/writing/logging.rst b/docs/writing/logging.rst index 0cee15d1c..70e20de54 100644 --- a/docs/writing/logging.rst +++ b/docs/writing/logging.rst @@ -83,19 +83,19 @@ application environment. There are at least three ways to configure a logger: - Using an INI-formatted file: - - *Pro* -- possible to update configuration while running + - **Pro**: possible to update configuration while running using the function :func:`logging.config.listen` to listen on a socket. - - *Con* -- less control (*e.g.* custom subclassed filters or loggers) + - **Con**: less control (*e.g.* custom subclassed filters or loggers) than possible when configuring a logger in code. - Using a dictionary or a JSON-formatted file: - - *Pro* -- in addition to updating while running, it is possible to + - **Pro**: in addition to updating while running, it is possible to load from a file using the :mod:`json` module, in the standard library since Python 2.6. - - *Con* -- less control than when configuring a logger in code. + - **Con**: less control than when configuring a logger in code. - Using code: - - *Pro* -- complete control over the configuration. - - *Con* -- modifications require a change to source code. + - **Pro**: complete control over the configuration. + - **Con**: modifications require a change to source code. Example Configuration via an INI File