Implementation of GLog by vlad-perevezentsev · Pull Request #718 · IntelPython/dpctl

vlad-perevezentsev · 2021-12-09T13:24:41Z

This PR contains the primary implementation of using GLog in dpctl.

The implementation used Glog 5.0
conda install glog

Added the --glog flag for setup in development mode
python setup.py develop --glog=True

Implemented 2 functions for initialization and deinitialization of GLog
void DPCTLService_InitLogger and void DPCTLService_ShutdownLogger

diptorupd · 2021-12-10T03:14:17Z

@vlad-perevezentsev @oleksandr-pavlyk

IMO glog usage in dpctl is a debugging feature intended for developers of dpctl or library writers who want to investigate some problem by turning on logging.

I propose the following design:

libsyclinterface provides two helper functions: dpctl_initialize_glog and dpctl_shutdown_glog. These functions should be in the main library and not the helper and be exposed to Python.
Change error_handler to behave one way if glog was initialized and another way if not. The function IsGoogleLoggingInitialized can be used for this purpose.
So, to use advanced logging a user needs to initialize glog and the shut it down explicitly.

Few unknowns:

I am not sure if glog can be shutdown and reinitialized.
Can all glog-specific options be set via command line?

vlad-perevezentsev · 2021-12-10T14:02:40Z

@diptorupd Glog has little information in the documentation so I can't answer using it.

I think Glog can be reinitiated after it is shutdown . But it doesn't continue writing to the created logfile. Glog will create a new logfile after initialization.
Parameters with FLAG_ prefix can be passed via the command line.

oleksandr-pavlyk · 2021-12-10T14:35:03Z

I think Glog can be reinitiated after it is shutdown . But it doesn't continue writing to the created logfile. Glog will create a new logfile after initialization.

Another important aspect is whether IsGoogleLoggingInitialized output is toggled with initialization/deinitialization. If it does, we can use it in error_handler function. User could then locally enable logging (in Python scripts with a context manager, in Google Tests with calling init/deinit functions).

coveralls · 2021-12-13T13:23:51Z

Coverage increased (+0.4%) to 81.629% when pulling 84e7dc5 on vlad-perevezentsev:use_glog into 05a5abd on IntelPython:master.

oleksandr-pavlyk · 2022-01-27T14:36:46Z

Some of the changes in this PR will be obsolete due to #746. But we should merge this once feedback is addressed, and I will update #746 after rebasing.

oleksandr-pavlyk · 2022-01-27T21:21:12Z

Note for posterity: glog=0.5 is required. The 0.4 does not declare google::EnableLogCleaner.

…or_handlers

Replaced use of `stat` function with use of `filesystem::is_directory`. Stat was always erroring out even for path to existing directory. Created `dpctl._dev` module for developers. It adds ```python dpctl._dev.init_logger(log_dir=None) ``` Function to initialize the logger. It uses `"dpctl"` for application name, and supplied log directory. `log_dir=None` is interpreted as use the current working directory. ```pyton dpctl._dev.fini_logger() ``` Call ShutdownLogger. ```python dpctl._dev.verbose(verbostiy="warning", log_dir=None) ``` Context manager which initializes the logger, and sets the verbosity level for a given block of Python code: Example: ``` In [1]: import dpctl, dpctl._dev as dd In [2]: with dd.verbose(): dpctl.SyclDevice().parent_device No parent for device because it is not a subdevice -33 (CL_INVALID_DEVICE) in DPCTLDevice_GetParentDevice at /localdisk/work/opavlyk/repos/dpctl/libsyclinterface/source/dpctl_sycl_device_interface.cpp:540 In [3]: dpctl.SyclDevice().parent_device # no message In [4]: with dd.verbose(): dpctl.SyclDevice().parent_device No parent for device because it is not a subdevice -33 (CL_INVALID_DEVICE) in DPCTLDevice_GetParentDevice at /localdisk/work/opavlyk/repos/dpctl/libsyclinterface/source/dpctl_sycl_device_interface.cpp:540 In [5]: dpctl.SyclDevice().parent_device # no message In [6]: quit ``` Export DPCTLService_InitLogger and DPCTLService_FiniLogger

1. GLOG defaults (log_dir=None) to outputing to color-enhanced std::cerr 2. Specifying valid log_dir changes the mode to logging 3. Context manager renamed from verbose to syclinterface_diagnostic Test updated.

vlad-perevezentsev requested review from diptorupd and oleksandr-pavlyk December 9, 2021 13:24

vlad-perevezentsev self-assigned this Dec 9, 2021

vlad-perevezentsev force-pushed the use_glog branch 2 times, most recently from ae328c7 to 11371db Compare December 9, 2021 14:51

diptorupd reviewed Dec 10, 2021

View reviewed changes

Comment thread libsyclinterface/helper/source/dpctl_error_handlers.cpp Outdated

diptorupd reviewed Dec 10, 2021

View reviewed changes

Comment thread libsyclinterface/helper/source/dpctl_error_handlers.cpp Outdated

oleksandr-pavlyk reviewed Dec 13, 2021

View reviewed changes

Comment thread libsyclinterface/CMakeLists.txt

oleksandr-pavlyk reviewed Dec 16, 2021

View reviewed changes

Comment thread libsyclinterface/source/dpctl_service.cpp

oleksandr-pavlyk reviewed Dec 16, 2021

View reviewed changes

Comment thread libsyclinterface/helper/source/dpctl_error_handlers.cpp Outdated

vlad-perevezentsev force-pushed the use_glog branch from 0ebf25e to 420d8c5 Compare December 17, 2021 23:34

vlad-perevezentsev marked this pull request as ready for review December 17, 2021 23:50

vlad-perevezentsev force-pushed the use_glog branch 2 times, most recently from 8daf2e3 to a1ced1a Compare January 26, 2022 12:00

oleksandr-pavlyk reviewed Jan 27, 2022

View reviewed changes

Comment thread libsyclinterface/CMakeLists.txt Outdated

oleksandr-pavlyk reviewed Jan 27, 2022

View reviewed changes

Comment thread libsyclinterface/helper/source/dpctl_error_handlers.cpp Outdated

oleksandr-pavlyk force-pushed the use_glog branch from a1ced1a to 6a4d00f Compare January 27, 2022 21:18

oleksandr-pavlyk previously approved these changes Jan 27, 2022

View reviewed changes

oleksandr-pavlyk requested a review from diptorupd January 27, 2022 21:54