-
Notifications
You must be signed in to change notification settings - Fork 9
Expand file tree
/
Copy pathmodel-builders.html
More file actions
273 lines (252 loc) · 18.3 KB
/
model-builders.html
File metadata and controls
273 lines (252 loc) · 18.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
<!DOCTYPE html>
<html class="writer-html5" lang="en">
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Model Builders for the Gradient Boosting Frameworks — daal4py 2021.1 documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="_static/css/theme.css" />
<link rel="stylesheet" type="text/css" href="_static/style.css" />
<!--[if lt IE 9]>
<script src="_static/js/html5shiv.min.js"></script>
<![endif]-->
<script src="_static/jquery.js"></script>
<script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/sphinx_highlight.js"></script>
<script src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Algorithms" href="algorithms.html" />
<link rel="prev" title="Input Data" href="data.html" />
<script defer type="text/javascript" src="https://www.intel.com/content/dam/www/global/wap/performance-config.js" ></script>
<script type="text/javascript">
// Configure TMS settings
var wapLocalCode = 'us-en'; // Dynamically set per localized site, see mapping table for values
var wapSection = "scikit-learn"; // WAP team will give you a unique section for your site
// Load TMS
if(document.location.href.contains("intel.github.io/scikit-learn-intelex")){
(function () {
var url = 'https://www.intel.com/content/dam/www/global/wap/tms-loader.js'; // WAP file URL
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = url;
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
}
</script>
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="contents.html" class="icon icon-home">
daal4py
</a>
<div class="version">
2021
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="index.html">About daal4py</a></li>
<li class="toctree-l1"><a class="reference internal" href="data.html">Data</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Model Builders</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#introduction">Introduction</a></li>
<li class="toctree-l2"><a class="reference internal" href="#conversion">Conversion</a></li>
<li class="toctree-l2"><a class="reference internal" href="#classification-and-regression-inference">Classification and Regression Inference</a></li>
<li class="toctree-l2"><a class="reference internal" href="#shap-value-calculation-for-regression-models">SHAP Value Calculation for Regression Models</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#scikit-learn-style-estimators">Scikit-learn-style Estimators</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#limitations">Limitations</a></li>
<li class="toctree-l2"><a class="reference internal" href="#examples">Examples</a></li>
<li class="toctree-l2"><a class="reference internal" href="#articles-and-blog-posts">Articles and Blog Posts</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="algorithms.html">Supported Algorithms</a></li>
<li class="toctree-l1"><a class="reference internal" href="scaling.html">Distributed Mode</a></li>
<li class="toctree-l1"><a class="reference internal" href="streaming.html">Streaming Mode</a></li>
<li class="toctree-l1"><a class="reference internal" href="examples.html">Examples</a></li>
<li class="toctree-l1"><a class="reference internal" href="sklearn.html">Scikit-Learn API</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="contents.html">daal4py</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="contents.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item active">Model Builders for the Gradient Boosting Frameworks</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/model-builders.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="model-builders-for-the-gradient-boosting-frameworks">
<span id="model-builders"></span><h1>Model Builders for the Gradient Boosting Frameworks<a class="headerlink" href="#model-builders-for-the-gradient-boosting-frameworks" title="Permalink to this heading"></a></h1>
<div class="admonition note" id="note">
<p class="admonition-title">Note</p>
<p>Scikit-learn patching functionality in daal4py was deprecated and moved to a separate package, <a class="reference external" href="https://github.com/intel/scikit-learn-intelex">Intel(R) Extension for Scikit-learn*</a>.
All future patches will be available only in Intel(R) Extension for Scikit-learn*. Use the scikit-learn-intelex package instead of daal4py for the scikit-learn acceleration.</p>
</div>
<section id="introduction">
<h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to this heading"></a></h2>
<p>Gradient boosting on decision trees is one of the most accurate and efficient
machine learning algorithms for classification and regression.
The most popular implementations of it are:</p>
<ul class="simple">
<li><p>XGBoost*</p></li>
<li><p>LightGBM*</p></li>
<li><p>CatBoost*</p></li>
</ul>
<p>daal4py Model Builders deliver the accelerated
models inference of those frameworks. The inference is performed by the oneDAL GBT implementation tuned
for the best performance on the Intel(R) Architecture.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Currently, experimental support for XGBoost* and LightGBM* categorical data is not supported.
For the model conversion to work with daal4py, convert non-numeric data to numeric data
before training and converting the model.</p>
</div>
</section>
<section id="conversion">
<h2>Conversion<a class="headerlink" href="#conversion" title="Permalink to this heading"></a></h2>
<p>The first step is to convert already trained model. The
API usage for different frameworks is the same:</p>
<p>XGBoost:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">daal4py</span> <span class="k">as</span> <span class="nn">d4p</span>
<span class="n">d4p_model</span> <span class="o">=</span> <span class="n">d4p</span><span class="o">.</span><span class="n">mb</span><span class="o">.</span><span class="n">convert_model</span><span class="p">(</span><span class="n">xgb_model</span><span class="p">)</span>
</pre></div>
</div>
<p>LightGBM:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">daal4py</span> <span class="k">as</span> <span class="nn">d4p</span>
<span class="n">d4p_model</span> <span class="o">=</span> <span class="n">d4p</span><span class="o">.</span><span class="n">mb</span><span class="o">.</span><span class="n">convert_model</span><span class="p">(</span><span class="n">lgb_model</span><span class="p">)</span>
</pre></div>
</div>
<p>CatBoost:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">daal4py</span> <span class="k">as</span> <span class="nn">d4p</span>
<span class="n">d4p_model</span> <span class="o">=</span> <span class="n">d4p</span><span class="o">.</span><span class="n">mb</span><span class="o">.</span><span class="n">convert_model</span><span class="p">(</span><span class="n">cb_model</span><span class="p">)</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Convert model only once and then use it for the inference.</p>
</div>
</section>
<section id="classification-and-regression-inference">
<h2>Classification and Regression Inference<a class="headerlink" href="#classification-and-regression-inference" title="Permalink to this heading"></a></h2>
<p>The API is the same for classification and regression inference.
Based on the original model passed to the <code class="docutils literal notranslate"><span class="pre">convert_model()</span></code>, <code class="docutils literal notranslate"><span class="pre">d4p_prediction</span></code> is either the classification or regression output.</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">d4p_prediction</span> <span class="o">=</span> <span class="n">d4p_model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">test_data</span><span class="p">)</span>
</pre></div>
</div>
</div></blockquote>
<p>Here, the <code class="docutils literal notranslate"><span class="pre">predict()</span></code> method of <code class="docutils literal notranslate"><span class="pre">d4p_model</span></code> is being used to make predictions on the <code class="docutils literal notranslate"><span class="pre">test_data</span></code> dataset.
The <code class="docutils literal notranslate"><span class="pre">d4p_prediction</span></code> variable stores the predictions made by the <code class="docutils literal notranslate"><span class="pre">predict()</span></code> method.</p>
</section>
<section id="shap-value-calculation-for-regression-models">
<h2>SHAP Value Calculation for Regression Models<a class="headerlink" href="#shap-value-calculation-for-regression-models" title="Permalink to this heading"></a></h2>
<p>SHAP contribution and interaction value calculation are natively supported by models created with daal4py Model Builders.
For these models, the <code class="docutils literal notranslate"><span class="pre">predict()</span></code> method takes additional keyword arguments:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">d4p_model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">test_data</span><span class="p">,</span> <span class="n">pred_contribs</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="c1"># for SHAP contributions</span>
<span class="n">d4p_model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">test_data</span><span class="p">,</span> <span class="n">pred_interactions</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="c1"># for SHAP interactions</span>
</pre></div>
</div>
</div></blockquote>
<p>The returned prediction has the shape:</p>
<blockquote>
<div><ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">(n_rows,</span> <span class="pre">n_features</span> <span class="pre">+</span> <span class="pre">1)</span></code> for SHAP contributions</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">(n_rows,</span> <span class="pre">n_features</span> <span class="pre">+</span> <span class="pre">1,</span> <span class="pre">n_features</span> <span class="pre">+</span> <span class="pre">1)</span></code> for SHAP interactions</p></li>
</ul>
</div></blockquote>
<p>Here, <code class="docutils literal notranslate"><span class="pre">n_rows</span></code> is the number of rows (i.e., observations) in
<code class="docutils literal notranslate"><span class="pre">test_data</span></code>, and <code class="docutils literal notranslate"><span class="pre">n_features</span></code> is the number of features in the dataset.</p>
<p>The prediction result for SHAP contributions includes a feature attribution value for each feature and a bias term for each observation.</p>
<p>The prediction result for SHAP interactions comprises <code class="docutils literal notranslate"><span class="pre">(n_features</span> <span class="pre">+</span> <span class="pre">1)</span> <span class="pre">x</span> <span class="pre">(n_features</span> <span class="pre">+</span> <span class="pre">1)</span></code> values for all possible
feature combinations, along with their corresponding bias terms.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The shapes of SHAP contributions and interactions are consistent with the XGBoost results.
In contrast, the <a class="reference external" href="https://shap.readthedocs.io/en/latest/">SHAP Python package</a> drops bias terms, resulting
in SHAP contributions (SHAP interactions) with one fewer column (one fewer column and row) per observation.</p>
</div>
<section id="scikit-learn-style-estimators">
<h3>Scikit-learn-style Estimators<a class="headerlink" href="#scikit-learn-style-estimators" title="Permalink to this heading"></a></h3>
<p>You can also use the scikit-learn-style classes <code class="docutils literal notranslate"><span class="pre">GBTDAALClassifier</span></code> and <code class="docutils literal notranslate"><span class="pre">GBTDAALRegressor</span></code> to convert and infer your models. For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">daal4py.sklearn.ensemble</span> <span class="kn">import</span> <span class="n">GBTDAALRegressor</span>
<span class="n">reg</span> <span class="o">=</span> <span class="n">xgb</span><span class="o">.</span><span class="n">XGBRegressor</span><span class="p">()</span>
<span class="n">reg</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">d4p_predt</span> <span class="o">=</span> <span class="n">GBTDAALRegressor</span><span class="o">.</span><span class="n">convert_model</span><span class="p">(</span><span class="n">reg</span><span class="p">)</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X</span><span class="p">)</span>
</pre></div>
</div>
</section>
</section>
<section id="limitations">
<h2>Limitations<a class="headerlink" href="#limitations" title="Permalink to this heading"></a></h2>
<p>Model Builders support only base inference with prediction and probabilities prediction. The functionality is to be extended.
Therefore, there are the following limitations:
- The categorical features are not supported for conversion and prediction.
- The multioutput models are not supported for conversion and prediction.
- SHAP values can be calculated for regression models only.</p>
</section>
<section id="examples">
<h2>Examples<a class="headerlink" href="#examples" title="Permalink to this heading"></a></h2>
<p>Model Builders models conversion</p>
<ul class="simple">
<li><p><a class="reference external" href="https://github.com/intel/scikit-learn-intelex/blob/main/examples/daal4py/model_builders_xgboost.py">XGBoost model conversion</a></p></li>
<li><p><a class="reference external" href="https://github.com/intel/scikit-learn-intelex/blob/main/examples/daal4py/model_builders_xgboost_shap.py">SHAP value prediction from an XGBoost model</a></p></li>
<li><p><a class="reference external" href="https://github.com/intel/scikit-learn-intelex/blob/main/examples/daal4py/model_builders_lightgbm.py">LightGBM model conversion</a></p></li>
<li><p><a class="reference external" href="https://github.com/intel/scikit-learn-intelex/blob/main/examples/daal4py/model_builders_catboost.py">CatBoost model conversion</a></p></li>
</ul>
</section>
<section id="articles-and-blog-posts">
<h2>Articles and Blog Posts<a class="headerlink" href="#articles-and-blog-posts" title="Permalink to this heading"></a></h2>
<ul class="simple">
<li><p><a class="reference external" href="https://medium.com/intel-analytics-software/improving-the-performance-of-xgboost-and-lightgbm-inference-3b542c03447e">Improving the Performance of XGBoost and LightGBM Inference</a></p></li>
</ul>
</section>
</section>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="data.html" class="btn btn-neutral float-left" title="Input Data" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="algorithms.html" class="btn btn-neutral float-right" title="Algorithms" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright Intel.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>