Profilers help to identify performance problems. These are tools designed to give the metrics to find the slowest parts of the code so that we can optimize what really matters. Profilers can gather a wide variety of metrics: wall time, CPU time, network or memory consumption, I/O operations, etc.
Profilers can answer questions like,

• How many times is each method in my code called?
• How long does each of these methods take?
• How much memory does the method consume?

• Deterministic Profiling: Deterministic profilers execute trace functions at various points of interest (function call, function return) and record precise timings of these events. It means the code runs slower under profiling. Its use in production systems is often impractical.

• Statistical profiling:  Instead of tracking every event (call to every function), statistical profilers interrupt applications periodically and collect samples of the execution state (call stack snapshots). The call stacks are then analyzed to determine the execution time of different parts of the application. This method is less accurate, but it also reduces the overhead.

We will begin profiling with a Starter Kit:

• Scikit-Learn Intelligent Indexing for Incoming Correspondence – Ref Kit

Follow the steps mentioned in the github readme to setup the environment.
The process involves

• Setting up a virtual environment for stock/Intel®-accelerated ML
• Preprocessing data using Pandas/Intel® oneAPI Modin and NLTK
• Training an NLP model for text classification using scikit-learn/Intel® oneAPI scikit-learn extension
• Predicting from the trained model on new data using scikit-learn/Intel® oneAPI scikit-learn extension.

Here we'll discuss profiling Tools Available for Python

• Vtune (both line and function profiler)
• Cprofile (function profiler)
• Profile (function profiler)
• line_profiler (line profiler)
• memory_profiler (line)
• Scalene (both line and function profiler)
• Function-Trace (function profiler)