How to use Log like a Pro Vip in Python

Tram Ho

Happy New Year

Happy New Year, wishing everyone a year filled with joy, happiness and achieving everything they wish for.


This article is only for the purpose of translating and rewriting according to my own understanding, so if there is anything missing, please give me more suggestions.

If you’ve ever coded Python, chances are high that you’ve had to use quite a few print statements to see how your code runs, is it correct and ok or not, or to find out where the error is. But using print so much is extremely overwhelming so today I will show you how you can start using Log as an aka provip in python.

Logging is a very, very important thing in software development. It helps developers better understand program execution, unexpected errors and their reasons. Logging can store information such as the current state of the program or where the program is running. If an error occurs, developers can quickly find the line of code that is causing the problem and fix it. Finding these can be made easier using Python’s built-in logging tools.

Log Levels

It is a side note that the loggers as well as the underlying processing layers all use a so-called logging levels. This is a log hierarchy of how things are handled.

For example: When running the project locally when you use DEBUG level, it will receive debug commands and all levels on it. These levels are used to filter data by its importance. For example, when selecting the DEBUG log level, the system will log everything related to DEBUG , INFO , WARNING , ERROR , CRITICAL .



Logger is simply a named instance of the logging object. All loggers are “global” in the program. Eg.

The above code defined a logger named app in .

Now if I want, I can import into another file and have access to the same logger instance I created earlier. It will look like this:

Now has access to the same logger instance . This means I can declare all the loggers I want before I need them.

Note: the best practice is to use __name__ to name the logger like this: logging.getLogger(__name__) . This way, each module has its own logger name.

Logger ‘s core job is to send LogRecords to different handlers. LogRecords records details of what happened in the system at runtime.

The above is an example of what it means to fire an event. Events are fired with an associated level . This allows to know the urgency of the LogRecord (Log Level). The problem with the above code is that the logger is completely unaware of the destination of these logs such as console or file.


Handlers control where LogRecords return. Handlers are independent objects that can be attached to logger instances.

The above code tells the logger that, whenever a LogRecord is created, it will be sent to the file logging.log . The problem right now is that logging.log will be crammed with a lot of logs.

Setting the log level that Handlers have to manage helps me filter my logs by level.

Above is how to filter the log using log level . As in the log levels , ERROR is higher than WARNING so it will be processed and INFO is lower than WARNING so it will be discarded.

There are several types of handlers that can be used for example StreamHandler, FileHandler . For a complete list, see the handlers document in python: handlers

StreamHandler will send output to console. FileHandler sends information to an output file. The most dynamic handlers are QueueHandler and QueueListener , but we’ll talk about that later.


Assuming you only have a message saying: “An error has occurred”, this message is almost meaningless. So formatters were born to solve this problem. Formatters exist on Handlers when Handlers process log records.

You can easily add some additional information such as where, what, why, when and how.

The above is the output in the generated console.

Formatters in the above example have been inserted with the time, logger name, record level and message.

QueueHandler and QueueListener

The problem is that whenever the program runs to the code“message”) , your program will be blocked until it finishes processing the logging. It won’t be a big deal if you just output the log to the console but when it’s running fine and the server crashes, the log will sing “yes, don’t keep it, don’t look”.

If you send the log to the database, waiting for the record to be created may take some time, your program will be blocked every time the log record is created. QueueHandler and QueueListener will solve this problem. QueueHandler places the message in a queue that exists on a separate Thread . Then the QueueListener sends the Log Record to all other handler . This way your program is not blocked waiting for the Log Record to be created. To learn more about using QueueHandler and QueueListener, you can see more at this link

Using File to initialize loggers

The last thing to do is create a File to launch all the loggers. This will help avoid code duplication, easy management and changes, saving time and effort. You can completely create and manage all loggers in a single place. There are many ways to store the loggers configuration for the project such as using dictionary , JSON or yaml file to initialize the loggers. Here is an example with yaml file:

In the end you just need to write a file to read the config from a yaml file, dictionary or JSON to initialize loggers and then just import and run it once in the project. Examples are as follows:

People can refer to the sample code on using logging with python at the following link: GITHUB

tags: Basic Python, Python, Programming, Logging, Loggers, FileHandlers, Handlers, Formatters, QueueHandler , QueueListener, Log Levels

Source: Mediummmm

Share the news now

Source : Viblo