Python code formats automatically use isort, black, flake8, and pre-commit

Tram Ho

TL; DR

Before implementing Git commit , I will use isort and black to format the code automatically, then use flake8 to check again with standard PEP8 (all configured by pre-commit ). The commit will be successful if there is no error. If an error occurs, we will go back and fix where necessary and commit again. This workflow helps to reduce the time to reformat the code manually, so we can focus more on the logic. Team working together is also easier and more efficient.

Code Convention: Do you spend too much effort on it?

PEP8

Code convention is a set of standard programming rules. It includes variable naming conventions, function names; how to indent, how to comment; how to declare; or suggestions for coding and organizing code, … Every language has its own code convention. With Python, PEP8 (Python Enhancement Proposals) is one of the most widely used normative sets. You can learn about the rules here (One thing that surprised me is that it has been in 2001 but almost 20 years later I learned about it).

Why must code correctly?

In my opinion, the greatest value of code properly is to ensure consistency in the code . This makes your code clear, easy to read, and understandable for yourself and others. Imagine if you have:

  • THIS HAS A TEXT THAT IT WAS FORMATED AS THIS HAT?
  • Or a custom text # this time?! ??

(Even though you almost 100% understand it?

That’s not to mention nested loops with all kinds of spacing and descent; meaningless variable names like a , b , c , … appear dense.

After all, writing code is essentially the same as writing writing. Finishing up your words is an issue that deserves attention.

Struggling with things that don’t really matter

Ensuring the right code when working in groups is a lot harder than you might think. What if everyone were to code in a different style? Okay, maybe you set your team up with PEP8 standards from the start. However, sometimes when reviewing code, you will come across comments like:

  • “Add a newline on Line 23 for one.”
  • “The operator symbol precedes the variable name instead of the end of the line before it.”
  • “The line of code is too long, only be limited to xy characters.” (This is convenient for merging code. When two windows are opened side by side, we can see all source code of the two versions without dragging horizontal pointer bar)
  • “Add a space after the colon on Line 47.”
  • “Removing the extra space after the # on Line 117 for me.”
  • “Import statement must be on top of file.”

(Oh how petty I look when I keep walking other people’s backs like that all day long!

The above issues may be known to them, but not everyone will remember and notice all – all – the – the – first – line in the standard rule set. And so, the team members spend time and effort reviewing, feedback, fixing code, updating with upstream, merging conflict if any, re-committing, then reviewing for the second time … sometimes with just small things. so.

Focus more on what’s important with an auto format tool

PEP8 is just a style guide, not a tool or library. The setup of a PEP8 automatic format tool is a very important matter, as it will help you:

  • No need to memorize lengthy rules or waste time on manual review. This tool will take care of most of those little problems for you: it shows you where to fix it and can format the source code automatically.
  • Reduce the time to discuss small things, so you spend more time focusing on more important things like writing processing logic or optimizing performance.
  • Minimize merge conflicts because the code convention is now consistent between members from newbie to expert.

The necessary packages

What “ingredients” do we need to get an auto tool like that?

isort

This package will sort all import expressions in Python file alphabetically and automatically split into different types of imports. Install just run pip install isort .

Consider the following example (taken from the homepage):

Results after running the command isort myPythonFile.py :

Black

Black is a standard code format tool of PEP8. It comes with a few other style options that the development team believes will be helpful in creating easy to read and maintainable lines of code:

  • The number of characters in a line is 88 instead of 79 like PEP8.
  • Use double quotes instead of single quotes when working with strings.
  • If a function has multiple parameters, each parameter will be on one line.

Of course we can easily tweak the default configurations using the pyproject.toml file. Install by running the command line: pip install black

Consider the following example:

If you are proficient at PEP8, you can immediately find that there are at least 19 places that need fixing  With only 9 lines of code, the reviewer must feedback at least 19 style issues. Even though we are correct, we are not happy to hear it.

black myAnotherPythonFile.py run black myAnotherPythonFile.py and see the result:

Elephant-is! All bugs have been fixed in just one note.

flake8

flake8 is a powerful tool to check if your code is actually compliant with the PEP8 standard. In order for black work well with flake8 (preventing it from generating various errors and warnings), we need to list some error codes to ignore (I will specify in the configuration section below).

Run the command pip install flake8 to install. Then test with the above example (not formatted with black ) with the command flake8 myAnotherPythonFile.py . Immediately, below the console will display the typing errors that you are having:

pre-commit

pre-commit is a framework for managing pre-commit hooks in Git. Oh, but what is Git Hook ?

Git Hook is script that is run automatically every time a specific event occurs in the Git repository.

In this case, the event here is the commit code. We will use the pre-commit hook to check the changes in the code of each style convention automatically before commit and integrated into the system. If something goes wrong, the commit will fail and we’ll get the associated error messages to fix. Commit is only successful when no errors occur.


So far we have just introduced and run the above tools manually. Now it’s time to combine them to run automatically!

Workflow with pre-commit hooks

overview

Workflow with pre-commit (Image taken from the end of post link)

Before implementing Git commit , I will use isort and black to format the code automatically, then use flake8 to check again with standard PEP8 (all configured by pre-commit ). The commit will be successful if there is no error. If an error occurs, we will go back and fix where necessary and commit again. This workflow helps to reduce the time to reformat the code manually, thereby focusing more on the logic. Team working together is also easier and more efficient.

Set-up step by step

(These are the configuration files that I am using. You can customize them to suit your own style or needs!)

  • Create virtualvenv if needed and install pre-commit :pip3 install pre-commit

    Don’t forget to add it to requirements.txt or Pipfile after installation.

  • Configure pre-commit by creating a .pre-commit-config.yaml in the root directory with the following content (you can adjust to the latest version at the time of reading this article):

  • Configure isort by creating a .isort.cfg file in the root directory with the following content:

If you have noticed in the pre-commit configuration file, along with isort , we use add seed-isort-config to automatically add packages to known_third_party in the isort configuration (instead of doing it manually).

  • Configure black by creating a pyproject.toml file in the root directory with the following content:

  • Configure flake8 by creating .flake8 file with the following content:

  • After configuration is complete, run the following command to complete the installation:pre-commit install
  • Finally, before executing Git commit , run the following command:pre-commit run --all-files

As you can see, installing the above workflow is very easy for the team because all the configuration files are already in the project, members just need to run the pre-commit install command.

Regarding the issue of CI / CD

For even more assurance the code convention and style checks should also be integrated into the CI (Continuous Integration) pipeline. Remember that there is still a case of someone on the team skipping the pre-commit with the —no-verify flag when committing. However, I have not implemented this setup. I will update with you in another post!

Some alternative methods

If the method above is quite complicated and not necessary, you can manually run isort , blake , flake8 or use other packages like autopep8 or yapf . In addition, today’s IDEs and Text Editor support automatic reformat code. If you use Pycharm, you can try Code > Reformat see why!

Epilogue

The advantages of the above workflow have also been mentioned several times in this article. You can consider using it according to the installation steps I introduced above. But, really, code style is just a recommendation. You can do whatever you want with the codebase, as long as it’s consistent, especially when working with others.

See you in the following articles! Happy coding!

Original post

https://nguyendhn.wordpress.com/2020/09/28/format-code-python-tu-dong-su-dung-isort-black-flake8-va-pre-commit/

Reference

https://ljvmiranda921.github.io/notebook/2018/06/21/precommits-using-black-and-flake8/

https://medium.com/staqu-dev-logs/keeping-python-code-clean-with-pre-commit-hooks-black-flake8-and-isort-cac8b01e0ea1

Share the news now

Source : Viblo