Adding a Python sleep()
Call With Decorators
There are times when you need to retry a faulty function. A common case is when downloading files when the server is busy. You often don’t want to send requests to the server too often, so adding a sleep()
call between requests is essential.
Another situation I’ve experienced myself is when I need to check the status of a UI during an automated test session. The UI may load faster or slower than usual, depending on the computer you are running the test on. This can change what appears on the screen at the time the program authenticates something.
In this case, I can let the program sleep for a while and check everything awake a second or two later. Successful or unsuccessful tests run in these sleepy moments.
You can use decorator to add sleep()
calls in these cases. If you are unfamiliar with decorators or you want to improve your knowledge, see the Primer on Python Decorators tutorial. See the following example:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | <span class="token keyword">import</span> time <span class="token keyword">import</span> urllib <span class="token punctuation">.</span> request <span class="token keyword">import</span> urllib <span class="token punctuation">.</span> error <span class="token keyword">def</span> <span class="token function">sleep</span> <span class="token punctuation">(</span> timeout <span class="token punctuation">,</span> retry <span class="token operator">=</span> <span class="token number">3</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span> <span class="token keyword">def</span> <span class="token function">the_real_decorator</span> <span class="token punctuation">(</span> function <span class="token punctuation">)</span> <span class="token punctuation">:</span> <span class="token keyword">def</span> <span class="token function">wrapper</span> <span class="token punctuation">(</span> <span class="token operator">*</span> args <span class="token punctuation">,</span> <span class="token operator">**</span> kwargs <span class="token punctuation">)</span> <span class="token punctuation">:</span> retries <span class="token operator">=</span> <span class="token number">0</span> <span class="token keyword">while</span> retries <span class="token operator"><</span> retry <span class="token punctuation">:</span> <span class="token keyword">try</span> <span class="token punctuation">:</span> value <span class="token operator">=</span> function <span class="token punctuation">(</span> <span class="token operator">*</span> args <span class="token punctuation">,</span> <span class="token operator">**</span> kwargs <span class="token punctuation">)</span> <span class="token keyword">if</span> value <span class="token keyword">is</span> <span class="token boolean">None</span> <span class="token punctuation">:</span> <span class="token keyword">return</span> <span class="token keyword">except</span> <span class="token punctuation">:</span> <span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string-interpolation"><span class="token string">f'Sleeping for </span> <span class="token interpolation"><span class="token punctuation">{</span> timeout <span class="token punctuation">}</span></span> <span class="token string"> seconds'</span></span> <span class="token punctuation">)</span> time <span class="token punctuation">.</span> sleep <span class="token punctuation">(</span> timeout <span class="token punctuation">)</span> retries <span class="token operator">+=</span> <span class="token number">1</span> <span class="token keyword">return</span> wrapper <span class="token keyword">return</span> the_real_decorator |
sleep()
is your decorator. It takes a timeout
value and the number of times it should retry
– the default is 3. In sleep()
is another function, the_real_decorator()
, which accepts the decorated function.
Finally, the function in the same wrapper()
takes the arguments and keyword arguments that you pass into the decorated function. This is where magic appears! You use the while
loop to retry the function call. If there are exceptions, call time.sleep()
, increase the count of retries
and run the function again.
Now, rewrite uptime_bot()
to use your new decorator:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | <span class="token decorator annotation punctuation">@sleep</span> <span class="token punctuation">(</span> <span class="token number">3</span> <span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">uptime_bot</span> <span class="token punctuation">(</span> url <span class="token punctuation">)</span> <span class="token punctuation">:</span> <span class="token keyword">try</span> <span class="token punctuation">:</span> conn <span class="token operator">=</span> urllib <span class="token punctuation">.</span> request <span class="token punctuation">.</span> urlopen <span class="token punctuation">(</span> url <span class="token punctuation">)</span> <span class="token keyword">except</span> urllib <span class="token punctuation">.</span> error <span class="token punctuation">.</span> HTTPError <span class="token keyword">as</span> e <span class="token punctuation">:</span> <span class="token comment"># Email admin / log</span> <span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string-interpolation"><span class="token string">f'HTTPError: </span> <span class="token interpolation"><span class="token punctuation">{</span> e <span class="token punctuation">.</span> code <span class="token punctuation">}</span></span> <span class="token string"> for </span> <span class="token interpolation"><span class="token punctuation">{</span> url <span class="token punctuation">}</span></span> <span class="token string">'</span></span> <span class="token punctuation">)</span> <span class="token comment"># Re-raise the exception for the decorator</span> <span class="token keyword">raise</span> urllib <span class="token punctuation">.</span> error <span class="token punctuation">.</span> HTTPError <span class="token keyword">except</span> urllib <span class="token punctuation">.</span> error <span class="token punctuation">.</span> URLError <span class="token keyword">as</span> e <span class="token punctuation">:</span> <span class="token comment"># Email admin / log</span> <span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string-interpolation"><span class="token string">f'URLError: </span> <span class="token interpolation"><span class="token punctuation">{</span> e <span class="token punctuation">.</span> code <span class="token punctuation">}</span></span> <span class="token string"> for </span> <span class="token interpolation"><span class="token punctuation">{</span> url <span class="token punctuation">}</span></span> <span class="token string">'</span></span> <span class="token punctuation">)</span> <span class="token comment"># Re-raise the exception for the decorator</span> <span class="token keyword">raise</span> urllib <span class="token punctuation">.</span> error <span class="token punctuation">.</span> URLError <span class="token keyword">else</span> <span class="token punctuation">:</span> <span class="token comment"># Website is up</span> <span class="token keyword">print</span> <span class="token punctuation">(</span> <span class="token string-interpolation"><span class="token string">f'</span> <span class="token interpolation"><span class="token punctuation">{</span> url <span class="token punctuation">}</span></span> <span class="token string"> is up'</span></span> <span class="token punctuation">)</span> <span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">'__main__'</span> <span class="token punctuation">:</span> url <span class="token operator">=</span> <span class="token string">'http://www.google.com/py'</span> uptime_bot <span class="token punctuation">(</span> url <span class="token punctuation">)</span> |
Here, you decorate uptime_bot()
with the command sleep()
3 seconds. You have also removed the initial while
loop as well as a call to sleep(60)
. Decorator will take care of this from now on.
Another change you’ve made is to add a raise
statement within the exception handler blocks. This ensures the decorator will work properly. You can write decorators to handle these errors, but because exceptions only apply to urllib
, it’s better to keep the decorator as before. As such, you can use it in many places.
There are a few things you can do to improve your decorator. If the program has tried it enough times and it still fails, you will see it raise the last error. Decorator will also wait 3 seconds after the last failure and this may be something you do not want. Feel free to change it to your liking!
Adding a Python sleep()
Call With Threads
There are times when you want to add a sleep()
call to a thread . Maybe you’re running a migration script with millions of records in production. You don’t want to cause downtime but don’t want to wait longer than necessary to complete the migration, so you decide to use threads.
Note: Thread is a way of implementing concurrency in Python. You can run several threads at the same time to increase program performance. If you are new to the thread, you can check out the article An Intro to Threading in Python .
To prevent customers from paying attention to any delays, each thread needs to run for a short time, then go to bed. There are two ways to do this:
- Use
time.sleep()
as above - Use
Event.wait()
from thethreading
module
Please start from time.sleep()
.
Using time.sleep()
Logging Cookbook gave a great example using time.sleep()
. The logging
module is thread-safe, so it’s a little more useful than print () for this exercise. The following code is based on this example:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | <span class="token keyword">import</span> logging <span class="token keyword">import</span> threading <span class="token keyword">import</span> time <span class="token keyword">def</span> <span class="token function">worker</span> <span class="token punctuation">(</span> arg <span class="token punctuation">)</span> <span class="token punctuation">:</span> <span class="token keyword">while</span> <span class="token keyword">not</span> arg <span class="token punctuation">[</span> <span class="token string">"stop"</span> <span class="token punctuation">]</span> <span class="token punctuation">:</span> logging <span class="token punctuation">.</span> debug <span class="token punctuation">(</span> <span class="token string">"worker thread checking in"</span> <span class="token punctuation">)</span> time <span class="token punctuation">.</span> sleep <span class="token punctuation">(</span> <span class="token number">1</span> <span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">main</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span> logging <span class="token punctuation">.</span> basicConfig <span class="token punctuation">(</span> level <span class="token operator">=</span> logging <span class="token punctuation">.</span> DEBUG <span class="token punctuation">,</span> <span class="token builtin">format</span> <span class="token operator">=</span> <span class="token string">"%(relativeCreated)6d %(threadName)s %(message)s"</span> <span class="token punctuation">)</span> info <span class="token operator">=</span> <span class="token punctuation">{</span> <span class="token string">"stop"</span> <span class="token punctuation">:</span> <span class="token boolean">False</span> <span class="token punctuation">}</span> thread <span class="token operator">=</span> threading <span class="token punctuation">.</span> Thread <span class="token punctuation">(</span> target <span class="token operator">=</span> worker <span class="token punctuation">,</span> args <span class="token operator">=</span> <span class="token punctuation">(</span> info <span class="token punctuation">,</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> thread_two <span class="token operator">=</span> threading <span class="token punctuation">.</span> Thread <span class="token punctuation">(</span> target <span class="token operator">=</span> worker <span class="token punctuation">,</span> args <span class="token operator">=</span> <span class="token punctuation">(</span> info <span class="token punctuation">,</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> thread <span class="token punctuation">.</span> start <span class="token punctuation">(</span> <span class="token punctuation">)</span> thread_two <span class="token punctuation">.</span> start <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token keyword">while</span> <span class="token boolean">True</span> <span class="token punctuation">:</span> <span class="token keyword">try</span> <span class="token punctuation">:</span> logging <span class="token punctuation">.</span> debug <span class="token punctuation">(</span> <span class="token string">"Checking in from main thread"</span> <span class="token punctuation">)</span> time <span class="token punctuation">.</span> sleep <span class="token punctuation">(</span> <span class="token number">0.75</span> <span class="token punctuation">)</span> <span class="token keyword">except</span> KeyboardInterrupt <span class="token punctuation">:</span> info <span class="token punctuation">[</span> <span class="token string">"stop"</span> <span class="token punctuation">]</span> <span class="token operator">=</span> <span class="token boolean">True</span> logging <span class="token punctuation">.</span> debug <span class="token punctuation">(</span> <span class="token string">'Stopping'</span> <span class="token punctuation">)</span> <span class="token keyword">break</span> thread <span class="token punctuation">.</span> join <span class="token punctuation">(</span> <span class="token punctuation">)</span> thread_two <span class="token punctuation">.</span> join <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">"__main__"</span> <span class="token punctuation">:</span> main <span class="token punctuation">(</span> <span class="token punctuation">)</span> |
Here, you use the threading
module to create two threads. You also create a logging object that is responsible for logging threadName
into stdout. Next, you start both threads and initialize a loop from the main thread continuously. You use the KeyboardInterrupt
to start the operation ^Ctrl + C
from the user.
Try running the above code in the terminal, you will see the output like the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | <span class="token number">0</span> Thread-1 worker thread checking <span class="token keyword">in</span> <span class="token number">1</span> Thread-2 worker thread checking <span class="token keyword">in</span> <span class="token number">1</span> MainThread Checking <span class="token keyword">in</span> from main thread <span class="token number">752</span> MainThread Checking <span class="token keyword">in</span> from main thread <span class="token number">1001</span> Thread-1 worker thread checking <span class="token keyword">in</span> <span class="token number">1001</span> Thread-2 worker thread checking <span class="token keyword">in</span> <span class="token number">1502</span> MainThread Checking <span class="token keyword">in</span> from main thread <span class="token number">2003</span> Thread-1 worker thread checking <span class="token keyword">in</span> <span class="token number">2003</span> Thread-2 worker thread checking <span class="token keyword">in</span> <span class="token number">2253</span> MainThread Checking <span class="token keyword">in</span> from main thread <span class="token number">3005</span> Thread-1 worker thread checking <span class="token keyword">in</span> <span class="token number">3005</span> MainThread Checking <span class="token keyword">in</span> from main thread <span class="token number">3005</span> Thread-2 worker thread checking <span class="token keyword">in</span> |
When each thread runs and then sleep, logging will be printed to the console. Through this example, you can use these concepts in your own code.
Using Event.wait()
The threading
module provides Event()
that you can use as time.sleep()
. However, Event()
has the advantage that it responds better. The reason for this is that when the event is set, the program exits the loop immediately. With time.sleep()
, your code will need to wait for the sleep()
call to finish before the thread can exit.
The reason you want to use wait()
here is because wait()
is non-blocking while time.sleep()
is blocking . What this means is that when you use time.sleep()
, you will block the main thread and it will not continue running while waiting for the sleep()
call to finish. wait()
solves this problem. You can read more about all threading-related things in the documentation of Python.
This is how you add a sleep()
call to Event.wait()
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | <span class="token keyword">import</span> logging <span class="token keyword">import</span> threading <span class="token keyword">def</span> <span class="token function">worker</span> <span class="token punctuation">(</span> event <span class="token punctuation">)</span> <span class="token punctuation">:</span> <span class="token keyword">while</span> <span class="token keyword">not</span> event <span class="token punctuation">.</span> isSet <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span> logging <span class="token punctuation">.</span> debug <span class="token punctuation">(</span> <span class="token string">"worker thread checking in"</span> <span class="token punctuation">)</span> event <span class="token punctuation">.</span> wait <span class="token punctuation">(</span> <span class="token number">1</span> <span class="token punctuation">)</span> <span class="token keyword">def</span> <span class="token function">main</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span> logging <span class="token punctuation">.</span> basicConfig <span class="token punctuation">(</span> level <span class="token operator">=</span> logging <span class="token punctuation">.</span> DEBUG <span class="token punctuation">,</span> <span class="token builtin">format</span> <span class="token operator">=</span> <span class="token string">"%(relativeCreated)6d %(threadName)s %(message)s"</span> <span class="token punctuation">)</span> event <span class="token operator">=</span> threading <span class="token punctuation">.</span> Event <span class="token punctuation">(</span> <span class="token punctuation">)</span> thread <span class="token operator">=</span> threading <span class="token punctuation">.</span> Thread <span class="token punctuation">(</span> target <span class="token operator">=</span> worker <span class="token punctuation">,</span> args <span class="token operator">=</span> <span class="token punctuation">(</span> event <span class="token punctuation">,</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> thread_two <span class="token operator">=</span> threading <span class="token punctuation">.</span> Thread <span class="token punctuation">(</span> target <span class="token operator">=</span> worker <span class="token punctuation">,</span> args <span class="token operator">=</span> <span class="token punctuation">(</span> event <span class="token punctuation">,</span> <span class="token punctuation">)</span> <span class="token punctuation">)</span> thread <span class="token punctuation">.</span> start <span class="token punctuation">(</span> <span class="token punctuation">)</span> thread_two <span class="token punctuation">.</span> start <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token keyword">while</span> <span class="token keyword">not</span> event <span class="token punctuation">.</span> isSet <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token punctuation">:</span> <span class="token keyword">try</span> <span class="token punctuation">:</span> logging <span class="token punctuation">.</span> debug <span class="token punctuation">(</span> <span class="token string">"Checking in from main thread"</span> <span class="token punctuation">)</span> event <span class="token punctuation">.</span> wait <span class="token punctuation">(</span> <span class="token number">0.75</span> <span class="token punctuation">)</span> <span class="token keyword">except</span> KeyboardInterrupt <span class="token punctuation">:</span> event <span class="token punctuation">.</span> <span class="token builtin">set</span> <span class="token punctuation">(</span> <span class="token punctuation">)</span> <span class="token keyword">break</span> <span class="token keyword">if</span> __name__ <span class="token operator">==</span> <span class="token string">"__main__"</span> <span class="token punctuation">:</span> main <span class="token punctuation">(</span> <span class="token punctuation">)</span> |
In this example, you create threading.Event()
and pass it to worker()
. (Recall in the previous example, you passed a dict.) Next, you can set up loops to check if the event
has been set. If not, your code will print a message and wait a bit before checking again. To set up an event, press ^Ctrl + C
Once the event has been set, worker()
will return, the loop will be broken and the program will end.