Iterator and Generators in Python: Explained with example

Python generator is a simple way of creating iterator.


There is a lot of overhead in building an iterator in python. We have to implement a class with __iter__() and __next__() method, keep track of internal states, raise StopIteration when there was no values to be returned etc.



What is an iterator:

Iterator in Python is an object that can be iterated upon. An object which will return data, one element at a time. Iterator in python is any python type that can be used with a for in loop.

Python lists, tuples, dicts and sets are all examples of inbuilt iterators.


Python iterator object must implement two special methods,  __iter__() and __next__()The __iter__() method returns the iterator object itself. We use the next() function to manually iterate through all the items of an iterator. When we reach the end and there is no more data to be returned, it will raise StopIteration.


Example:


# define a list
my_list = [4, 7, 0, 3]
# get an iterator using iter()
my_iter = iter(my_list)
# iterate through it using next()
# prints 4
print(next(my_iter))
# prints 7
print(next(my_iter))
# next(obj) is same as obj.__next__()
# prints 0
print(my_iter.__next__())
# prints 3
print(my_iter.__next__())
# This will raise error, no items left
next(my_iter)


In the next example we will implement a function which give us next power of 2 in each iteration. Power exponent starts from zero up to a user set number.
 

class PowTwo:
"""Class to implement an iterator of powers of two"""
def __init__(self, max = 0):
self.max = max
def __iter__(self): self.n = 0
return self
def __next__(self):
if self.n <= self.max:
result = 2 ** self.n
self.n += 1
return result
else:
raise StopIteration
obj  = PowTwo(4)
iter = iter(obj)
print(next(iter))  # print 1
print(next(iter))  # print 2
print(next(iter))  # print 4


All the overhead we mentioned above are automatically handled by generators in Python. Simply speaking, a generator is a function that returns an object (iterator) which we can iterate over, one value at a time.


Creating Generator:

It is fairly simple to create a generator in Python. It is as easy as defining a normal function with yield statement instead of a return statement. If a function contains at least one yield statement, it becomes a generator function.

Both yield and return will return some value from a function. The difference is that, while a return statement terminates a function entirely, yield statement pauses the function saving all its states and later continues from there on successive calls.

Example:

We have a generator function named my_gen() with several yield statements.
 

# Generator function
def my_gen():
n = 1
print('print 1')
# Generator function contains yield statements
yield n
n += 1
   print('print 2')
yield n
n += 1
print('print 3 - last')
yield n
# create iterator object
obj = my_gen()
# iterate through the items using next() function
print(next(obj))

 

 

Normally, generator functions are implemented with a loop having a suitable terminating condition. 

Example:

def rev_str(my_str):
length = len(my_str)
for i in range(length-1, -1, -1):
yield my_str[i]


for char in rev_str("hello"):
print(char)

 

 

 

Reasons to use Generators:


- Easy to Implement
Since generators keep track of details automatically, they can be implemented in a clear and concise way as compared to their iterator class counterpart.

- Memory Efficient
A normal function to return a sequence will create the entire sequence in memory before returning the result. This is an overkill if the number of items in the sequence is very large. Generator implementation of such sequence is memory friendly and is preferred since it only produces one item at a time.
 
- Represent Infinite Stream
Generators are excellent medium to represent an infinite stream of data. Infinite streams cannot be stored in memory and since generators produce only one item at a time, it can represent infinite stream of data.



Virtual Environment in Python - A Pocket Guide

In almost every article, we recommended the use of virtual environment for developing any Python or Django project.

In this article, we will briefly cover the virtual environment in python, installation and usage.


What is a Virtual Environment:

Virtual environment is an isolated python environment which can be created using virtualenv python tool. This virtual environment contains all the packages that a python package would require. 

Python project running in virtual environment does not use the system wide installed python package.  



Installing Virtual Environment:

We can install virtual environment using pip.

pip install virtualenv


You can confirm the installation by running below command.

virtualenv --version
rana@Brahma ~$ virtualenv --version
15.1.0


Using virtual environment:

Create a virtual environment:

virtualenv project_1_venv


This will create a virtual environment with name project_1_env  and install python binaries in this folder. This will use the system's default python' version.

To create virtual environment with specific python version use below command.

virtualenv -p /usr/bin/python2.7 venv2
or

virtualenv -p /usr/bin/python3 venv3


To work inside a virtual environment, we need to activate it.

source virtual_env_folder/bin/activate


Now you may install python packages inside this isolated environment using pip install command.

Once you are done working with your project, you may deactivate the virtual environment by deactivate  command.   



Deleting virtual environment:

To delete a virtual environment simply remove the directory.

rm -rf virtual_env_directory


Freezing the requirements:

When you deploy your project on any other server like PythonAnyWhere, you need to replicate the virtual environment.

You can not simply copy the folder as it will break the things. Also this is not a good idea as the size of the folder will be large.

We will first list all the packages installed in virtual environment and dump in a file.

pip freeze > requirement.txt


Now commit this file along with your code. On production server, clone the code repository and create a virtual environment. Once virtual environment is activated, install all the dependencies using the pip install command below.

pip install -r requirement.txt


This will create the exact copy of virtual environment you were working on development server.   


Pipenv:

Pipenv is a dependency manager for python projects. It is similar to Symfony's composer and node.js's npm.

Its a higher lever tool than pip. We recommend to use pipenv for managing your virtual environments.

With pipenv, we need not to manage requirement.txt file separately.

As per pienv readthedocs:

It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever–important Pipfile.lock, which is used to produce deterministic builds.


Install pipenv using pip.

pip install pipenv



Creating virtual environment using pipenv:

Go to your project directory. Install python packages using pipenv command.

rana@Brahma: ~$ cd  /tmp
rana@Brahma: tmp$ mkdir django_project_1
rana@Brahma: tmp$ cd django_project_1/
rana@Brahma: django_project_1$ pipenv install django requests
Creating a virtualenv for this project…
?Using base prefix '/usr'
New python executable in /home/rana/.local/share/virtualenvs/django_project_1-NloTdiFm/bin/python3
Also creating executable in /home/rana/.local/share/virtualenvs/django_project_1-NloTdiFm/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/rana/.local/share/virtualenvs/django_project_1-NloTdiFm
Creating a Pipfile for this project…
Installing django…
Collecting django
  Using cached Django-2.0-py3-none-any.whl
Collecting pytz (from django)
  Using cached pytz-2017.3-py2.py3-none-any.whl
Installing collected packages: pytz, django
Successfully installed django-2.0 pytz-2017.3

Adding django to Pipfile's [packages]…
Installing requests…
Collecting requests
  Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Using cached certifi-2017.11.5-py2.py3-none-any.whl
Collecting idna<2.7,>=2.5 (from requests)
  Using cached idna-2.6-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests)
  Using cached urllib3-1.22-py2.py3-none-any.whl
Installing collected packages: chardet, certifi, idna, urllib3, requests
Successfully installed certifi-2017.11.5 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22

Adding requests to Pipfile's [packages]…
  PS: You have excellent taste! ? 


Designing custom 404 and 500 error pages in Django

Update 1: Please refer this updated article for Django 2.0 and source code.  


It happens very frequently that a visitor on your website typed a wrong URL or the page user is looking for no longer exists.

What do you do to handle such cases. You have three options.

  • Redirect the visitor to home page, silently.
  • Show a boring 404 page and then ask them to click on a link.
  • Create your own funny/awesome/informative custom 404 error page.


In this article we will discuss the third option.


How to create your own custom 404 error page in Django:

A custom 404 error page can serve multiple other purposes apart from just telling the user that the link you visited is not correct.

You can ask user to subscribe or sign-up. Or you may show some funny stuff.


custom 404 error page in django


Default 404 error page in django is quite boring. Also creating a custom 404 page in django is very simple.  

So lets see how to create custom 404 error page in django.

  • In urls.py  file of your project, import handler404  and handler505 .
  • In urls.py  file import views from your app.
  • After urlpatterns assign the views to handler404 and handler500.

    from myapp import views as myapp_views
    from django.conf.urls import handler404, handler500
    
    
    urlpatterns = [
        url(r'^admin/', admin.site.urls),
        url(r'^myapp/', include('myapp.urls', namespace='myapp')),
    ] 
    
    handler404 = myapp_views.error_404
    handler500 = myapp_views.error_500


  • In settings.py file, set DEBUG=False  and ALLOWED_HOST=["*"] . Custom 404 and 500 pages works only when Debug is set to false and there is appropriate entry in allowed_hosts .
  • Create a new view function in your views file. Return the rendered 404 html file from this view. Make sure name of view function created in this file matches the name used in urls file above.

    from django.shortcuts import render
    
    
    def error_404(request):
            data = {}
            return render(request,'myapp/error_404.html', data)
    
    def error_500(request):
            data = {}
            return render(request,'myapp/error_500.html', data)


  • Now create an HTML template in your app's template directory myapp/templates/myapp/error_404.html . Place whatever content you want to put there. You can create a subscribe form or sign up form.
  • Now type an incorrect url and you will be shown your custom 404 page. In case of any internal server error like syntax error in some template, custom 500 page will be shown.


  Comment in case of any query.


SUBSCRIBE
Please subscribe to get the latest articles in your mailbox.


Recent Posts:






© pythoncircle.com 2018-2019
Contact Us: code108labs [at] gmail.com
Address: 3747 Smithfield Avenue, Houston, Texas