How to download large csv files in Django

We know how to return CSV file as response in Django. In most of the cases building such CSV doesn't take much time and response is returned immediately. 

However in some exceptional cases, you might require to return a large CSV file in response.  

In such case one the below thing can happen:
- Generating large file may take more time and hence web server or load balancer may drop the connection due to timeout.
- Generating large file may eat up all the memory available to process

To avoid above situation, we can generate and stream the large CSV files in Django. For this we will use StreamingHttpResponse class.

Note: It is recommended to perform the time taking tasks asynchronously. We should not include such tasks in request response cycle. Web worker remain blocked during the whole download process. If you have fewer number of workers and more simultaneous requests, users may not be able to access your application.


First lets understand the writer method of csv.

The csv.writer() method accepts a file-like object and returns a writer object which converts the user's data into delimited strings on the given file-like object. See the code example below.

import csv

numbers = [
    [1, 2, 3],
    [4, 5, 6]

with open('pythoncircle-dot-com.csv', 'w') as f:
    writer = csv.writer(f)
    for row in numbers:

writerow method will take care of comma and escaping part and write the row on to file-like object.

If you look at the source code of StreamingHttpResponse class, you will see that its __init__ method accepts the streaming_content as first parameter which should be an iterable of bytestrings. To create streaming_content we need to use modified version of write method.

Here in our streaming-csv code, we will not actually write the row on file but will return it as it is.

For this we need to create a dummy file-like object with write method which will return the value. 

Here is a class with write method.

class DummyFile:
    def write(self, value_to_write):
        return value_to_write

Now create the writer object using object of this DummyFile class. 

writer = csv.writer(DummyFile())

Now pass multiple values to this writer object's writerow method one by one.

rows = [[str(i), str(2*i), str(3*i)] for i in range(50000)]
data = [writer.writerow(row) for row in rows]

Now create and return response (Streaming response to be specific) using this data.

response = StreamingHttpResponse(data, content_type="text/csv")
response['Content-Disposition'] = 'attachment; filename="pythoncircle-dot-com.csv"'
return response

Complete code will look something like below. file:

import csv
from django.http import StreamingHttpResponse

class DummyFile:
    def write(self, value_to_write):
        return value_to_write

def large_csv(request):
    rows = ([str(i), str(2 * i), str(3 * i)] for i in range(555555))
    writer = csv.writer(DummyFile())
    data = [writer.writerow(row) for row in rows]
    response = StreamingHttpResponse(data, content_type="text/csv")
    response['Content-Disposition'] = 'attachment; filename="pythoncircle-dot-com.csv"'
    return response

Create appropriate entry in file to trigger large_csv view. Once you hit the url in browser, file will start downloading.

Host your Django Apps for free.

Related Articles:
How to use AJAX with Django
How to use AJAX in Django projects?, Checking username availability without submitting form, Making AJAX calls from Django code, loading data without refreshig page in django templates, AJAX and Django,...
Logging databases changes in Django Application
Logging databases changes in Django Application, Using Django signals to log the changes made to models, Model auditing using Django signals, Creating signals to track and log changes made to database tables, tracking model changes in Django, Change in table Django, Database auditing Django, tracking Django database table editing...
Solving Django Error: TemplateDoesNotExist at /app_name/
How to solve TemplateDoesNotExist error in Django projects, reason for TemplateDoesNotExist error in Django, fixing TemplateDoesNotExist error in Django...
Generating and Returning PDF as response in Django
Creating PDF documents in Python and Django, Returning PDF as response in Django, Generating PDF from HTML template in Django, Using pdfkit and wkhtmltopdf...

0 thoughts on 'How To Download Large Csv Files In Django'
Leave a comment:

*All Fields are mandatory. **Email Id will not be published publicly.

Please subscribe to get the latest articles in your mailbox.

Recent Posts:

© 2018-2019
Contact Us: code108labs [at]
Address: 3747 Smithfield Avenue, Houston, Texas