csv download response   0   458
How to download large csv files in Django


We know how to return CSV file as response in Django. In most of the cases building such CSV doesn't take much time and response is returned immediately. 

However in some exceptional cases, you might require to return a large CSV file in response.  

In such case one the below thing can happen:
- Generating large file may take more time and hence web server or load balancer may drop the connection due to timeout.
- Generating large file may eat up all the memory available to process

To avoid above situation, we can generate and stream the large CSV files in Django. For this we will use StreamingHttpResponse class.


Note: It is recommended to perform the time taking tasks asynchronously. We should not include such tasks in request response cycle. Web worker remain blocked during the whole download process. If you have fewer number of workers and more simultaneous requests, users may not be able to access your application.


Code:

First lets understand the writer method of csv.

The csv.writer() method accepts a file-like object and returns a writer object which converts the user's data into delimited strings on the given file-like object. See the code example below.


import csv

numbers = [
    [1, 2, 3],
    [4, 5, 6]
]

with open('pythoncircle-dot-com.csv', 'w') as f:
    writer = csv.writer(f)
    for row in numbers:
        writer.writerow(row)


writerow method will take care of comma and escaping part and write the row on to file-like object.


If you look at the source code of StreamingHttpResponse class, you will see that its __init__ method accepts the streaming_content as first parameter which should be an iterable of bytestrings. To create streaming_content we need to use modified version of write method.


Here in our streaming-csv code, we will not actually write the row on file but will return it as it is.

For this we need to create a dummy file-like object with write method which will return the value. 

Here is a class with write method.


class DummyFile:
    def write(self, value_to_write):
        return value_to_write


Now create the writer object using object of this DummyFile class. 


writer = csv.writer(DummyFile())


Now pass multiple values to this writer object's writerow method one by one.


rows = [[str(i), str(2*i), str(3*i)] for i in range(50000)]
data = [writer.writerow(row) for row in rows]


Now create and return response (Streaming response to be specific) using this data.


response = StreamingHttpResponse(data, content_type="text/csv")
response['Content-Disposition'] = 'attachment; filename="pythoncircle-dot-com.csv"'
return response


Complete code will look something like below.

view.py file:

import csv
from django.http import StreamingHttpResponse


class DummyFile:
    def write(self, value_to_write):
        return value_to_write


def large_csv(request):
    rows = ([str(i), str(2 * i), str(3 * i)] for i in range(555555))
    writer = csv.writer(DummyFile())
    data = [writer.writerow(row) for row in rows]
    response = StreamingHttpResponse(data, content_type="text/csv")
    response['Content-Disposition'] = 'attachment; filename="pythoncircle-dot-com.csv"'
    return response


Create appropriate entry in urls.py file to trigger large_csv view. Once you hit the url in browser, file will start downloading.


Host your Django Apps for free.


csv download response   0   458

Related Articles:
Generating and Returning PDF as response in Django
Creating PDF documents in Python and Django, Returning PDF as response in Django, Generating PDF from HTML template in Django, Using pdfkit and wkhtmltopdf in python and django...
How to upload and process the Excel file in Django
How to upload and process the content of excel file in django without storing the file on server. Process Excel file content in Django. Uploading and reading Excel file content in Django 2.0 without storing it on server....
How to download data as CSV and Excel file in Django
How to download excel file in django, download data csv and excel file in django, excel file in python, csv and excel file in django python, download excel python-Django...

0 thoughts on 'How To Download Large Csv Files In Django'
Leave a comment:


*All Fields are mandatory. **Email Id will not be published publicly.


SUBSCRIBE
Please subscribe to get the latest articles in your mailbox.


Recent Posts:






© pythoncircle.com 2018-2019
Contact Us: code108labs [at] gmail.com
Address: 3747 Smithfield Avenue, Houston, Texas