Posts tagged by python
Posted on October 11, 2015 by: Rob van der Linde
Filed under:
Go
Tags:
python,
django,
cornice,
go,
colander,
pyramid
This post isn't really about the actual switch from Python to Go (or maybe a little), but mostly it's just an announcement that I have, also I haven't posted anything to my blog for years, which isn't in the best state at the moment and is due for a complete rewrite.
For the past few months now, I have mostly been using the Go programming language for new projects. I had tried using Go a few times in the past and really loved it, got my feet wet and tried various web frameworks, but didn't quite stick to it long enough to go through with it, eventually switching back to Python each time. That has all changed now, as I have started using Go almost exclusively now for new projects, although I still use mostly Python at work which should keep my Python skills sharp 🙂
So what happened to my old Django CMS (rvcms) then? The problem with the old Django CMS is that I seem to be constantly fighting to keep up with Django upgrades and never quite getting there before the next version comes out, and so on. It isn't entirely Django's fault, though it's 6-monthly release cycles can be a bit annoying, at least they do provide LTS releases now which helps a lot. I think the problem was mostly to do with the libraries that I had chosen getting abandoned, so that I had to choose all new libraries and rewrite large parts of my code (at least several times now) to do so, making the upgrade to new versions of Django way too challenging and I just end up losing interest.
The other problem I had with Django, is that my choice to write the CMS admin interface on top of the Django admin interface was wrong (and is actually considered to be a bit of an anti-pattern that is used way too often in Django). Several libraries exist like Grappelli to extend the Django interface, but they may not be compatible with other libraries that also extend the Django interface but in a different way, leading to compatibility problems between libraries. The other issue you get when you start extending the Django interface too much, is that it makes it harder to upgrade to new versions of Django and you often end up having to override internal admin templates.
I tried Pyramid for a bit, Pyramid is great and so is SQLAlchemy, a library for which I have the utmost respect. But unfortunately there isn't a huge community around Pyramid anymore, and Flask seems to be a lot more popular these days. I also had issues with some libraries I chose (colander, cornice) for various reasons. My issue with Colander was that it "wasn't the right tool" for serializing to JSON unless you subclassed every field to "fix" the library, why???, maybe I would have been better off with Marshmallow for serializing to JSON.
My issue with Cornice, is a problem I see with API libraries in general in Python, is that they are made for specific frameworks only. Cornice is made for Pyramid, so you can't use it with Flask. So if I want to switch to Flask, I am screwed and have to rewrite my entire API layer. Likewise, TastyPie and Django Rest Framework are designed for Django only, and Flask-Restful is designed for Flask only.
Why can't there just be a "generic" API library that is framework-agnostic? In Go, libraries often are framework-agnostic, because there are so many frameworks to choose from: Gin, Negroni, Martini, Goji, Ace, Beego, Revel, just to name a few, or some people prefer to use no framework at all and just use the standard library which is really good in Go.
Other than that, I want to start using all the cool new features like websockets, http2 and two way communication, server push, Python can do all these if you run your web application in async mode, but it just doesn't do these as well as Go does, which is designed from ground up to be a modern concurrent language.
I will continue fixing rvcms, as I still have to support it for several older websites, but going forward I will probably continue to use Go exclusively for new projects.
Posted on August 28, 2011 by: Rob van der Linde
Filed under:
Events,
Python
Tags:
pycon,
python,
event
I just came back from the last day of Kiwi Pycon 2011. I really enjoyed the event, the talks have been really good and there were a lot of people to meet, there were over 150 people at the event in total. I have taken some photos, located in my all new Django gallery, although there are not a lot of them and they aren't really that good.
More photos are available on Flickr from other people, and you may also want to check out #kiwipycon2011 on twitter. The next Kiwi Pycon is going to be in Dunedin in 2012, hopefully I can make it there next year.
Posted on August 14, 2011 by: Rob van der Linde
Filed under:
Events,
Python
Tags:
python,
event,
weta,
datacenter
I got an email Friday morning, that I had been successful in the draw for a tour of the Weta Digital datacenter, by registering early for the Kiwi Pycon 2011 conference. There are only 8 places for this tour, and were drawn from the first 100 early registrations.
I am very excited about going to this event, it is the day before the Kiwi Pycon conference starts. The email states that the Weta Digital datacenter is often cited as the largest collection of data processing power in New Zealand, and Python is at the heart of the operation, how cool is that!
I am really looking forward to this event, as well as the Kiwi Pycon, which is only two weeks away now.
Posted on August 11, 2011 by: Rob van der Linde
Filed under:
Django,
Python
Tags:
latex,
pdf,
python,
django
In the last week, I have been learning how to use LaTeX, a markup style language used to create professional documents. I have been using Texmaker on Linux, an excellent cross-platform LaTeX editor. I thought it would be pretty cool to generate PDF documents from a Django view using the pdflatex command line app, and using the Django template engine for adding dynamic content to .tex files before rendering them to PDF.
Here is the code I wrote that does this, it renders the LaTeX template to a string first. It then uses tempfile.mkstemp() to generate a unique temporary file name for use for the generated PDF file, mkstemp returns both the filename and a file descriptor to the temporary file. When we load the contents of the PDF file into a string, we should use os.fdopen() with this file descriptor, rather than just use open(), which is the proper way to work with temporary files.
When we run pdflatex, we pipe the rendered .tex template to stdin, so we don't have to save it to disk. Unfortunately, pdflatex cannot return a generated PDF over stdout the same way and it only saves a PDF to disk, which is why we have to use the temporary file method.
After the PDF is rendered, we load ithe PDF file and return it as an attachment over the response. We then delete any temporary files created, including the PDF file.
import os
from subprocess import Popen, PIPE
from tempfile import mkstemp
from django.http import HttpResponse, Http404
from django.template.loader import render_to_string
from django.template import RequestContext
def render_latex(request, template, dictionary, filename):
# render latex template and vars to a string
latex = render_to_string(template, dictionary, context_instance=None)
# create a unique temorary filename
fd, path = mkstemp(prefix="latex_", suffix=".pdf")
folder, fname = os.path.split(path)
jobname, ext = os.path.splitext(fname) # jobname is just the filename without .pdf, it's what pdflatex uses
# for the TOC to be built, pdflatex must be run twice, on the second run it will generate a .toc file
for i in range(2):
# start pdflatex, we can send the tex file from stdin, but the output file can only be saved to disk, not piped to stdout unfortunately/
process = Popen(["pdflatex", "-output-directory", folder, "-jobname", jobname], stdin=PIPE, stdout=PIPE) # piping stdout suppresses output messages
process.communicate(latex)
# open the temporary pdf file for reading.
try:
pdf = os.fdopen(fd, "rb")
output = pdf.read()
pdf.close()
except OSError:
raise Http404("Error generating PDF file") # maybe we should use an http 500 here, 404 makes no sense
# generate the response with pdf attachment
response = HttpResponse(mimetype="application/pdf")
response["Content-Disposition"] = "attachment; filename=" + filename
response.write(output)
# delete the pdf from temp directory, and other generated files ending on .aux and .log
for ext in (".pdf", ".aux", ".log", ".toc", ".lof", ".lot", ".synctex.gz"):
try:
os.remove(os.path.join(folder, jobname) + ext)
except OSError:
pass
# return the response
return response
#### Actual Usage ####
def someview(request):
return render_latex(request, "reports/test.tex", {"foo": "bar"}, filename="latex_test.pdf")