Opportunities to actively engage and learn are vital to our work at Poplin Data so it was great to attend the recent PyCon AU 2019 with other members of the engineering team.
The event brought together professionals, students and enthusiast developers to discuss the many joys and challenges of programming in Python, which is literally the lingua franca for most of what we build.
For Poplin Data, Python is seen as the glue that holds many things together. All of our integrations use it, while other projects we leverage like DBT are built with it. We also leverage the PyData ecosystem a lot so for us. PyCon Au 2019 was always an important date on our calendar.
The schedule is inevitably crowded and we thought we’d use this week’s blog to pick out some of the highlights.
That way, those of you who weren’t able to make it can catch up by watching the videos, most of which are already up!
Here are some of the top-level insights we thought were pretty cool:
Yaakov Smith had an interesting take on the state of authentication and authorisation in the digital age, particularly with respect to the vulnerabilities and potential pitfalls of digital driver licenses, due to be trialled in NSW late this year.
Identity and trust are crucial in everyday life but, as our previous blog showed they’re also important business considerations for Poplin’s clients.
That’s true when it comes to managing relationships with customers and, increasingly, their obligations under the law.
Speeding things up
Python is great for solving many of the data-related issues our clients face all the time but growing workloads are running up against the limits of CPU performance.
In his talk Mark Harris outlined the latest developments of cuDF, a CUDA based framework that provides similar interfaces to pandas dataframes whilst leveraging NVIDIA GPUs with some impressive benchmarks across both data loading and transformations.
Python provides some useful tools for identifying bugs without going down the traditional route of actually executing code. Jon Manning’s talk on this topic, Let’s Build a Symbolic Analyser And Automatically Find Bugs, was unexpectedly awesome.
Jon went right down into the cool stuff like using Python bytecode disassembly to determine function branches and find unreachable code, and other neat static analysis tricks. His enthusiasm helped keep what could have been a really dry talk entertaining and educational.
Bug out bag of tricks
On a similar theme, Nina Zakharenko’s talk showed how debugging Python can be much simpler than writing print statements and re-running your code for the 100th time.
Historically it’s been about writing print statements and re-running the code figure out code flow but Nina explained the inefficiencies of this and how debuggers like pdb and others can make the process simpler.
Keeping it simple
Avoiding unnecessary complications is central to our philosophy at Poplin and it’s something we feel resonates pretty well within the broader Python community, the so-called Zen of Python.
One example is the way we try to avoid introducing technology for its own sake. That’s because a lot of the problems we deal with are human ones and merely adding more technology won’t necessarily solve them. Just because it exists, doesn’t mean we should use it.
There are a lot of quirks in Python which can be used to achieve results that are not quite expected. These may be beneficial in some cases but almost always make the code unreadable, which goes against the Zen of Python.
Metaclasses are awesome
Mark Smith’s talk on Python metaclasses took a deep dive into this sometimes intimidating topic.
It proved very interesting and helped demystify some of the concepts, explaining the core Python types and how attribute lookup works in an inheritance model.
He also took a close look at how metaclasses work and fit into the class inheritance hierarchy.
Here are the team’s favourites among the remaining talks from Friday to Sunday.
Just Add Await: Retrofitting Async into Django – Andrew Godwin
This was one of the few double-length sessions of the day but was easily worth it. An in-depth look at the challenges of trying to change the architecture of such a large project while also maintaining backwards-compatibility. Andrew was super knowledgeable on the topic, and also entertaining. He gave a good overview of asyncio, to boot.
The Unspeakable Horror of Discovering You Didn’t Write Tests – Melanie Crutchfield
Securing your AWS Identity Management pipeline with PyTest – Sean Johnson
Creating Lasting Change – Aurynn Shaw
Saturday Lightning Talks – Various
Pretty vector graphics – Playing with SVG in Python – Amanda J Hogan
You Might Not Need Gender – Opal Symes
Environment Variables: how software impacts climate change – Merrin Macleod
Insights into Social Media Data using Entropy Theory – Mars Geldard
Sunday Lightning Talks – Various
The New COBOL – Benno Rice
Overall it was a great and enlightening event. The event is usually held in two year blocks in the same city – this was Sydney’s second year hosting the event. We’ll definitely be travelling interstate next year.